[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:04:18.662989 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:44067
I20260709 13:04:18.664626 20746 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:04:18.665762 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:18.684418 20753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:18.684449 20752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:18.685353 20755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:19.921370 20754 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 13:04:19.921543 20746 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 13:04:19.926225 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:19.926515 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:19.926754 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602259926724 us; error 0 us; skew 500 ppm
I20260709 13:04:19.927610 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:19.935002 20746 webserver.cc:533] Webserver started at http://127.20.66.190:41339/ using document root <none> and password file <none>
I20260709 13:04:19.936039 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:19.936309 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:19.936802 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:19.944617 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "bbdccef5b87c4890a2912bdc40c679fa"
format_stamp: "Formatted at 2026-07-09 13:04:19 on dist-test-slave-rgrd"
I20260709 13:04:19.954125 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:04:19.960412 20761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:19.962558 20746 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.001s
I20260709 13:04:19.963142 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "bbdccef5b87c4890a2912bdc40c679fa"
format_stamp: "Formatted at 2026-07-09 13:04:19 on dist-test-slave-rgrd"
I20260709 13:04:19.963704 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:20.022593 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:20.024206 20746 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:04:20.024706 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:20.103097 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:44067
I20260709 13:04:20.103169 20812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:44067 every 8 connection(s)
I20260709 13:04:20.112237 20813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:20.138645 20813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa: Bootstrap starting.
I20260709 13:04:20.145308 20813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:20.147334 20813 log.cc:826] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa: Log is configured to *not* fsync() on all Append() calls
I20260709 13:04:20.152899 20813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa: No bootstrap required, opened a new log
I20260709 13:04:20.175606 20813 raft_consensus.cc:359] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdccef5b87c4890a2912bdc40c679fa" member_type: VOTER }
I20260709 13:04:20.176214 20813 raft_consensus.cc:385] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:20.176425 20813 raft_consensus.cc:740] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbdccef5b87c4890a2912bdc40c679fa, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.177416 20813 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdccef5b87c4890a2912bdc40c679fa" member_type: VOTER }
I20260709 13:04:20.177927 20813 raft_consensus.cc:399] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:20.178174 20813 raft_consensus.cc:493] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:20.178467 20813 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:20.182986 20813 raft_consensus.cc:515] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdccef5b87c4890a2912bdc40c679fa" member_type: VOTER }
I20260709 13:04:20.183759 20813 leader_election.cc:304] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbdccef5b87c4890a2912bdc40c679fa; no voters: 
I20260709 13:04:20.185599 20813 leader_election.cc:290] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:20.186189 20816 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:20.189499 20816 raft_consensus.cc:697] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 1 LEADER]: Becoming Leader. State: Replica: bbdccef5b87c4890a2912bdc40c679fa, State: Running, Role: LEADER
I20260709 13:04:20.190553 20816 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdccef5b87c4890a2912bdc40c679fa" member_type: VOTER }
I20260709 13:04:20.191689 20813 sys_catalog.cc:565] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:20.204079 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbdccef5b87c4890a2912bdc40c679fa. Latest consensus state: current_term: 1 leader_uuid: "bbdccef5b87c4890a2912bdc40c679fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdccef5b87c4890a2912bdc40c679fa" member_type: VOTER } }
I20260709 13:04:20.204109 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbdccef5b87c4890a2912bdc40c679fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdccef5b87c4890a2912bdc40c679fa" member_type: VOTER } }
I20260709 13:04:20.205058 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:20.205065 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:20.208214 20826 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:20.222450 20826 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:20.224800 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:20.240464 20826 catalog_manager.cc:1383] Generated new cluster ID: d7db21a3855241babcb0016224a16fb2
I20260709 13:04:20.240758 20826 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:20.261655 20826 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:20.264074 20826 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:20.291702 20826 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa: Generated new TSK 0
I20260709 13:04:20.292734 20826 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:20.357688 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:20.367463 20835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.370265 20836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.376789 20838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:20.378147 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:20.379130 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:20.379418 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:20.379618 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602260379602 us; error 0 us; skew 500 ppm
I20260709 13:04:20.380452 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:20.384832 20746 webserver.cc:533] Webserver started at http://127.20.66.129:38195/ using document root <none> and password file <none>
I20260709 13:04:20.385881 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:20.386173 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:20.386556 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:20.388127 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "be6289a3181c4df88c501af7dca2509a"
format_stamp: "Formatted at 2026-07-09 13:04:20 on dist-test-slave-rgrd"
I20260709 13:04:20.395589 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.004s
I20260709 13:04:20.401378 20843 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.402426 20746 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:04:20.403009 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "be6289a3181c4df88c501af7dca2509a"
format_stamp: "Formatted at 2026-07-09 13:04:20 on dist-test-slave-rgrd"
I20260709 13:04:20.403576 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:20.447736 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:20.449622 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:20.452494 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:20.458191 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:20.458559 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.458875 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:20.459113 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.555699 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:34005
I20260709 13:04:20.555761 20905 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:34005 every 8 connection(s)
I20260709 13:04:20.565358 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:20.576387 20910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.578217 20911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.584697 20913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:20.585609 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:20.587060 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:20.587356 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:20.587565 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602260587549 us; error 0 us; skew 500 ppm
I20260709 13:04:20.588433 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:20.591766 20906 heartbeater.cc:344] Connected to a master server at 127.20.66.190:44067
I20260709 13:04:20.592279 20906 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:20.593510 20906 heartbeater.cc:507] Master 127.20.66.190:44067 requested a full tablet report, sending...
I20260709 13:04:20.593668 20746 webserver.cc:533] Webserver started at http://127.20.66.130:45853/ using document root <none> and password file <none>
I20260709 13:04:20.594390 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:20.594641 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:20.595010 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:20.596818 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "efe3327107f74e5381dbfd4784493422"
format_stamp: "Formatted at 2026-07-09 13:04:20 on dist-test-slave-rgrd"
I20260709 13:04:20.597509 20778 ts_manager.cc:194] Registered new tserver with Master: be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005)
I20260709 13:04:20.599933 20778 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35820
I20260709 13:04:20.604789 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:04:20.610096 20918 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.611105 20746 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:04:20.611451 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "efe3327107f74e5381dbfd4784493422"
format_stamp: "Formatted at 2026-07-09 13:04:20 on dist-test-slave-rgrd"
I20260709 13:04:20.611822 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:20.633931 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:20.635742 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:20.638410 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:20.642304 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:20.642592 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.642885 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:20.643095 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.733875 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:34925
I20260709 13:04:20.733964 20980 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:34925 every 8 connection(s)
I20260709 13:04:20.741762 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:20.751840 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.754958 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:20.755916 20981 heartbeater.cc:344] Connected to a master server at 127.20.66.190:44067
I20260709 13:04:20.756337 20981 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:20.757475 20981 heartbeater.cc:507] Master 127.20.66.190:44067 requested a full tablet report, sending...
W20260709 13:04:20.759531 20987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:20.760181 20778 ts_manager.cc:194] Registered new tserver with Master: efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:20.762230 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:20.762253 20778 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35830
I20260709 13:04:20.763285 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:20.763588 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:20.763778 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602260763760 us; error 0 us; skew 500 ppm
I20260709 13:04:20.764539 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:20.767550 20746 webserver.cc:533] Webserver started at http://127.20.66.131:34417/ using document root <none> and password file <none>
I20260709 13:04:20.768213 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:20.768451 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:20.768764 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:20.770169 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "093007c6361840548ef627624cd5ecd4"
format_stamp: "Formatted at 2026-07-09 13:04:20 on dist-test-slave-rgrd"
I20260709 13:04:20.776232 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:04:20.780848 20992 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.781932 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:20.782289 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "093007c6361840548ef627624cd5ecd4"
format_stamp: "Formatted at 2026-07-09 13:04:20 on dist-test-slave-rgrd"
I20260709 13:04:20.782662 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:20.828902 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:20.830320 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:20.831853 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:20.834277 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:20.834550 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.834831 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:20.835031 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.936838 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:44945
I20260709 13:04:20.937389 21054 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:44945 every 8 connection(s)
I20260709 13:04:20.953584 21055 heartbeater.cc:344] Connected to a master server at 127.20.66.190:44067
I20260709 13:04:20.954087 21055 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:20.954921 21055 heartbeater.cc:507] Master 127.20.66.190:44067 requested a full tablet report, sending...
I20260709 13:04:20.957118 20778 ts_manager.cc:194] Registered new tserver with Master: 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:20.957223 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015338579s
I20260709 13:04:20.958935 20778 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35844
I20260709 13:04:21.012730 20778 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35852:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:21.016585 20778 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:21.132364 20871 tablet_service.cc:1511] Processing CreateTablet for tablet 3eaf33c724214480b0189e88cda20dae (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:21.134169 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eaf33c724214480b0189e88cda20dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.132498 20870 tablet_service.cc:1511] Processing CreateTablet for tablet 4b38e78a5bfa4f208c7ac504922476ee (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:21.134487 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 24d0eb363cde4b08b4ccbd3ca2a4181b (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:21.137177 20870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b38e78a5bfa4f208c7ac504922476ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.138240 20868 tablet_service.cc:1511] Processing CreateTablet for tablet aaf253c68b8f4f5ebd7cc66a6257c974 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:21.139164 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24d0eb363cde4b08b4ccbd3ca2a4181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.140920 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf253c68b8f4f5ebd7cc66a6257c974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.151685 20867 tablet_service.cc:1511] Processing CreateTablet for tablet 189df0a1bbfc41538170cf1c9c4bed19 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:21.154590 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189df0a1bbfc41538170cf1c9c4bed19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.170004 21019 tablet_service.cc:1511] Processing CreateTablet for tablet 4b38e78a5bfa4f208c7ac504922476ee (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:21.172245 21017 tablet_service.cc:1511] Processing CreateTablet for tablet aaf253c68b8f4f5ebd7cc66a6257c974 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:21.166879 21018 tablet_service.cc:1511] Processing CreateTablet for tablet 24d0eb363cde4b08b4ccbd3ca2a4181b (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:21.173272 20866 tablet_service.cc:1511] Processing CreateTablet for tablet ab44179cb3654a38a29d9b4aacfd346f (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:21.174279 21016 tablet_service.cc:1511] Processing CreateTablet for tablet 189df0a1bbfc41538170cf1c9c4bed19 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:21.177405 21017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf253c68b8f4f5ebd7cc66a6257c974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.174917 21020 tablet_service.cc:1511] Processing CreateTablet for tablet 3eaf33c724214480b0189e88cda20dae (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:21.177665 20866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab44179cb3654a38a29d9b4aacfd346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.180310 21018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24d0eb363cde4b08b4ccbd3ca2a4181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.184551 21016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189df0a1bbfc41538170cf1c9c4bed19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.184643 21015 tablet_service.cc:1511] Processing CreateTablet for tablet ab44179cb3654a38a29d9b4aacfd346f (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:21.185997 21020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eaf33c724214480b0189e88cda20dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.187389 21019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b38e78a5bfa4f208c7ac504922476ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.190879 21015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab44179cb3654a38a29d9b4aacfd346f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:21.214857 20825 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:21.219102 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 17494b1be7e34346aec3c408a3005de9 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:21.221953 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17494b1be7e34346aec3c408a3005de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.251718 20943 tablet_service.cc:1511] Processing CreateTablet for tablet aaf253c68b8f4f5ebd7cc66a6257c974 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:21.253268 20944 tablet_service.cc:1511] Processing CreateTablet for tablet 24d0eb363cde4b08b4ccbd3ca2a4181b (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:21.258826 20942 tablet_service.cc:1511] Processing CreateTablet for tablet 189df0a1bbfc41538170cf1c9c4bed19 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:21.260598 20946 tablet_service.cc:1511] Processing CreateTablet for tablet 3eaf33c724214480b0189e88cda20dae (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:21.265269 20941 tablet_service.cc:1511] Processing CreateTablet for tablet ab44179cb3654a38a29d9b4aacfd346f (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:21.265714 20944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24d0eb363cde4b08b4ccbd3ca2a4181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.271279 20940 tablet_service.cc:1511] Processing CreateTablet for tablet 17494b1be7e34346aec3c408a3005de9 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:21.262495 20945 tablet_service.cc:1511] Processing CreateTablet for tablet 4b38e78a5bfa4f208c7ac504922476ee (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:21.275696 21014 tablet_service.cc:1511] Processing CreateTablet for tablet 17494b1be7e34346aec3c408a3005de9 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:21.278189 20942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189df0a1bbfc41538170cf1c9c4bed19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.282518 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17494b1be7e34346aec3c408a3005de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.285959 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 3012666e8acc4b48ac09ba71af060586 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:21.287802 20946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eaf33c724214480b0189e88cda20dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.289352 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab44179cb3654a38a29d9b4aacfd346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.290261 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3012666e8acc4b48ac09ba71af060586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.291102 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17494b1be7e34346aec3c408a3005de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.296708 20945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b38e78a5bfa4f208c7ac504922476ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.306210 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 3012666e8acc4b48ac09ba71af060586 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:21.301575 21073 tablet_bootstrap.cc:492] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.311144 20943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf253c68b8f4f5ebd7cc66a6257c974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.314142 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3012666e8acc4b48ac09ba71af060586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.339783 21019 tablet_service.cc:1511] Processing CreateTablet for tablet 3012666e8acc4b48ac09ba71af060586 (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:21.341789 21019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3012666e8acc4b48ac09ba71af060586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.374846 21016 tablet_service.cc:1511] Processing CreateTablet for tablet 3c50bcc3ac3a4c91b9222e710337a34a (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:21.377195 21016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c50bcc3ac3a4c91b9222e710337a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.379880 20938 tablet_service.cc:1511] Processing CreateTablet for tablet 3c50bcc3ac3a4c91b9222e710337a34a (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:21.382985 20938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c50bcc3ac3a4c91b9222e710337a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.370044 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 3c50bcc3ac3a4c91b9222e710337a34a (DEFAULT_TABLE table=test-workload [id=149d6ec7319c40f38046d8056cc1e435]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:21.401556 21074 tablet_bootstrap.cc:492] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:21.400202 21073 tablet_bootstrap.cc:654] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.419314 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c50bcc3ac3a4c91b9222e710337a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.451037 21074 tablet_bootstrap.cc:654] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.468827 21075 tablet_bootstrap.cc:492] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.488770 21075 tablet_bootstrap.cc:654] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.498077 21073 tablet_bootstrap.cc:492] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.498678 21073 ts_tablet_manager.cc:1403] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.209s	user 0.048s	sys 0.034s
I20260709 13:04:21.501760 21073 raft_consensus.cc:359] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.502573 21073 raft_consensus.cc:385] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.502923 21073 raft_consensus.cc:740] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.503829 21073 consensus_queue.cc:260] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.505426 21074 tablet_bootstrap.cc:492] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:21.506003 21074 ts_tablet_manager.cc:1403] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.021s
I20260709 13:04:21.509135 21074 raft_consensus.cc:359] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.509895 21074 raft_consensus.cc:385] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.510239 21074 raft_consensus.cc:740] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.511071 21074 consensus_queue.cc:260] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.512859 20906 heartbeater.cc:499] Master 127.20.66.190:44067 was elected leader, sending a full tablet report...
I20260709 13:04:21.514026 21073 ts_tablet_manager.cc:1434] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 13:04:21.515123 21073 tablet_bootstrap.cc:492] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.516598 21075 tablet_bootstrap.cc:492] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.517087 21075 ts_tablet_manager.cc:1403] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.006s
I20260709 13:04:21.519773 21055 heartbeater.cc:499] Master 127.20.66.190:44067 was elected leader, sending a full tablet report...
I20260709 13:04:21.520175 21075 raft_consensus.cc:359] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.520869 21074 ts_tablet_manager.cc:1434] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.007s
I20260709 13:04:21.520898 21075 raft_consensus.cc:385] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.521499 21075 raft_consensus.cc:740] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.521914 21074 tablet_bootstrap.cc:492] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:21.522281 21075 consensus_queue.cc:260] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.528920 21074 tablet_bootstrap.cc:654] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.531121 20981 heartbeater.cc:499] Master 127.20.66.190:44067 was elected leader, sending a full tablet report...
I20260709 13:04:21.531246 21073 tablet_bootstrap.cc:654] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.532141 21075 ts_tablet_manager.cc:1434] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.005s
I20260709 13:04:21.533241 21075 tablet_bootstrap.cc:492] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.543398 21074 tablet_bootstrap.cc:492] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:21.543921 21074 ts_tablet_manager.cc:1403] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:04:21.544340 21075 tablet_bootstrap.cc:654] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.547092 21074 raft_consensus.cc:359] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.547886 21074 raft_consensus.cc:385] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.548215 21074 raft_consensus.cc:740] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.551714 21073 tablet_bootstrap.cc:492] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.552158 21073 ts_tablet_manager.cc:1403] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.011s
I20260709 13:04:21.554495 21073 raft_consensus.cc:359] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.555068 21073 raft_consensus.cc:385] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.555320 21073 raft_consensus.cc:740] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.555892 21073 consensus_queue.cc:260] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.556023 21074 consensus_queue.cc:260] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.557879 21073 ts_tablet_manager.cc:1434] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:04:21.558533 21074 ts_tablet_manager.cc:1434] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:04:21.558800 21073 tablet_bootstrap.cc:492] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.559278 21075 tablet_bootstrap.cc:492] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.559528 21074 tablet_bootstrap.cc:492] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:21.559908 21075 ts_tablet_manager.cc:1403] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.003s
I20260709 13:04:21.563172 21075 raft_consensus.cc:359] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.564116 21075 raft_consensus.cc:385] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.564486 21075 raft_consensus.cc:740] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
W20260709 13:04:21.565176 20907 tablet.cc:2406] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:21.565338 21075 consensus_queue.cc:260] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.566265 21073 tablet_bootstrap.cc:654] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.568187 21075 ts_tablet_manager.cc:1434] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:21.569231 21075 tablet_bootstrap.cc:492] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.570937 21074 tablet_bootstrap.cc:654] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.573686 21081 raft_consensus.cc:493] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.574266 21081 raft_consensus.cc:515] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.577682 21081 leader_election.cc:290] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:21.582371 21075 tablet_bootstrap.cc:654] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.596626 21073 tablet_bootstrap.cc:492] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.597194 21073 ts_tablet_manager.cc:1403] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.015s
I20260709 13:04:21.600661 21073 raft_consensus.cc:359] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.602607 21073 raft_consensus.cc:385] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.602952 21073 raft_consensus.cc:740] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.603732 21080 raft_consensus.cc:493] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.604230 21080 raft_consensus.cc:515] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.603700 21073 consensus_queue.cc:260] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.609553 21074 tablet_bootstrap.cc:492] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:21.610049 21074 ts_tablet_manager.cc:1403] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.051s	user 0.005s	sys 0.007s
I20260709 13:04:21.611300 21073 ts_tablet_manager.cc:1434] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:04:21.610952 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:21.612288 21073 tablet_bootstrap.cc:492] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
W20260709 13:04:21.613377 20921 leader_election.cc:343] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.613621 21075 tablet_bootstrap.cc:492] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.614106 21075 ts_tablet_manager.cc:1403] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.007s
I20260709 13:04:21.616402 21080 leader_election.cc:290] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:21.622575 21074 raft_consensus.cc:359] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.623184 21075 raft_consensus.cc:359] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.623600 21074 raft_consensus.cc:385] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.623932 21075 raft_consensus.cc:385] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.624001 21074 raft_consensus.cc:740] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.624337 21075 raft_consensus.cc:740] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.630621 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:21.631282 21073 tablet_bootstrap.cc:654] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:21.632551 20922 leader_election.cc:343] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.633062 20922 leader_election.cc:304] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe3327107f74e5381dbfd4784493422; no voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a
I20260709 13:04:21.633383 21075 consensus_queue.cc:260] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.634104 21081 raft_consensus.cc:2749] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:21.636536 21074 consensus_queue.cc:260] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.641567 21079 raft_consensus.cc:493] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.642211 21079 raft_consensus.cc:515] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.644111 21074 ts_tablet_manager.cc:1434] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.034s	user 0.012s	sys 0.000s
I20260709 13:04:21.645671 21079 leader_election.cc:290] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:21.684728 21075 ts_tablet_manager.cc:1434] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.070s	user 0.023s	sys 0.027s
I20260709 13:04:21.686218 21075 tablet_bootstrap.cc:492] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.684835 21073 tablet_bootstrap.cc:492] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.683145 21074 tablet_bootstrap.cc:492] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:21.688057 21073 ts_tablet_manager.cc:1403] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.076s	user 0.022s	sys 0.023s
I20260709 13:04:21.690244 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:21.690984 20881 raft_consensus.cc:2468] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 0.
I20260709 13:04:21.693184 20996 leader_election.cc:304] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:21.694329 21080 raft_consensus.cc:2804] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:21.694697 21080 raft_consensus.cc:493] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.695243 21080 raft_consensus.cc:3060] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.698217 21073 raft_consensus.cc:359] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.699239 21073 raft_consensus.cc:385] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.700155 21073 raft_consensus.cc:740] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.700906 21073 consensus_queue.cc:260] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.704083 21073 ts_tablet_manager.cc:1434] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:04:21.704830 21075 tablet_bootstrap.cc:654] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.705232 21073 tablet_bootstrap.cc:492] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.704380 21080 raft_consensus.cc:515] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.710681 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:21.711431 20881 raft_consensus.cc:3060] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.708894 21080 leader_election.cc:290] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:21.729728 20881 raft_consensus.cc:2468] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 1.
I20260709 13:04:21.731364 20996 leader_election.cc:304] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:21.734073 21080 raft_consensus.cc:2804] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:21.739931 21073 tablet_bootstrap.cc:654] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.743734 21075 tablet_bootstrap.cc:492] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.744298 21075 ts_tablet_manager.cc:1403] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.004s
I20260709 13:04:21.745042 21080 raft_consensus.cc:697] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:21.746032 21080 consensus_queue.cc:237] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.746982 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:21.747707 20956 raft_consensus.cc:2468] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:21.748679 21075 raft_consensus.cc:359] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.749469 21075 raft_consensus.cc:385] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.749467 20846 leader_election.cc:304] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:21.749915 21075 raft_consensus.cc:740] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.750516 21079 raft_consensus.cc:2804] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:21.750952 21079 raft_consensus.cc:493] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.750799 21075 consensus_queue.cc:260] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.751423 21079 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.753984 21075 ts_tablet_manager.cc:1434] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:04:21.755061 21075 tablet_bootstrap.cc:492] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.754820 20955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
W20260709 13:04:21.757165 20995 leader_election.cc:343] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.758397 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
W20260709 13:04:21.760083 20995 leader_election.cc:343] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.764987 21073 tablet_bootstrap.cc:492] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.769627 21073 ts_tablet_manager.cc:1403] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.004s
I20260709 13:04:21.771584 21079 raft_consensus.cc:515] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.766449 21075 tablet_bootstrap.cc:654] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.773389 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:21.773914 21079 leader_election.cc:290] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:21.767750 21074 tablet_bootstrap.cc:654] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.767807 20777 catalog_manager.cc:5697] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.775388 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:21.775998 20956 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.781505 21073 raft_consensus.cc:359] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.782025 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:21.782509 21073 raft_consensus.cc:385] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.782891 21073 raft_consensus.cc:740] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
W20260709 13:04:21.783335 20846 leader_election.cc:343] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.783684 21073 consensus_queue.cc:260] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
W20260709 13:04:21.784986 20846 leader_election.cc:343] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.788061 21073 ts_tablet_manager.cc:1434] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:04:21.789191 21073 tablet_bootstrap.cc:492] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.795375 20956 raft_consensus.cc:2468] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:21.797145 20846 leader_election.cc:304] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 093007c6361840548ef627624cd5ecd4
I20260709 13:04:21.798202 21079 raft_consensus.cc:2804] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:21.799700 21073 tablet_bootstrap.cc:654] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.801223 21075 tablet_bootstrap.cc:492] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.801811 21075 ts_tablet_manager.cc:1403] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.006s
I20260709 13:04:21.804908 21075 raft_consensus.cc:359] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.805727 21075 raft_consensus.cc:385] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.806026 21075 raft_consensus.cc:740] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.807061 21075 consensus_queue.cc:260] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.809710 21075 ts_tablet_manager.cc:1434] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:04:21.810885 21075 tablet_bootstrap.cc:492] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.820993 21074 tablet_bootstrap.cc:492] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:21.821321 21075 tablet_bootstrap.cc:654] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.821774 21074 ts_tablet_manager.cc:1403] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.020s
I20260709 13:04:21.826383 21074 raft_consensus.cc:359] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.827401 21074 raft_consensus.cc:385] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.827806 21074 raft_consensus.cc:740] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.828894 21074 consensus_queue.cc:260] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.832521 21074 ts_tablet_manager.cc:1434] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:04:21.835687 21079 raft_consensus.cc:697] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:21.837180 21079 consensus_queue.cc:237] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.842227 21074 tablet_bootstrap.cc:492] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:21.845181 21075 tablet_bootstrap.cc:492] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.845715 21075 ts_tablet_manager.cc:1403] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.008s
I20260709 13:04:21.848877 21075 raft_consensus.cc:359] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.849673 21075 raft_consensus.cc:385] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.849970 21075 raft_consensus.cc:740] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.850721 21075 consensus_queue.cc:260] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.853278 21075 ts_tablet_manager.cc:1434] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:21.854285 21075 tablet_bootstrap.cc:492] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.861135 21073 tablet_bootstrap.cc:492] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.861637 21073 ts_tablet_manager.cc:1403] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.073s	user 0.026s	sys 0.041s
I20260709 13:04:21.868358 21074 tablet_bootstrap.cc:654] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.872700 21075 tablet_bootstrap.cc:654] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.872289 21073 raft_consensus.cc:359] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.873183 21073 raft_consensus.cc:385] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.873526 21073 raft_consensus.cc:740] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.874388 21073 consensus_queue.cc:260] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.873993 20778 catalog_manager.cc:5697] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 0 to 1, leader changed from <none> to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 1 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.876780 21073 ts_tablet_manager.cc:1434] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 13:04:21.877727 21073 tablet_bootstrap.cc:492] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.884013 21093 raft_consensus.cc:493] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.884600 21093 raft_consensus.cc:515] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.885386 21075 tablet_bootstrap.cc:492] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:21.885949 21075 ts_tablet_manager.cc:1403] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.006s
I20260709 13:04:21.886987 21093 leader_election.cc:290] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:21.888065 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:21.888599 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:21.888816 21030 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 0.
I20260709 13:04:21.889257 20881 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 0.
I20260709 13:04:21.890522 20922 leader_election.cc:304] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:21.891338 21093 raft_consensus.cc:2804] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:21.891690 21093 raft_consensus.cc:493] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.892011 21093 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.893962 21075 raft_consensus.cc:359] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.897223 21073 tablet_bootstrap.cc:654] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.897212 21075 raft_consensus.cc:385] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.897944 21075 raft_consensus.cc:740] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.899201 21093 raft_consensus.cc:515] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.899659 21075 consensus_queue.cc:260] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.903674 21080 raft_consensus.cc:493] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.904155 21080 raft_consensus.cc:515] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.906651 21080 leader_election.cc:290] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:21.908392 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:21.909034 20881 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.910456 21075 ts_tablet_manager.cc:1434] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:04:21.912900 21075 tablet_bootstrap.cc:492] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:21.913630 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:21.914677 20880 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 0.
I20260709 13:04:21.916119 20996 leader_election.cc:304] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:21.916458 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:21.917035 21080 raft_consensus.cc:2804] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:21.917433 21080 raft_consensus.cc:493] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.917802 21080 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:21.918030 20995 leader_election.cc:343] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.918622 21079 raft_consensus.cc:493] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.919071 21079 raft_consensus.cc:515] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.921134 21079 leader_election.cc:290] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:21.922652 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:21.923252 21074 tablet_bootstrap.cc:492] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:21.923483 21030 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:21.923724 21074 ts_tablet_manager.cc:1403] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.082s	user 0.017s	sys 0.016s
I20260709 13:04:21.924337 21080 raft_consensus.cc:515] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.925884 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:21.926491 20956 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:21.926597 20846 leader_election.cc:304] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:21.927136 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:21.927394 21079 raft_consensus.cc:2804] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:21.927796 20880 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.927798 21079 raft_consensus.cc:493] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.928417 21079 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.929662 20881 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 1.
I20260709 13:04:21.932252 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:21.933238 21030 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.926829 21074 raft_consensus.cc:359] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.931751 21093 leader_election.cc:290] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:21.934567 21074 raft_consensus.cc:385] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.938783 21080 leader_election.cc:290] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:21.938581 20922 leader_election.cc:304] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:21.941283 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:21.941469 21079 raft_consensus.cc:515] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.942128 21030 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 1.
I20260709 13:04:21.935406 21074 raft_consensus.cc:740] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
W20260709 13:04:21.943403 20995 leader_election.cc:343] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.944161 21074 consensus_queue.cc:260] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.943540 21079 leader_election.cc:290] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Requested vote from peers 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:21.946424 21074 ts_tablet_manager.cc:1434] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:04:21.947120 21075 tablet_bootstrap.cc:654] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.948818 21074 tablet_bootstrap.cc:492] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:21.946904 21093 raft_consensus.cc:2804] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:21.949401 21081 raft_consensus.cc:493] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.948297 20880 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 1.
I20260709 13:04:21.950080 21081 raft_consensus.cc:515] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.947525 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:21.951064 21029 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.952788 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:21.952899 20996 leader_election.cc:304] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: efe3327107f74e5381dbfd4784493422
I20260709 13:04:21.953415 20956 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.954828 21080 raft_consensus.cc:2804] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:21.956259 21081 leader_election.cc:290] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:21.957176 21080 raft_consensus.cc:697] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:21.957762 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:21.958072 21080 consensus_queue.cc:237] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
W20260709 13:04:21.959182 20922 leader_election.cc:343] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.959879 21030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
W20260709 13:04:21.961068 20921 leader_election.cc:343] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:21.961512 20921 leader_election.cc:304] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe3327107f74e5381dbfd4784493422; no voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a
I20260709 13:04:21.962324 21081 raft_consensus.cc:2749] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:21.965382 21029 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:21.966652 20956 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:21.966532 20846 leader_election.cc:304] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:21.967449 21079 raft_consensus.cc:2804] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:21.968415 21079 raft_consensus.cc:697] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:21.969126 21093 raft_consensus.cc:697] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:21.969355 21079 consensus_queue.cc:237] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:21.970000 21093 consensus_queue.cc:237] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.971583 21073 tablet_bootstrap.cc:492] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:21.972077 21073 ts_tablet_manager.cc:1403] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.037s
I20260709 13:04:21.975236 21073 raft_consensus.cc:359] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.975986 21073 raft_consensus.cc:385] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.976274 21073 raft_consensus.cc:740] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.976976 21073 consensus_queue.cc:260] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:21.979437 21073 ts_tablet_manager.cc:1434] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:04:21.980433 21073 tablet_bootstrap.cc:492] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:21.989225 21074 tablet_bootstrap.cc:654] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.990145 20778 catalog_manager.cc:5697] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 0 to 1, leader changed from <none> to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 1 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.999318 21073 tablet_bootstrap.cc:654] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.007359 20777 catalog_manager.cc:5697] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.009902 21093 raft_consensus.cc:493] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.010445 21093 raft_consensus.cc:515] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.012483 21093 leader_election.cc:290] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.013330 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:22.014299 20880 raft_consensus.cc:2393] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efe3327107f74e5381dbfd4784493422 in current term 1: Already voted for candidate 093007c6361840548ef627624cd5ecd4 in this term.
I20260709 13:04:22.019209 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:22.020898 20921 leader_election.cc:304] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe3327107f74e5381dbfd4784493422; no voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a
I20260709 13:04:22.021832 21093 raft_consensus.cc:3060] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.022629 20778 catalog_manager.cc:5697] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.028496 21093 raft_consensus.cc:2749] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:22.031333 21074 tablet_bootstrap.cc:492] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:22.031461 21073 tablet_bootstrap.cc:492] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:22.031950 21074 ts_tablet_manager.cc:1403] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.083s	user 0.016s	sys 0.009s
I20260709 13:04:22.031944 21073 ts_tablet_manager.cc:1403] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.012s
I20260709 13:04:22.035528 21073 raft_consensus.cc:359] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.036839 21073 raft_consensus.cc:385] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.037145 21073 raft_consensus.cc:740] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.035910 21074 raft_consensus.cc:359] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.037977 21074 raft_consensus.cc:385] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.037807 21073 consensus_queue.cc:260] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.038305 21074 raft_consensus.cc:740] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.039325 21075 tablet_bootstrap.cc:492] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:22.039073 21074 consensus_queue.cc:260] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.039907 21075 ts_tablet_manager.cc:1403] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.127s	user 0.018s	sys 0.016s
I20260709 13:04:22.041327 21074 ts_tablet_manager.cc:1434] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:22.042402 21074 tablet_bootstrap.cc:492] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:22.042773 21075 raft_consensus.cc:359] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.043483 21075 raft_consensus.cc:385] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.043809 21075 raft_consensus.cc:740] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.044420 21079 raft_consensus.cc:493] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.044633 21075 consensus_queue.cc:260] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.044872 21079 raft_consensus.cc:515] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.047114 21079 leader_election.cc:290] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.048187 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:22.048411 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:22.049237 20956 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
W20260709 13:04:22.049974 20846 leader_election.cc:343] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.050885 20846 leader_election.cc:304] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 093007c6361840548ef627624cd5ecd4
I20260709 13:04:22.051122 21074 tablet_bootstrap.cc:654] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.054591 21079 raft_consensus.cc:2804] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.055228 21079 raft_consensus.cc:493] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.055789 21079 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.069785 21073 ts_tablet_manager.cc:1434] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.000s
I20260709 13:04:22.071312 21074 tablet_bootstrap.cc:492] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:22.071373 21073 tablet_bootstrap.cc:492] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a: Bootstrap starting.
I20260709 13:04:22.071884 21074 ts_tablet_manager.cc:1403] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.008s
I20260709 13:04:22.074909 21074 raft_consensus.cc:359] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.075675 21074 raft_consensus.cc:385] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.076000 21074 raft_consensus.cc:740] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.076831 21075 ts_tablet_manager.cc:1434] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:04:22.076740 21074 consensus_queue.cc:260] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.077858 21073 tablet_bootstrap.cc:654] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.080459 21075 tablet_bootstrap.cc:492] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: Bootstrap starting.
I20260709 13:04:22.075276 21079 raft_consensus.cc:515] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.079429 21074 ts_tablet_manager.cc:1434] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:04:22.088181 21074 tablet_bootstrap.cc:492] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:22.088258 21079 leader_election.cc:290] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.089530 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.090147 21029 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.090510 21075 tablet_bootstrap.cc:654] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.098248 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:22.099218 20956 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.102160 21074 tablet_bootstrap.cc:654] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.103834 21075 tablet_bootstrap.cc:492] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: No bootstrap required, opened a new log
I20260709 13:04:22.104406 21075 ts_tablet_manager.cc:1403] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.000s
I20260709 13:04:22.108773 21075 raft_consensus.cc:359] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.110134 21075 raft_consensus.cc:385] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.110612 21075 raft_consensus.cc:740] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.110823 20956 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:22.111832 21075 consensus_queue.cc:260] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.112730 20846 leader_election.cc:304] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.114164 21079 raft_consensus.cc:2804] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.114990 21079 raft_consensus.cc:697] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.116251 21079 consensus_queue.cc:237] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.122694 21029 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:22.127357 21075 ts_tablet_manager.cc:1434] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:04:22.132899 21073 tablet_bootstrap.cc:492] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a: No bootstrap required, opened a new log
I20260709 13:04:22.133419 21073 ts_tablet_manager.cc:1403] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a: Time spent bootstrapping tablet: real 0.062s	user 0.036s	sys 0.013s
I20260709 13:04:22.136137 21073 raft_consensus.cc:359] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.136858 21073 raft_consensus.cc:385] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.137151 21073 raft_consensus.cc:740] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.137764 21073 consensus_queue.cc:260] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.140129 21073 ts_tablet_manager.cc:1434] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:22.144574 21080 consensus_queue.cc:1048] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.151051 21074 tablet_bootstrap.cc:492] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:22.151824 21080 consensus_queue.cc:1048] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.153302 21074 ts_tablet_manager.cc:1403] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.031s
I20260709 13:04:22.154194 20777 catalog_manager.cc:5697] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 0 to 1, leader changed from <none> to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 1 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.159893 21074 raft_consensus.cc:359] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.160857 21074 raft_consensus.cc:385] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.161242 21074 raft_consensus.cc:740] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.162104 21074 consensus_queue.cc:260] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.165343 21074 ts_tablet_manager.cc:1434] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:04:22.166544 21074 tablet_bootstrap.cc:492] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4: Bootstrap starting.
I20260709 13:04:22.172484 21074 tablet_bootstrap.cc:654] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.185822 21079 raft_consensus.cc:493] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.186362 21079 raft_consensus.cc:515] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.188516 21079 leader_election.cc:290] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.190464 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:22.191099 21029 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:22.192638 20955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:22.193485 20955 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:22.196004 20846 leader_election.cc:304] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:22.196791 21079 raft_consensus.cc:2804] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.197163 21079 raft_consensus.cc:493] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.197500 21079 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.204187 21079 raft_consensus.cc:515] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.211896 21074 tablet_bootstrap.cc:492] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4: No bootstrap required, opened a new log
I20260709 13:04:22.211910 20955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:22.212606 21074 ts_tablet_manager.cc:1403] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.035s
I20260709 13:04:22.212694 20955 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.215274 21074 raft_consensus.cc:359] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.216166 21074 raft_consensus.cc:385] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.216447 21074 raft_consensus.cc:740] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.217130 21074 consensus_queue.cc:260] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.219136 21074 ts_tablet_manager.cc:1434] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:04:22.220258 20955 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
W20260709 13:04:22.221092 20825 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:22.221943 20846 leader_election.cc:304] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.224601 21079 leader_election.cc:290] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.224838 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.225167 21079 raft_consensus.cc:2804] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.225447 21029 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.225883 21079 raft_consensus.cc:697] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.226701 21079 consensus_queue.cc:237] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.233106 21029 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:22.238502 20776 catalog_manager.cc:5697] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 0 to 1, leader changed from <none> to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 1 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.303138 21029 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.310086 21079 consensus_queue.cc:1048] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.326015 21106 consensus_queue.cc:1048] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.339490 21093 raft_consensus.cc:493] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.340147 21093 raft_consensus.cc:515] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.342764 21093 leader_election.cc:290] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.343480 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:22.343847 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:22.344398 20880 raft_consensus.cc:2393] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efe3327107f74e5381dbfd4784493422 in current term 1: Already voted for candidate 093007c6361840548ef627624cd5ecd4 in this term.
I20260709 13:04:22.345747 20922 leader_election.cc:304] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe3327107f74e5381dbfd4784493422; no voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a
I20260709 13:04:22.346727 21093 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.352226 21093 raft_consensus.cc:2749] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:22.371542 21079 raft_consensus.cc:493] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.372097 21079 raft_consensus.cc:515] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.374107 21079 leader_election.cc:290] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.375348 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4" is_pre_election: true
I20260709 13:04:22.375484 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:22.375905 21029 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:22.376143 20956 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 0.
I20260709 13:04:22.376984 20846 leader_election.cc:304] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:22.377699 21079 raft_consensus.cc:2804] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.378064 21079 raft_consensus.cc:493] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.378378 21079 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.382836 21079 raft_consensus.cc:515] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.384646 21079 leader_election.cc:290] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.385591 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:22.385800 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.386092 20956 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.386412 21029 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.390722 20956 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:22.391594 21099 raft_consensus.cc:493] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.391894 20846 leader_election.cc:304] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.392128 21099 raft_consensus.cc:515] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.392774 21079 raft_consensus.cc:2804] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.393289 21079 raft_consensus.cc:697] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.394137 21079 consensus_queue.cc:237] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.394533 21029 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 1.
I20260709 13:04:22.396481 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c50bcc3ac3a4c91b9222e710337a34a" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a" is_pre_election: true
I20260709 13:04:22.396656 21099 leader_election.cc:290] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:22.397154 20880 raft_consensus.cc:2468] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 0.
I20260709 13:04:22.397583 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c50bcc3ac3a4c91b9222e710337a34a" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422" is_pre_election: true
I20260709 13:04:22.398232 20956 raft_consensus.cc:2468] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 0.
I20260709 13:04:22.398769 20996 leader_election.cc:304] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:22.399842 21099 raft_consensus.cc:2804] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.400214 21099 raft_consensus.cc:493] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.400542 21099 raft_consensus.cc:3060] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.405680 20778 catalog_manager.cc:5697] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 0 to 1, leader changed from <none> to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 1 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.408577 21099 raft_consensus.cc:515] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.412906 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c50bcc3ac3a4c91b9222e710337a34a" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:22.413527 20880 raft_consensus.cc:3060] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.413266 21131 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.417469 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c50bcc3ac3a4c91b9222e710337a34a" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:22.418092 20956 raft_consensus.cc:3060] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.421237 20880 raft_consensus.cc:2468] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 1.
I20260709 13:04:22.422634 20996 leader_election.cc:304] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:22.423470 21131 raft_consensus.cc:2804] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.423993 21131 raft_consensus.cc:697] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:22.424685 21099 leader_election.cc:290] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 1 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:22.424809 21131 consensus_queue.cc:237] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.426044 20956 raft_consensus.cc:2468] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 1.
I20260709 13:04:22.434438 21093 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.441896 21080 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.442752 20776 catalog_manager.cc:5697] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.469869 21114 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.520566 21079 consensus_queue.cc:1048] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.534067 21079 consensus_queue.cc:1048] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.541749 21079 consensus_queue.cc:1048] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:04:22.568313 21106 consensus_queue.cc:1048] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.594089 20955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:53404
I20260709 13:04:22.594882 20955 raft_consensus.cc:606] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 LEADER]: Received request to transfer leadership to TS be6289a3181c4df88c501af7dca2509a
I20260709 13:04:22.613106 21029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
mode: GRACEFUL
new_leader_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:39166
I20260709 13:04:22.613792 21029 raft_consensus.cc:606] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Received request to transfer leadership to TS be6289a3181c4df88c501af7dca2509a
I20260709 13:04:22.629694 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:37700
I20260709 13:04:22.630275 20880 raft_consensus.cc:606] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:22.635713 21029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:39166
I20260709 13:04:22.636341 21029 raft_consensus.cc:606] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:22.641814 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:37700
I20260709 13:04:22.642452 20880 raft_consensus.cc:606] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:22.648918 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:37700
I20260709 13:04:22.649580 20880 raft_consensus.cc:606] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:22.667618 21106 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.680052 21079 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.691561 21106 raft_consensus.cc:993] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:22.692039 21106 raft_consensus.cc:1081] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:22.694562 20955 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53390
I20260709 13:04:22.695155 20955 raft_consensus.cc:493] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.695484 20955 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.700106 20955 raft_consensus.cc:515] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.703068 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:22.703552 20955 leader_election.cc:290] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.703727 20880 raft_consensus.cc:3055] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.704365 20880 raft_consensus.cc:740] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.705293 20880 consensus_queue.cc:260] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.705338 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.706703 21029 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.706889 20880 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.712842 20880 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:22.713718 21029 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:22.714031 20922 leader_election.cc:304] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.714998 21114 raft_consensus.cc:2804] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.715477 21114 raft_consensus.cc:697] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:22.716331 21114 consensus_queue.cc:237] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.723776 20776 catalog_manager.cc:5697] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 1 to 2, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.748966 21131 raft_consensus.cc:993] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:22.749468 21131 raft_consensus.cc:1081] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:22.751094 20955 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53380
I20260709 13:04:22.751754 20955 raft_consensus.cc:493] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.752133 20955 raft_consensus.cc:3060] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.758939 20955 raft_consensus.cc:515] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.761171 20955 leader_election.cc:290] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.762333 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.762336 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab44179cb3654a38a29d9b4aacfd346f" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:22.763159 20880 raft_consensus.cc:3060] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.763139 21029 raft_consensus.cc:3055] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.763671 21029 raft_consensus.cc:740] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:22.764518 21029 consensus_queue.cc:260] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.765528 21029 raft_consensus.cc:3060] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.770601 21029 raft_consensus.cc:2468] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:22.771736 20921 leader_election.cc:304] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.771834 20880 raft_consensus.cc:2468] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:22.772595 20881 raft_consensus.cc:1240] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Rejecting Update request from peer 093007c6361840548ef627624cd5ecd4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:22.774963 21114 raft_consensus.cc:2804] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.775480 21114 raft_consensus.cc:697] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:22.776266 21114 consensus_queue.cc:237] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.785048 20776 catalog_manager.cc:5697] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 1 to 2, leader changed from 093007c6361840548ef627624cd5ecd4 (127.20.66.131) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.861384 21131 consensus_queue.cc:1048] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.869668 21106 consensus_queue.cc:1048] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.874722 21154 consensus_queue.cc:1048] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:04:22.887827 21106 consensus_queue.cc:1048] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.908720 21080 raft_consensus.cc:993] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: : Instructing follower be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:22.909319 21080 raft_consensus.cc:1081] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Signalling peer be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:22.914608 20879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:37684
I20260709 13:04:22.915294 20879 raft_consensus.cc:493] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.915665 20879 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.923564 20879 raft_consensus.cc:515] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.925876 21135 raft_consensus.cc:993] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:22.926342 21106 raft_consensus.cc:1081] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:22.926952 20879 leader_election.cc:290] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 2 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.928328 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.928992 21029 raft_consensus.cc:3055] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.929088 20955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:22.929591 21029 raft_consensus.cc:740] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:22.929792 20955 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.928936 20956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53390
I20260709 13:04:22.930794 20956 raft_consensus.cc:493] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.930454 21029 consensus_queue.cc:260] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.931195 20956 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.931705 21029 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.939693 20956 raft_consensus.cc:515] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.941089 20954 raft_consensus.cc:1240] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Rejecting Update request from peer be6289a3181c4df88c501af7dca2509a for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:22.943172 21114 raft_consensus.cc:993] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: : Instructing follower be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:22.943724 21114 raft_consensus.cc:1081] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 LEADER]: Signalling peer be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:22.946437 21029 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 2.
I20260709 13:04:22.947808 20846 leader_election.cc:304] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:22.947958 20955 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 2.
I20260709 13:04:22.948731 21106 raft_consensus.cc:2804] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.949262 21106 raft_consensus.cc:697] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.949333 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:37676
I20260709 13:04:22.949992 20881 raft_consensus.cc:493] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.950402 20881 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.950234 21106 consensus_queue.cc:237] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.952931 21135 consensus_queue.cc:1059] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.953832 21106 raft_consensus.cc:3055] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.954190 21106 raft_consensus.cc:740] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.954991 21106 consensus_queue.cc:260] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.956033 21106 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.957957 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:22.961948 20775 catalog_manager.cc:5697] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 1 to 2, leader changed from 093007c6361840548ef627624cd5ecd4 (127.20.66.131) to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 2 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.963168 20881 raft_consensus.cc:515] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.959563 20956 leader_election.cc:290] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.966282 20954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:22.966897 20954 raft_consensus.cc:3055] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.967213 20954 raft_consensus.cc:740] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:22.968111 20954 consensus_queue.cc:260] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.969025 21029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.969754 21029 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.969251 20954 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.973193 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:22.973963 21028 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.979470 20954 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 2.
I20260709 13:04:22.980204 21029 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:22.980654 20846 leader_election.cc:304] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.981508 21028 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 2.
I20260709 13:04:22.981598 21106 raft_consensus.cc:2804] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.981660 20921 leader_election.cc:304] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:22.982599 21093 raft_consensus.cc:2804] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.972011 20879 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:22.983291 20881 leader_election.cc:290] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 2 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:22.983070 21093 raft_consensus.cc:697] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:22.984187 21093 consensus_queue.cc:237] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.982244 21106 raft_consensus.cc:697] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:22.985737 21106 consensus_queue.cc:237] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:22.992728 20778 catalog_manager.cc:5697] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 1 to 2, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 2 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.993280 20775 catalog_manager.cc:5697] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 1 to 2, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.120919 21028 raft_consensus.cc:1275] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.122460 21093 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.132880 20879 raft_consensus.cc:1275] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.134223 21093 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.160472 21135 raft_consensus.cc:993] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:23.161031 21106 raft_consensus.cc:1081] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:23.162599 20954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53390
I20260709 13:04:23.163233 20954 raft_consensus.cc:493] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.163533 20954 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.168046 20954 raft_consensus.cc:515] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:23.169895 20954 leader_election.cc:290] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:23.170836 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:23.171461 20879 raft_consensus.cc:3055] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.172081 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:23.171792 20879 raft_consensus.cc:740] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:23.172768 21028 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.173535 20881 raft_consensus.cc:1275] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.173336 20879 consensus_queue.cc:260] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:23.174624 20879 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.175272 21093 consensus_queue.cc:1048] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.186619 21029 raft_consensus.cc:1275] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.188230 21081 consensus_queue.cc:1048] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.206629 20879 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:23.207952 20922 leader_election.cc:304] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:23.208230 21028 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:23.208861 21081 raft_consensus.cc:2804] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.209394 21081 raft_consensus.cc:697] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:23.210209 21081 consensus_queue.cc:237] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:23.219508 20778 catalog_manager.cc:5697] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 1 to 2, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.401152 20879 raft_consensus.cc:1275] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.403314 21093 consensus_queue.cc:1048] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.407907 21029 raft_consensus.cc:1275] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.414098 21028 raft_consensus.cc:1275] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.416285 21106 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.423290 21160 consensus_queue.cc:1048] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.426453 20954 raft_consensus.cc:1275] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.428855 21106 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.516285 21030 raft_consensus.cc:1275] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.517750 21106 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.526479 20954 raft_consensus.cc:1275] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.528121 21157 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.653609 21030 raft_consensus.cc:1275] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.655195 21160 consensus_queue.cc:1048] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.664471 20879 raft_consensus.cc:1275] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.666327 21081 consensus_queue.cc:1048] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:25.104259 20809 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 20749
W20260709 13:04:25.105335 20809 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 20812
W20260709 13:04:25.106397 20809 debug-util.cc:398] Leaking SignalData structure 0x7b080009b940 after lost signal to thread 20905
W20260709 13:04:25.107357 20809 debug-util.cc:398] Leaking SignalData structure 0x7b08001b1c40 after lost signal to thread 20980
W20260709 13:04:25.108392 20809 debug-util.cc:398] Leaking SignalData structure 0x7b08001807e0 after lost signal to thread 21054
I20260709 13:04:25.727772 20746 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:04:25.728826 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:25.754743 20956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:53418
I20260709 13:04:25.755626 20956 raft_consensus.cc:606] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Received request to transfer leadership to TS 093007c6361840548ef627624cd5ecd4
I20260709 13:04:25.770105 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:04:25.770764 20881 raft_consensus.cc:606] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Received request to transfer leadership to TS 093007c6361840548ef627624cd5ecd4
I20260709 13:04:25.772322 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:25.772927 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:25.932741 21167 raft_consensus.cc:993] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: : Instructing follower 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:25.933373 21167 raft_consensus.cc:1081] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Signalling peer 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:25.935335 21028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:39160
I20260709 13:04:25.936019 21028 raft_consensus.cc:493] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:25.936448 21028 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.943624 21028 raft_consensus.cc:515] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:25.946707 21028 leader_election.cc:290] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:25.948063 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:25.948064 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:25.948835 20956 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.949195 20881 raft_consensus.cc:3055] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:25.949647 20881 raft_consensus.cc:740] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:25.950863 20881 consensus_queue.cc:260] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:25.952746 20881 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.963099 20956 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:25.963290 20881 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:25.965306 20996 leader_election.cc:304] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:25.966544 21170 raft_consensus.cc:2804] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:25.969573 21170 raft_consensus.cc:697] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:25.970633 21170 consensus_queue.cc:237] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:25.983773 20775 catalog_manager.cc:5697] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 2 to 3, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.094981 21168 raft_consensus.cc:993] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: : Instructing follower 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:26.095640 21168 raft_consensus.cc:1081] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Signalling peer 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:26.097692 21028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:39150
I20260709 13:04:26.098433 21028 raft_consensus.cc:493] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:26.098888 21028 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.107724 21028 raft_consensus.cc:515] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.110381 21028 leader_election.cc:290] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:26.111836 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:26.112468 20956 raft_consensus.cc:3055] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:26.112784 20956 raft_consensus.cc:740] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:26.113265 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:26.113938 20881 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.113847 20956 consensus_queue.cc:260] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.115841 20956 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.122792 20881 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:26.123551 20956 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:26.124572 20996 leader_election.cc:304] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:26.125658 21170 raft_consensus.cc:2804] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:26.126214 21170 raft_consensus.cc:697] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:26.127144 21170 consensus_queue.cc:237] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.144332 20775 catalog_manager.cc:5697] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 2 to 3, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.371551 20956 raft_consensus.cc:1275] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:26.373266 21170 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:26.394711 20881 raft_consensus.cc:1275] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:26.397315 21174 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:26.546224 20881 raft_consensus.cc:1275] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:26.547752 21174 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:26.561785 20956 raft_consensus.cc:1275] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:26.564069 21174 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:26.774103 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:26.786669 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:37700
I20260709 13:04:26.787387 20881 raft_consensus.cc:606] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:26.793808 21028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:39166
I20260709 13:04:26.794489 21028 raft_consensus.cc:606] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:26.800755 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:37700
I20260709 13:04:26.801506 20881 raft_consensus.cc:606] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:26.807768 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:37700
I20260709 13:04:26.809242 20881 raft_consensus.cc:606] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:26.815969 21028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
mode: GRACEFUL
new_leader_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:39166
I20260709 13:04:26.816661 21028 raft_consensus.cc:606] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Received request to transfer leadership to TS efe3327107f74e5381dbfd4784493422
I20260709 13:04:26.872958 21170 raft_consensus.cc:993] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:26.873572 21170 raft_consensus.cc:1081] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:26.875499 20956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53380
I20260709 13:04:26.876103 20956 raft_consensus.cc:493] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:26.876459 20956 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.882284 20956 raft_consensus.cc:515] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.884346 20956 leader_election.cc:290] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 4 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:26.885687 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:26.885807 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b38e78a5bfa4f208c7ac504922476ee" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:26.886739 21028 raft_consensus.cc:3055] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.886739 20881 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.887215 21028 raft_consensus.cc:740] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:26.888208 21028 consensus_queue.cc:260] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.889772 21028 raft_consensus.cc:3060] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.894073 20881 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 4.
I20260709 13:04:26.895536 20922 leader_election.cc:304] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:26.896374 21180 raft_consensus.cc:2804] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:26.897831 21180 raft_consensus.cc:697] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 4 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:26.898800 21180 consensus_queue.cc:237] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.899614 21028 raft_consensus.cc:2468] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 4.
I20260709 13:04:26.909524 20777 catalog_manager.cc:5697] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 3 to 4, leader changed from 093007c6361840548ef627624cd5ecd4 (127.20.66.131) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:26.947453 21174 raft_consensus.cc:993] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:26.948055 21194 raft_consensus.cc:1081] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:26.949975 20956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53380
I20260709 13:04:26.950680 20956 raft_consensus.cc:493] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:26.951033 20956 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.957930 20956 raft_consensus.cc:515] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.960767 20956 leader_election.cc:290] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 4 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:26.961184 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:26.961373 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189df0a1bbfc41538170cf1c9c4bed19" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:26.961987 20881 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.962028 21028 raft_consensus.cc:3055] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.962504 21028 raft_consensus.cc:740] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:26.963326 21028 consensus_queue.cc:260] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.964284 21028 raft_consensus.cc:3060] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.972314 21028 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 4.
I20260709 13:04:26.972867 20881 raft_consensus.cc:2468] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 4.
I20260709 13:04:26.973675 20921 leader_election.cc:304] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:26.974629 21180 raft_consensus.cc:2804] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:26.975461 21180 raft_consensus.cc:697] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 4 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:26.976451 21180 consensus_queue.cc:237] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:26.984815 20777 catalog_manager.cc:5697] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 3 to 4, leader changed from 093007c6361840548ef627624cd5ecd4 (127.20.66.131) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.025120 21195 raft_consensus.cc:993] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:27.025619 21167 raft_consensus.cc:1081] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:27.027611 20956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53390
I20260709 13:04:27.028213 20956 raft_consensus.cc:493] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.028555 20956 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.035504 20956 raft_consensus.cc:515] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:27.038125 20956 leader_election.cc:290] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 3 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:27.039346 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:27.039973 21028 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.039716 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:27.040572 20881 raft_consensus.cc:3055] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:27.041054 20881 raft_consensus.cc:740] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:27.042137 20881 consensus_queue.cc:260] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:27.043875 20881 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.046094 21028 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 3.
I20260709 13:04:27.047484 20921 leader_election.cc:304] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:27.048313 21180 raft_consensus.cc:2804] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:27.049293 21180 raft_consensus.cc:697] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:27.050146 21180 consensus_queue.cc:237] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:27.053709 20881 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 3.
I20260709 13:04:27.059572 20777 catalog_manager.cc:5697] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 2 to 3, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.170691 21195 raft_consensus.cc:993] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:27.171254 21167 raft_consensus.cc:1081] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:27.172849 20956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53390
I20260709 13:04:27.173470 20956 raft_consensus.cc:493] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.173769 20956 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.178411 20956 raft_consensus.cc:515] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:27.180229 20956 leader_election.cc:290] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:27.181535 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:27.181751 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:27.182495 20881 raft_consensus.cc:3055] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:27.182512 21028 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.183976 21195 raft_consensus.cc:993] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: : Instructing follower efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:27.184110 20881 raft_consensus.cc:740] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:27.184445 21195 raft_consensus.cc:1081] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Signalling peer efe3327107f74e5381dbfd4784493422 to start an election
I20260709 13:04:27.185140 20881 consensus_queue.cc:260] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:27.186722 20881 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.186666 20956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
 from {username='slave'} at 127.0.0.1:53390
I20260709 13:04:27.187808 20956 raft_consensus.cc:493] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.188265 20956 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.191488 21028 raft_consensus.cc:2468] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:27.193329 20921 leader_election.cc:304] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:27.194412 21180 raft_consensus.cc:2804] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:27.195264 21180 raft_consensus.cc:697] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:27.196310 21180 consensus_queue.cc:237] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:27.197204 20881 raft_consensus.cc:2468] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:27.197160 20956 raft_consensus.cc:515] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:27.199865 20956 leader_election.cc:290] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:27.200847 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:27.201635 20881 raft_consensus.cc:3055] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:27.201771 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "efe3327107f74e5381dbfd4784493422" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:27.202081 20881 raft_consensus.cc:740] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:27.202375 21028 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.203135 20881 consensus_queue.cc:260] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:27.205030 20881 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.213518 21028 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:27.214190 20881 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe3327107f74e5381dbfd4784493422 in term 2.
I20260709 13:04:27.215929 20922 leader_election.cc:304] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6289a3181c4df88c501af7dca2509a, efe3327107f74e5381dbfd4784493422; no voters: 
I20260709 13:04:27.216614 20777 catalog_manager.cc:5697] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 1 to 2, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.217471 21180 raft_consensus.cc:2804] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:27.218190 21180 raft_consensus.cc:697] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Leader. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:27.219128 21180 consensus_queue.cc:237] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:27.235811 20777 catalog_manager.cc:5697] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 reported cstate change: term changed from 1 to 2, leader changed from be6289a3181c4df88c501af7dca2509a (127.20.66.129) to efe3327107f74e5381dbfd4784493422 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "efe3327107f74e5381dbfd4784493422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:27.342588 21028 raft_consensus.cc:1275] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:27.344229 21190 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:27.361485 20881 raft_consensus.cc:1275] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:27.363777 21190 consensus_queue.cc:1048] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:27.384074 21028 raft_consensus.cc:1275] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:27.385859 21180 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:27.398169 20881 raft_consensus.cc:1275] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:27.401302 21180 consensus_queue.cc:1048] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:27.438036 20881 raft_consensus.cc:1275] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:27.440281 21180 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:27.452442 21028 raft_consensus.cc:1275] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:27.454203 21190 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:27.667582 20881 raft_consensus.cc:1275] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:27.669497 21190 consensus_queue.cc:1048] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:27.681682 21028 raft_consensus.cc:1275] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:27.684746 21180 consensus_queue.cc:1048] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:27.732043 21028 raft_consensus.cc:1275] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:27.733512 21180 consensus_queue.cc:1048] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:27.744225 20881 raft_consensus.cc:1275] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Refusing update from remote peer efe3327107f74e5381dbfd4784493422: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:27.747248 21190 consensus_queue.cc:1048] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:29.834724 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:29.843003 20956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:53418
I20260709 13:04:29.843918 20956 raft_consensus.cc:606] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Received request to transfer leadership to TS 093007c6361840548ef627624cd5ecd4
I20260709 13:04:29.847628 20956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:53418
I20260709 13:04:29.848280 20956 raft_consensus.cc:606] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Received request to transfer leadership to TS be6289a3181c4df88c501af7dca2509a
I20260709 13:04:29.851548 20956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:53418
I20260709 13:04:29.852197 20956 raft_consensus.cc:606] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Received request to transfer leadership to TS be6289a3181c4df88c501af7dca2509a
I20260709 13:04:29.855504 20956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:53418
I20260709 13:04:29.856067 20956 raft_consensus.cc:606] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Received request to transfer leadership to TS 093007c6361840548ef627624cd5ecd4
I20260709 13:04:29.859198 21180 raft_consensus.cc:993] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: : Instructing follower 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:29.860071 21180 raft_consensus.cc:1081] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Signalling peer 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:29.859198 20956 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974"
dest_uuid: "efe3327107f74e5381dbfd4784493422"
mode: GRACEFUL
new_leader_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:53418
I20260709 13:04:29.861109 20956 raft_consensus.cc:606] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 LEADER]: Received request to transfer leadership to TS be6289a3181c4df88c501af7dca2509a
I20260709 13:04:29.861945 21028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:39150
I20260709 13:04:29.862401 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:29.862630 21028 raft_consensus.cc:493] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:29.863147 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:29.863135 21028 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.869369 21028 raft_consensus.cc:515] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:29.871666 21028 leader_election.cc:290] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:29.873045 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:29.872884 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17494b1be7e34346aec3c408a3005de9" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:29.873883 20881 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.873981 20956 raft_consensus.cc:3055] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:29.874372 20956 raft_consensus.cc:740] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:29.875213 20956 consensus_queue.cc:260] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:29.876384 20956 raft_consensus.cc:3060] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.879740 20881 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:29.881167 20996 leader_election.cc:304] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:29.881575 20956 raft_consensus.cc:2468] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:29.882319 21194 raft_consensus.cc:2804] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:29.883419 21194 raft_consensus.cc:697] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:29.884378 21194 consensus_queue.cc:237] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:29.893567 20777 catalog_manager.cc:5697] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 2 to 3, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:29.952349 21180 raft_consensus.cc:993] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: : Instructing follower be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:29.952966 21210 raft_consensus.cc:1081] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 LEADER]: Signalling peer be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:29.954898 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:37676
I20260709 13:04:29.955462 20881 raft_consensus.cc:493] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:29.955816 20881 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.962297 20881 raft_consensus.cc:515] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:29.964340 20881 leader_election.cc:290] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 4 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:29.965636 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:29.965830 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf253c68b8f4f5ebd7cc66a6257c974" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:29.966189 20956 raft_consensus.cc:3055] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:29.966418 21028 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.966535 20956 raft_consensus.cc:740] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:29.967293 20956 consensus_queue.cc:260] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:29.968220 20956 raft_consensus.cc:3060] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.972121 21028 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 4.
I20260709 13:04:29.973136 20956 raft_consensus.cc:2468] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 4.
I20260709 13:04:29.974200 20846 leader_election.cc:304] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:29.976373 21217 raft_consensus.cc:2804] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:29.977782 21217 raft_consensus.cc:697] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 4 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:29.978591 21217 consensus_queue.cc:237] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:29.987075 20777 catalog_manager.cc:5697] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 3 to 4, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 4 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.193450 21180 raft_consensus.cc:993] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: : Instructing follower be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:30.194016 21210 raft_consensus.cc:1081] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Signalling peer be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:30.195820 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:37676
I20260709 13:04:30.196460 20881 raft_consensus.cc:493] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.196916 20881 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.203691 20881 raft_consensus.cc:515] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:30.204614 21180 raft_consensus.cc:993] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: : Instructing follower be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:30.205372 21220 raft_consensus.cc:1081] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Signalling peer be6289a3181c4df88c501af7dca2509a to start an election
I20260709 13:04:30.205848 20881 leader_election.cc:290] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 3 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:30.207348 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:30.207489 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3012666e8acc4b48ac09ba71af060586" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:30.208107 21028 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.208326 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b"
dest_uuid: "be6289a3181c4df88c501af7dca2509a"
 from {username='slave'} at 127.0.0.1:37676
I20260709 13:04:30.209694 20881 raft_consensus.cc:493] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.210166 20881 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.208050 20956 raft_consensus.cc:3055] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:30.210592 20956 raft_consensus.cc:740] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:30.211361 20956 consensus_queue.cc:260] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:30.212685 20956 raft_consensus.cc:3060] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.213989 21220 raft_consensus.cc:993] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: : Instructing follower 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:30.214488 21220 raft_consensus.cc:1081] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Signalling peer 093007c6361840548ef627624cd5ecd4 to start an election
I20260709 13:04:30.216604 21030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae"
dest_uuid: "093007c6361840548ef627624cd5ecd4"
 from {username='slave'} at 127.0.0.1:39150
I20260709 13:04:30.217307 21030 raft_consensus.cc:493] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.217680 21030 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.218098 20881 raft_consensus.cc:515] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:30.221410 21028 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 3.
I20260709 13:04:30.221453 20881 leader_election.cc:290] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 3 election: Requested vote from peers efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925), 093007c6361840548ef627624cd5ecd4 (127.20.66.131:44945)
I20260709 13:04:30.221709 20955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:30.222553 20955 raft_consensus.cc:3055] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:30.222880 20955 raft_consensus.cc:740] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:30.223431 20846 leader_election.cc:304] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:30.223825 20955 consensus_queue.cc:260] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:30.224710 21217 raft_consensus.cc:2804] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:30.225616 21217 raft_consensus.cc:697] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 3 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:30.225567 21028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24d0eb363cde4b08b4ccbd3ca2a4181b" candidate_uuid: "be6289a3181c4df88c501af7dca2509a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "093007c6361840548ef627624cd5ecd4"
I20260709 13:04:30.225495 21030 raft_consensus.cc:515] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:30.226428 21028 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.224645 20956 raft_consensus.cc:2468] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 3.
I20260709 13:04:30.226487 20955 raft_consensus.cc:3060] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.226864 21217 consensus_queue.cc:237] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:30.229079 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "be6289a3181c4df88c501af7dca2509a"
I20260709 13:04:30.229677 20881 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.236815 20881 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:30.236469 20955 raft_consensus.cc:2468] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 3.
I20260709 13:04:30.236474 21028 raft_consensus.cc:2468] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6289a3181c4df88c501af7dca2509a in term 3.
I20260709 13:04:30.238744 20996 leader_election.cc:304] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:30.239923 21194 raft_consensus.cc:2804] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:30.239846 20846 leader_election.cc:304] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093007c6361840548ef627624cd5ecd4, be6289a3181c4df88c501af7dca2509a; no voters: 
I20260709 13:04:30.240595 21194 raft_consensus.cc:697] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Becoming Leader. State: Replica: 093007c6361840548ef627624cd5ecd4, State: Running, Role: LEADER
I20260709 13:04:30.241536 21217 raft_consensus.cc:2804] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:30.242127 21217 raft_consensus.cc:697] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 3 LEADER]: Becoming Leader. State: Replica: be6289a3181c4df88c501af7dca2509a, State: Running, Role: LEADER
I20260709 13:04:30.241832 21194 consensus_queue.cc:237] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:30.243093 21217 consensus_queue.cc:237] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } }
I20260709 13:04:30.244936 20956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf33c724214480b0189e88cda20dae" candidate_uuid: "093007c6361840548ef627624cd5ecd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe3327107f74e5381dbfd4784493422"
I20260709 13:04:30.245677 20956 raft_consensus.cc:3055] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:30.246109 20956 raft_consensus.cc:740] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efe3327107f74e5381dbfd4784493422, State: Running, Role: LEADER
I20260709 13:04:30.247474 20777 catalog_manager.cc:5697] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 2 to 3, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 3 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.238113 21030 leader_election.cc:290] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [CANDIDATE]: Term 3 election: Requested vote from peers be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005), efe3327107f74e5381dbfd4784493422 (127.20.66.130:34925)
I20260709 13:04:30.255620 20956 consensus_queue.cc:260] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } }
I20260709 13:04:30.257653 20956 raft_consensus.cc:3060] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.258188 20775 catalog_manager.cc:5697] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 reported cstate change: term changed from 2 to 3, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to 093007c6361840548ef627624cd5ecd4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "093007c6361840548ef627624cd5ecd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.263523 20956 raft_consensus.cc:2468] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 093007c6361840548ef627624cd5ecd4 in term 3.
I20260709 13:04:30.268640 20775 catalog_manager.cc:5697] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a reported cstate change: term changed from 2 to 3, leader changed from efe3327107f74e5381dbfd4784493422 (127.20.66.130) to be6289a3181c4df88c501af7dca2509a (127.20.66.129). New cstate: current_term: 3 leader_uuid: "be6289a3181c4df88c501af7dca2509a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.439594 20956 raft_consensus.cc:1275] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.441430 21194 consensus_queue.cc:1048] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:30.455672 20881 raft_consensus.cc:1275] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.458320 21211 consensus_queue.cc:1048] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:30.474264 20956 raft_consensus.cc:1275] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:30.475720 21217 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:30.487205 21030 raft_consensus.cc:1275] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:30.488874 21217 consensus_queue.cc:1048] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:30.639470 21030 raft_consensus.cc:1275] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.641054 21217 consensus_queue.cc:1048] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:30.652622 20956 raft_consensus.cc:1275] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.654299 21217 consensus_queue.cc:1048] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:30.668849 20956 raft_consensus.cc:1275] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.670341 21211 consensus_queue.cc:1048] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:30.679726 20881 raft_consensus.cc:1275] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Refusing update from remote peer 093007c6361840548ef627624cd5ecd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.681416 21194 consensus_queue.cc:1048] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6289a3181c4df88c501af7dca2509a" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:30.704092 21030 raft_consensus.cc:1275] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.705464 21217 consensus_queue.cc:1048] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "093007c6361840548ef627624cd5ecd4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 44945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:30.715936 20956 raft_consensus.cc:1275] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Refusing update from remote peer be6289a3181c4df88c501af7dca2509a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:30.717694 21217 consensus_queue.cc:1048] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe3327107f74e5381dbfd4784493422" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:30.864377 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:30.870966 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
I20260709 13:04:30.902530 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:30.903497 20746 tablet_replica.cc:333] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.904562 20746 raft_consensus.cc:2243] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.905282 20746 raft_consensus.cc:2272] T 17494b1be7e34346aec3c408a3005de9 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.908931 20746 tablet_replica.cc:333] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.909662 20746 raft_consensus.cc:2243] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.910190 20746 raft_consensus.cc:2272] T 3c50bcc3ac3a4c91b9222e710337a34a P be6289a3181c4df88c501af7dca2509a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.912377 20746 tablet_replica.cc:333] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.913163 20746 raft_consensus.cc:2243] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.913743 20746 raft_consensus.cc:2272] T 3eaf33c724214480b0189e88cda20dae P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.917106 20746 tablet_replica.cc:333] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.917737 20746 raft_consensus.cc:2243] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.918633 20746 raft_consensus.cc:2272] T 3012666e8acc4b48ac09ba71af060586 P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.920852 20746 tablet_replica.cc:333] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.921518 20746 raft_consensus.cc:2243] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.922384 20746 raft_consensus.cc:2272] T aaf253c68b8f4f5ebd7cc66a6257c974 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.924604 20746 tablet_replica.cc:333] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.925261 20746 raft_consensus.cc:2243] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.926052 20746 raft_consensus.cc:2272] T 24d0eb363cde4b08b4ccbd3ca2a4181b P be6289a3181c4df88c501af7dca2509a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.928483 20746 tablet_replica.cc:333] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.929129 20746 raft_consensus.cc:2243] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.929598 20746 raft_consensus.cc:2272] T ab44179cb3654a38a29d9b4aacfd346f P be6289a3181c4df88c501af7dca2509a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.931411 20746 tablet_replica.cc:333] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.931931 20746 raft_consensus.cc:2243] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.932374 20746 raft_consensus.cc:2272] T 4b38e78a5bfa4f208c7ac504922476ee P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.934335 20746 tablet_replica.cc:333] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a: stopping tablet replica
I20260709 13:04:30.935017 20746 raft_consensus.cc:2243] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.935645 20746 raft_consensus.cc:2272] T 189df0a1bbfc41538170cf1c9c4bed19 P be6289a3181c4df88c501af7dca2509a [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:31.044824 20996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34005: connect: Connection refused (error 111)
I20260709 13:04:31.054041 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
W20260709 13:04:31.054142 20996 consensus_peers.cc:597] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 -> Peer be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005): Couldn't send request to peer be6289a3181c4df88c501af7dca2509a. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:31.054723 20996 consensus_peers.cc:597] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 -> Peer be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005): Couldn't send request to peer be6289a3181c4df88c501af7dca2509a. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.076753 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
I20260709 13:04:31.107829 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:31.108990 20746 tablet_replica.cc:333] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.110183 20746 raft_consensus.cc:2243] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.111637 20746 raft_consensus.cc:2272] T 189df0a1bbfc41538170cf1c9c4bed19 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.115031 20746 tablet_replica.cc:333] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.115890 20746 raft_consensus.cc:2243] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.116645 20746 raft_consensus.cc:2272] T aaf253c68b8f4f5ebd7cc66a6257c974 P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.119779 20746 tablet_replica.cc:333] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.120424 20746 raft_consensus.cc:2243] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.121563 20746 raft_consensus.cc:2272] T ab44179cb3654a38a29d9b4aacfd346f P efe3327107f74e5381dbfd4784493422 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.123653 20746 tablet_replica.cc:333] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.124287 20746 raft_consensus.cc:2243] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.124827 20746 raft_consensus.cc:2272] T 24d0eb363cde4b08b4ccbd3ca2a4181b P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.127552 20746 tablet_replica.cc:333] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.128185 20746 raft_consensus.cc:2243] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.128677 20746 raft_consensus.cc:2272] T 3012666e8acc4b48ac09ba71af060586 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.131260 20746 tablet_replica.cc:333] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.131810 20746 raft_consensus.cc:2243] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.132550 20746 raft_consensus.cc:2272] T 17494b1be7e34346aec3c408a3005de9 P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.135095 20746 tablet_replica.cc:333] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.135890 20746 raft_consensus.cc:2243] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.136943 20746 raft_consensus.cc:2272] T 4b38e78a5bfa4f208c7ac504922476ee P efe3327107f74e5381dbfd4784493422 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.139557 20746 tablet_replica.cc:333] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.140103 20746 raft_consensus.cc:2243] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.140497 20746 raft_consensus.cc:2272] T 3c50bcc3ac3a4c91b9222e710337a34a P efe3327107f74e5381dbfd4784493422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.142894 20746 tablet_replica.cc:333] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422: stopping tablet replica
I20260709 13:04:31.143476 20746 raft_consensus.cc:2243] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.144014 20746 raft_consensus.cc:2272] T 3eaf33c724214480b0189e88cda20dae P efe3327107f74e5381dbfd4784493422 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.175117 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
I20260709 13:04:31.198277 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:04:31.200399 20996 consensus_peers.cc:597] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 -> Peer be6289a3181c4df88c501af7dca2509a (127.20.66.129:34005): Couldn't send request to peer be6289a3181c4df88c501af7dca2509a. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.228076 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:31.228889 20746 tablet_replica.cc:333] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.229656 20746 raft_consensus.cc:2243] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.230613 20746 raft_consensus.cc:2272] T 17494b1be7e34346aec3c408a3005de9 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.233498 20746 tablet_replica.cc:333] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.234097 20746 raft_consensus.cc:2243] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.234608 20746 raft_consensus.cc:2272] T 189df0a1bbfc41538170cf1c9c4bed19 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.236642 20746 tablet_replica.cc:333] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.237319 20746 raft_consensus.cc:2243] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.237810 20746 raft_consensus.cc:2272] T 3012666e8acc4b48ac09ba71af060586 P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.240542 20746 tablet_replica.cc:333] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.241291 20746 raft_consensus.cc:2243] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.241788 20746 raft_consensus.cc:2272] T ab44179cb3654a38a29d9b4aacfd346f P 093007c6361840548ef627624cd5ecd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.243780 20746 tablet_replica.cc:333] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.244321 20746 raft_consensus.cc:2243] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.244804 20746 raft_consensus.cc:2272] T 24d0eb363cde4b08b4ccbd3ca2a4181b P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.247123 20746 tablet_replica.cc:333] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.247642 20746 raft_consensus.cc:2243] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.248091 20746 raft_consensus.cc:2272] T aaf253c68b8f4f5ebd7cc66a6257c974 P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.250347 20746 tablet_replica.cc:333] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.250839 20746 raft_consensus.cc:2243] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.251221 20746 raft_consensus.cc:2272] T 4b38e78a5bfa4f208c7ac504922476ee P 093007c6361840548ef627624cd5ecd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.253099 20746 tablet_replica.cc:333] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.253574 20746 raft_consensus.cc:2243] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.254310 20746 raft_consensus.cc:2272] T 3c50bcc3ac3a4c91b9222e710337a34a P 093007c6361840548ef627624cd5ecd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.256558 20746 tablet_replica.cc:333] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4: stopping tablet replica
I20260709 13:04:31.257308 20746 raft_consensus.cc:2243] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.258222 20746 raft_consensus.cc:2272] T 3eaf33c724214480b0189e88cda20dae P 093007c6361840548ef627624cd5ecd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.279899 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:04:31.301736 20746 master.cc:562] Master@127.20.66.190:44067 shutting down...
I20260709 13:04:31.463609 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.464272 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.464592 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P bbdccef5b87c4890a2912bdc40c679fa: stopping tablet replica
I20260709 13:04:31.503526 20746 master.cc:584] Master@127.20.66.190:44067 shutdown complete.
093007c6361840548ef627624cd5ecd4  1
be6289a3181c4df88c501af7dca2509a  4
efe3327107f74e5381dbfd4784493422  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12895 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:04:31.555843 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:41239
I20260709 13:04:31.557358 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:31.564338 21241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.564595 21242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.566447 21244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:31.567489 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:31.568527 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:31.568790 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:31.568969 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602271568951 us; error 0 us; skew 500 ppm
I20260709 13:04:31.569639 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:31.572669 20746 webserver.cc:533] Webserver started at http://127.20.66.190:37191/ using document root <none> and password file <none>
I20260709 13:04:31.573827 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:31.574158 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:31.574498 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:31.575877 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "9d859ca2ffb54b5ba41e19d9e39b16af"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:31.581190 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:04:31.586292 21249 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.587558 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:04:31.587939 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "9d859ca2ffb54b5ba41e19d9e39b16af"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:31.588320 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:31.602780 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:31.604424 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:31.656081 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:41239
I20260709 13:04:31.656190 21300 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:41239 every 8 connection(s)
I20260709 13:04:31.661041 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:31.672555 21301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af: Bootstrap starting.
I20260709 13:04:31.678071 21301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:31.683547 21301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af: No bootstrap required, opened a new log
I20260709 13:04:31.686132 21301 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" member_type: VOTER }
I20260709 13:04:31.686618 21301 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:31.686817 21301 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d859ca2ffb54b5ba41e19d9e39b16af, State: Initialized, Role: FOLLOWER
I20260709 13:04:31.687403 21301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" member_type: VOTER }
I20260709 13:04:31.687903 21301 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:31.688120 21301 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:31.688361 21301 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.693256 21301 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" member_type: VOTER }
I20260709 13:04:31.694000 21301 leader_election.cc:304] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d859ca2ffb54b5ba41e19d9e39b16af; no voters: 
I20260709 13:04:31.695333 21301 leader_election.cc:290] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:31.695910 21304 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:31.697496 21304 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 1 LEADER]: Becoming Leader. State: Replica: 9d859ca2ffb54b5ba41e19d9e39b16af, State: Running, Role: LEADER
I20260709 13:04:31.698321 21304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" member_type: VOTER }
I20260709 13:04:31.698915 21301 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:31.704273 21306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d859ca2ffb54b5ba41e19d9e39b16af. Latest consensus state: current_term: 1 leader_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" member_type: VOTER } }
I20260709 13:04:31.704617 21305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d859ca2ffb54b5ba41e19d9e39b16af" member_type: VOTER } }
I20260709 13:04:31.705200 21306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:31.705214 21305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:31.708102 21311 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:31.716109 21311 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:31.720528 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:31.726248 21311 catalog_manager.cc:1383] Generated new cluster ID: 47cbe96bb6a9408ea08557bc846a6696
I20260709 13:04:31.726637 21311 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:31.753235 21311 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:31.754550 21311 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:31.768034 21311 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af: Generated new TSK 0
I20260709 13:04:31.768779 21311 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:31.787633 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:31.794328 21322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.795368 21323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.797997 21325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:31.798103 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:31.799137 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:31.799377 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:31.799575 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602271799560 us; error 0 us; skew 500 ppm
I20260709 13:04:31.800271 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:31.803040 20746 webserver.cc:533] Webserver started at http://127.20.66.129:34579/ using document root <none> and password file <none>
I20260709 13:04:31.803567 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:31.803735 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:31.804083 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:31.805457 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "30816458075e446eada4afc2f0e2796d"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:31.810233 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:31.815220 21330 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.816094 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:04:31.816438 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "30816458075e446eada4afc2f0e2796d"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:31.816726 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:31.828346 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:31.829697 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:31.831493 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:31.833951 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:31.834136 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.834455 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:31.834632 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.879452 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:32903
I20260709 13:04:31.879554 21392 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:32903 every 8 connection(s)
I20260709 13:04:31.884632 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:31.892132 21397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.894384 21398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.895645 21400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:31.895918 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:31.896931 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:31.897198 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:31.897325 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602271897313 us; error 0 us; skew 500 ppm
I20260709 13:04:31.897797 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:31.900936 20746 webserver.cc:533] Webserver started at http://127.20.66.130:43441/ using document root <none> and password file <none>
I20260709 13:04:31.901661 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:31.901929 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:31.902287 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:31.902421 21393 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41239
I20260709 13:04:31.902868 21393 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:31.903436 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:31.903625 21393 heartbeater.cc:507] Master 127.20.66.190:41239 requested a full tablet report, sending...
I20260709 13:04:31.905893 21266 ts_manager.cc:194] Registered new tserver with Master: 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903)
I20260709 13:04:31.907766 21266 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57422
I20260709 13:04:31.909075 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:04:31.912632 21405 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.913717 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:31.914152 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:31.914616 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:31.928292 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:31.929682 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:31.931151 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:31.933746 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:31.933995 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.934263 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:31.934454 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.979285 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:40509
I20260709 13:04:31.979394 21467 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:40509 every 8 connection(s)
I20260709 13:04:31.983945 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:31.990679 21471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.991448 21472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:31.995347 20746 server_base.cc:1061] running on GCE node
W20260709 13:04:31.995945 21475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:31.996942 21468 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41239
I20260709 13:04:31.997139 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:31.997428 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:31.997480 21468 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:31.997653 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602271997613 us; error 0 us; skew 500 ppm
I20260709 13:04:31.998366 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:31.998442 21468 heartbeater.cc:507] Master 127.20.66.190:41239 requested a full tablet report, sending...
I20260709 13:04:32.001070 21266 ts_manager.cc:194] Registered new tserver with Master: c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:32.003074 20746 webserver.cc:533] Webserver started at http://127.20.66.131:39055/ using document root <none> and password file <none>
I20260709 13:04:32.003590 21266 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57424
I20260709 13:04:32.003816 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.004078 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.004408 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.005923 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "344ed664705b49dfa97dee21a1b70411"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:32.011765 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:04:32.016633 21479 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.017834 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:32.018270 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "344ed664705b49dfa97dee21a1b70411"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-rgrd"
I20260709 13:04:32.018594 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.031883 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.033195 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.034742 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:32.037205 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:32.037448 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.037695 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:32.037894 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.085531 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:41469
I20260709 13:04:32.085592 21541 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:41469 every 8 connection(s)
I20260709 13:04:32.100634 21542 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41239
I20260709 13:04:32.101188 21542 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:32.102013 21542 heartbeater.cc:507] Master 127.20.66.190:41239 requested a full tablet report, sending...
I20260709 13:04:32.104404 21266 ts_manager.cc:194] Registered new tserver with Master: 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:32.104887 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016390643s
I20260709 13:04:32.105926 21266 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57426
I20260709 13:04:32.142019 21266 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57432:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:32.145237 21266 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:32.224817 21357 tablet_service.cc:1511] Processing CreateTablet for tablet 480c92b5319a4566b45b409b138eac0e (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:32.227851 21357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 480c92b5319a4566b45b409b138eac0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.226974 21358 tablet_service.cc:1511] Processing CreateTablet for tablet 6183ad54ee054404aa4ac5d460cdfbe4 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:32.229326 21358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6183ad54ee054404aa4ac5d460cdfbe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.237579 21356 tablet_service.cc:1511] Processing CreateTablet for tablet 9d62f7db1ffb4fdc850f526801d775be (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:32.239327 21356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d62f7db1ffb4fdc850f526801d775be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.242771 21355 tablet_service.cc:1511] Processing CreateTablet for tablet 2efde3d922144fbd9d837360977e20b9 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:32.244128 21355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efde3d922144fbd9d837360977e20b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.247337 21433 tablet_service.cc:1511] Processing CreateTablet for tablet 6183ad54ee054404aa4ac5d460cdfbe4 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:32.248749 21433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6183ad54ee054404aa4ac5d460cdfbe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.250304 21432 tablet_service.cc:1511] Processing CreateTablet for tablet 480c92b5319a4566b45b409b138eac0e (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:32.251659 21432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 480c92b5319a4566b45b409b138eac0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.254168 21431 tablet_service.cc:1511] Processing CreateTablet for tablet 9d62f7db1ffb4fdc850f526801d775be (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:32.255659 21431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d62f7db1ffb4fdc850f526801d775be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.258241 21430 tablet_service.cc:1511] Processing CreateTablet for tablet 2efde3d922144fbd9d837360977e20b9 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:32.259577 21430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efde3d922144fbd9d837360977e20b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.263262 21429 tablet_service.cc:1511] Processing CreateTablet for tablet be9e67348632445a92364fcca02e5ea5 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:32.264657 21429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9e67348632445a92364fcca02e5ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.266140 21428 tablet_service.cc:1511] Processing CreateTablet for tablet 06afeced037f44c799edc35d5b8c4dcd (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:32.267453 21428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06afeced037f44c799edc35d5b8c4dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.271643 21354 tablet_service.cc:1511] Processing CreateTablet for tablet be9e67348632445a92364fcca02e5ea5 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:32.274063 21352 tablet_service.cc:1511] Processing CreateTablet for tablet abdcd6cedd2745ab8091c067e79fee6a (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:32.275741 21351 tablet_service.cc:1511] Processing CreateTablet for tablet 442f9ad084ff4c8b8794e7291eae0a2b (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:32.275990 21353 tablet_service.cc:1511] Processing CreateTablet for tablet 06afeced037f44c799edc35d5b8c4dcd (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:32.277657 21350 tablet_service.cc:1511] Processing CreateTablet for tablet f73d334b1ef14d248f3ae2145223dcf6 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:32.278841 21427 tablet_service.cc:1511] Processing CreateTablet for tablet abdcd6cedd2745ab8091c067e79fee6a (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:32.279021 21426 tablet_service.cc:1511] Processing CreateTablet for tablet 442f9ad084ff4c8b8794e7291eae0a2b (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:32.280190 21427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abdcd6cedd2745ab8091c067e79fee6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.280974 21426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 442f9ad084ff4c8b8794e7291eae0a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.281670 21354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9e67348632445a92364fcca02e5ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.286118 21351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 442f9ad084ff4c8b8794e7291eae0a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.286402 21425 tablet_service.cc:1511] Processing CreateTablet for tablet f73d334b1ef14d248f3ae2145223dcf6 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:32.287716 21425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73d334b1ef14d248f3ae2145223dcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.287716 21350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73d334b1ef14d248f3ae2145223dcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.289314 21353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06afeced037f44c799edc35d5b8c4dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.291203 21352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abdcd6cedd2745ab8091c067e79fee6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.310592 21507 tablet_service.cc:1511] Processing CreateTablet for tablet 6183ad54ee054404aa4ac5d460cdfbe4 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:32.310847 21349 tablet_service.cc:1511] Processing CreateTablet for tablet c264fe14fc1847eeb087cc1fef0c5aec (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:32.312078 21507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6183ad54ee054404aa4ac5d460cdfbe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.312081 21349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c264fe14fc1847eeb087cc1fef0c5aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.314189 21506 tablet_service.cc:1511] Processing CreateTablet for tablet 480c92b5319a4566b45b409b138eac0e (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:32.315591 21506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 480c92b5319a4566b45b409b138eac0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.320492 21505 tablet_service.cc:1511] Processing CreateTablet for tablet 9d62f7db1ffb4fdc850f526801d775be (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:32.322021 21505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d62f7db1ffb4fdc850f526801d775be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.322183 21500 tablet_service.cc:1511] Processing CreateTablet for tablet 442f9ad084ff4c8b8794e7291eae0a2b (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:32.322580 21499 tablet_service.cc:1511] Processing CreateTablet for tablet f73d334b1ef14d248f3ae2145223dcf6 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:32.323544 21500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 442f9ad084ff4c8b8794e7291eae0a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.324196 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73d334b1ef14d248f3ae2145223dcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.331277 21504 tablet_service.cc:1511] Processing CreateTablet for tablet 2efde3d922144fbd9d837360977e20b9 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:32.331758 21502 tablet_service.cc:1511] Processing CreateTablet for tablet 06afeced037f44c799edc35d5b8c4dcd (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:32.334709 21503 tablet_service.cc:1511] Processing CreateTablet for tablet be9e67348632445a92364fcca02e5ea5 (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:32.328975 21501 tablet_service.cc:1511] Processing CreateTablet for tablet abdcd6cedd2745ab8091c067e79fee6a (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:32.344316 21424 tablet_service.cc:1511] Processing CreateTablet for tablet c264fe14fc1847eeb087cc1fef0c5aec (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:32.346220 21424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c264fe14fc1847eeb087cc1fef0c5aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.354179 21557 tablet_bootstrap.cc:492] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.375839 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efde3d922144fbd9d837360977e20b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.376638 21503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9e67348632445a92364fcca02e5ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.389395 21498 tablet_service.cc:1511] Processing CreateTablet for tablet c264fe14fc1847eeb087cc1fef0c5aec (DEFAULT_TABLE table=test-workload [id=6de601ca76c143efa5882f77aba1ea9f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:32.391462 21502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06afeced037f44c799edc35d5b8c4dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.397567 21501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abdcd6cedd2745ab8091c067e79fee6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.409386 21498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c264fe14fc1847eeb087cc1fef0c5aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.470404 21557 tablet_bootstrap.cc:700] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: Time spent opening tablet: real 0.110s	user 0.000s	sys 0.004s
I20260709 13:04:32.471199 21557 tablet_bootstrap.cc:654] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.478107 21559 tablet_bootstrap.cc:492] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:32.486027 21558 tablet_bootstrap.cc:492] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.499528 21559 tablet_bootstrap.cc:654] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.512135 21557 tablet_bootstrap.cc:492] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.512588 21557 ts_tablet_manager.cc:1403] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.159s	user 0.011s	sys 0.016s
I20260709 13:04:32.513821 21558 tablet_bootstrap.cc:654] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.514228 21559 tablet_bootstrap.cc:492] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:32.514730 21559 ts_tablet_manager.cc:1403] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.005s
I20260709 13:04:32.515687 21557 raft_consensus.cc:359] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.516444 21557 raft_consensus.cc:385] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.516857 21557 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.517618 21557 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.517776 21559 raft_consensus.cc:359] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.518430 21559 raft_consensus.cc:385] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.518908 21558 tablet_bootstrap.cc:492] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.518755 21559 raft_consensus.cc:740] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.519387 21558 ts_tablet_manager.cc:1403] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.002s
I20260709 13:04:32.519834 21559 consensus_queue.cc:260] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.522046 21393 heartbeater.cc:499] Master 127.20.66.190:41239 was elected leader, sending a full tablet report...
I20260709 13:04:32.522951 21557 ts_tablet_manager.cc:1434] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:04:32.524240 21557 tablet_bootstrap.cc:492] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.524966 21558 raft_consensus.cc:359] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.525714 21558 raft_consensus.cc:385] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.525995 21558 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.526800 21558 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.529480 21559 ts_tablet_manager.cc:1434] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.004s
I20260709 13:04:32.530263 21542 heartbeater.cc:499] Master 127.20.66.190:41239 was elected leader, sending a full tablet report...
I20260709 13:04:32.532137 21557 tablet_bootstrap.cc:654] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.535636 21559 tablet_bootstrap.cc:492] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:32.539491 21468 heartbeater.cc:499] Master 127.20.66.190:41239 was elected leader, sending a full tablet report...
I20260709 13:04:32.539852 21558 ts_tablet_manager.cc:1434] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.011s
I20260709 13:04:32.540827 21558 tablet_bootstrap.cc:492] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.547679 21557 tablet_bootstrap.cc:492] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.548193 21557 ts_tablet_manager.cc:1403] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.009s
I20260709 13:04:32.548779 21558 tablet_bootstrap.cc:654] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.551293 21557 raft_consensus.cc:359] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.552125 21557 raft_consensus.cc:385] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.552440 21557 raft_consensus.cc:740] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.553620 21558 tablet_bootstrap.cc:492] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.553202 21557 consensus_queue.cc:260] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.554069 21558 ts_tablet_manager.cc:1403] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.013s	user 0.002s	sys 0.009s
I20260709 13:04:32.555398 21557 ts_tablet_manager.cc:1434] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:32.556365 21557 tablet_bootstrap.cc:492] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.556345 21558 raft_consensus.cc:359] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.557229 21558 raft_consensus.cc:385] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.557552 21558 raft_consensus.cc:740] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.558488 21558 consensus_queue.cc:260] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.559823 21559 tablet_bootstrap.cc:654] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.560927 21558 ts_tablet_manager.cc:1434] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:32.561964 21558 tablet_bootstrap.cc:492] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.563948 21557 tablet_bootstrap.cc:654] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.569685 21558 tablet_bootstrap.cc:654] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.595227 21565 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.595942 21565 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.601610 21565 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:32.617597 21564 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.618378 21564 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.628574 21564 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:32.679814 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:32.683043 21368 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:32.685364 21409 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:32.686882 21565 raft_consensus.cc:2804] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:32.687389 21565 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.687800 21565 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.687887 21563 raft_consensus.cc:493] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.688359 21563 raft_consensus.cc:515] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.701514 21563 leader_election.cc:290] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:32.699854 21565 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.704278 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:32.706610 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:32.707466 21368 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.708014 21565 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
W20260709 13:04:32.714882 21314 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:32.714905 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
W20260709 13:04:32.717341 21406 leader_election.cc:343] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:32.718636 21406 leader_election.cc:343] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.733539 21558 tablet_bootstrap.cc:492] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.734131 21558 ts_tablet_manager.cc:1403] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.172s	user 0.038s	sys 0.034s
I20260709 13:04:32.736166 21559 tablet_bootstrap.cc:492] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:32.738777 21557 tablet_bootstrap.cc:492] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.738773 21559 ts_tablet_manager.cc:1403] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.203s	user 0.045s	sys 0.131s
I20260709 13:04:32.739482 21557 ts_tablet_manager.cc:1403] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.183s	user 0.037s	sys 0.097s
I20260709 13:04:32.739692 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
W20260709 13:04:32.741700 21481 leader_election.cc:343] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.742477 21559 raft_consensus.cc:359] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.743369 21559 raft_consensus.cc:385] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.743764 21559 raft_consensus.cc:740] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.744146 21558 raft_consensus.cc:359] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.744565 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:32.745168 21558 raft_consensus.cc:385] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.744588 21559 consensus_queue.cc:260] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.745487 21558 raft_consensus.cc:740] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.746356 21558 consensus_queue.cc:260] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.742856 21557 raft_consensus.cc:359] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.747256 21559 ts_tablet_manager.cc:1434] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:04:32.747299 21557 raft_consensus.cc:385] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.748057 21557 raft_consensus.cc:740] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.748497 21559 tablet_bootstrap.cc:492] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
W20260709 13:04:32.748551 21483 leader_election.cc:343] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.749131 21483 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411; no voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:32.750370 21558 ts_tablet_manager.cc:1434] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:04:32.751353 21558 tablet_bootstrap.cc:492] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.752434 21564 raft_consensus.cc:2749] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:32.755812 21559 tablet_bootstrap.cc:654] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.757486 21558 tablet_bootstrap.cc:654] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.748842 21557 consensus_queue.cc:260] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.759672 21557 ts_tablet_manager.cc:1434] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 13:04:32.760807 21557 tablet_bootstrap.cc:492] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.771373 21557 tablet_bootstrap.cc:654] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.784340 21557 tablet_bootstrap.cc:492] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.785066 21557 ts_tablet_manager.cc:1403] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:04:32.791638 21557 raft_consensus.cc:359] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.792663 21557 raft_consensus.cc:385] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.793076 21557 raft_consensus.cc:740] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.794124 21557 consensus_queue.cc:260] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.797469 21557 ts_tablet_manager.cc:1434] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:04:32.801585 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:32.802443 21443 raft_consensus.cc:2468] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 0.
I20260709 13:04:32.804113 21332 leader_election.cc:304] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:32.805516 21563 raft_consensus.cc:2804] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:32.806013 21563 raft_consensus.cc:493] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.806507 21563 raft_consensus.cc:3060] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.742581 21368 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:32.809427 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:32.810636 21409 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 344ed664705b49dfa97dee21a1b70411
W20260709 13:04:32.810889 21331 leader_election.cc:343] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.811489 21558 tablet_bootstrap.cc:492] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.811540 21565 raft_consensus.cc:2804] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:32.811893 21559 tablet_bootstrap.cc:492] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:32.812050 21558 ts_tablet_manager.cc:1403] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.061s	user 0.018s	sys 0.039s
I20260709 13:04:32.812343 21559 ts_tablet_manager.cc:1403] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.041s
I20260709 13:04:32.814955 21558 raft_consensus.cc:359] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.815677 21558 raft_consensus.cc:385] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.815977 21558 raft_consensus.cc:740] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.816830 21558 consensus_queue.cc:260] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.817366 21563 raft_consensus.cc:515] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.819011 21583 raft_consensus.cc:493] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.819461 21583 raft_consensus.cc:515] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.820565 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:32.820704 21559 raft_consensus.cc:359] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.821214 21443 raft_consensus.cc:3060] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.821491 21559 raft_consensus.cc:385] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.821478 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:32.821974 21559 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.822484 21563 leader_election.cc:290] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
W20260709 13:04:32.823096 21331 leader_election.cc:343] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.822757 21559 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.823940 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:32.825166 21559 ts_tablet_manager.cc:1434] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.004s
W20260709 13:04:32.825274 21331 leader_election.cc:343] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.825874 21583 leader_election.cc:290] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:32.826238 21559 tablet_bootstrap.cc:492] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:32.832255 21558 ts_tablet_manager.cc:1434] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.004s
I20260709 13:04:32.833328 21559 tablet_bootstrap.cc:654] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.833483 21558 tablet_bootstrap.cc:492] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.834501 21442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
W20260709 13:04:32.835942 21332 leader_election.cc:343] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.836593 21332 leader_election.cc:304] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30816458075e446eada4afc2f0e2796d; no voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:32.837834 21583 raft_consensus.cc:2749] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:32.840302 21558 tablet_bootstrap.cc:654] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.841428 21559 tablet_bootstrap.cc:492] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:32.841895 21559 ts_tablet_manager.cc:1403] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:04:32.843245 21443 raft_consensus.cc:2468] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 1.
I20260709 13:04:32.844401 21332 leader_election.cc:304] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:32.845188 21583 raft_consensus.cc:2804] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:32.844914 21559 raft_consensus.cc:359] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.845338 21558 tablet_bootstrap.cc:492] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.845674 21559 raft_consensus.cc:385] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.845711 21583 raft_consensus.cc:697] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:32.846000 21558 ts_tablet_manager.cc:1403] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:04:32.846143 21559 raft_consensus.cc:740] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.845595 21565 raft_consensus.cc:697] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:32.846721 21583 consensus_queue.cc:237] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.847126 21559 consensus_queue.cc:260] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.847324 21565 consensus_queue.cc:237] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.849442 21559 ts_tablet_manager.cc:1434] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:04:32.850411 21559 tablet_bootstrap.cc:492] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:32.849974 21558 raft_consensus.cc:359] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.850834 21558 raft_consensus.cc:385] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.846140 21557 tablet_bootstrap.cc:492] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.851197 21558 raft_consensus.cc:740] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.852056 21558 consensus_queue.cc:260] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.854563 21558 ts_tablet_manager.cc:1434] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:04:32.855562 21558 tablet_bootstrap.cc:492] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.857368 21559 tablet_bootstrap.cc:654] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.857955 21557 tablet_bootstrap.cc:654] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.861985 21558 tablet_bootstrap.cc:654] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.875264 21265 catalog_manager.cc:5697] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 1 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.884938 21559 tablet_bootstrap.cc:492] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:32.885883 21557 tablet_bootstrap.cc:492] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.883670 21264 catalog_manager.cc:5697] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 0 to 1, leader changed from <none> to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 1 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.888120 21557 ts_tablet_manager.cc:1403] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.002s
I20260709 13:04:32.885643 21559 ts_tablet_manager.cc:1403] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.024s
I20260709 13:04:32.891479 21559 raft_consensus.cc:359] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.892349 21559 raft_consensus.cc:385] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.892652 21559 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.892848 21557 raft_consensus.cc:359] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.893566 21557 raft_consensus.cc:385] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.893898 21557 raft_consensus.cc:740] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.893460 21559 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.894660 21557 consensus_queue.cc:260] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.896915 21557 ts_tablet_manager.cc:1434] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.003s
I20260709 13:04:32.897892 21557 tablet_bootstrap.cc:492] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.898777 21558 tablet_bootstrap.cc:492] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.899399 21558 ts_tablet_manager.cc:1403] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.030s
I20260709 13:04:32.903184 21559 ts_tablet_manager.cc:1434] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 13:04:32.904593 21557 tablet_bootstrap.cc:654] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.906322 21558 raft_consensus.cc:359] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.907145 21558 raft_consensus.cc:385] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.907462 21558 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.908306 21558 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.910107 21558 ts_tablet_manager.cc:1434] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.005s
I20260709 13:04:32.910853 21558 tablet_bootstrap.cc:492] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.911435 21559 tablet_bootstrap.cc:492] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:32.915954 21557 tablet_bootstrap.cc:492] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.916414 21558 tablet_bootstrap.cc:654] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.916486 21557 ts_tablet_manager.cc:1403] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.004s
I20260709 13:04:32.918980 21559 tablet_bootstrap.cc:654] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.927274 21557 raft_consensus.cc:359] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.928340 21557 raft_consensus.cc:385] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.928745 21557 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.933320 21557 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.937747 21559 tablet_bootstrap.cc:492] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:32.938285 21559 ts_tablet_manager.cc:1403] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.007s
I20260709 13:04:32.938573 21558 tablet_bootstrap.cc:492] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:32.939033 21558 ts_tablet_manager.cc:1403] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.009s
I20260709 13:04:32.939324 21557 ts_tablet_manager.cc:1434] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.007s
I20260709 13:04:32.940352 21557 tablet_bootstrap.cc:492] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.941326 21559 raft_consensus.cc:359] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.942085 21559 raft_consensus.cc:385] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.941781 21558 raft_consensus.cc:359] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.942407 21559 raft_consensus.cc:740] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.942577 21558 raft_consensus.cc:385] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.942932 21558 raft_consensus.cc:740] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.943356 21559 consensus_queue.cc:260] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.943668 21558 consensus_queue.cc:260] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.946296 21558 ts_tablet_manager.cc:1434] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:04:32.947273 21558 tablet_bootstrap.cc:492] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:32.947854 21557 tablet_bootstrap.cc:654] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.950067 21559 ts_tablet_manager.cc:1434] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:32.951066 21559 tablet_bootstrap.cc:492] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:32.951645 21564 raft_consensus.cc:493] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.952157 21564 raft_consensus.cc:515] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.954267 21564 leader_election.cc:290] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:32.954756 21558 tablet_bootstrap.cc:654] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.956220 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:32.956857 21368 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 0.
I20260709 13:04:32.963100 21557 tablet_bootstrap.cc:492] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:32.963632 21557 ts_tablet_manager.cc:1403] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 13:04:32.966086 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:32.966509 21557 raft_consensus.cc:359] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.966958 21443 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 0.
I20260709 13:04:32.967247 21557 raft_consensus.cc:385] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.967612 21557 raft_consensus.cc:740] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.968374 21557 consensus_queue.cc:260] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.968868 21481 leader_election.cc:304] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:32.970575 21564 raft_consensus.cc:2804] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:32.970724 21557 ts_tablet_manager.cc:1434] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:32.970985 21564 raft_consensus.cc:493] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.971343 21564 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.972064 21557 tablet_bootstrap.cc:492] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:32.974504 21583 raft_consensus.cc:493] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.975013 21583 raft_consensus.cc:515] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:32.977183 21583 leader_election.cc:290] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:32.979137 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efde3d922144fbd9d837360977e20b9" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:32.979602 21557 tablet_bootstrap.cc:654] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:32.980696 21331 leader_election.cc:343] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.982980 21559 tablet_bootstrap.cc:654] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.986378 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efde3d922144fbd9d837360977e20b9" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:32.987273 21443 raft_consensus.cc:2468] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 0.
I20260709 13:04:32.988940 21332 leader_election.cc:304] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:32.990502 21583 raft_consensus.cc:2804] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:32.991019 21583 raft_consensus.cc:493] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.991621 21583 raft_consensus.cc:3060] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.997167 21564 raft_consensus.cc:515] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.003976 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.005002 21564 leader_election.cc:290] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.004873 21368 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.006474 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.007179 21443 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.013381 21368 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 1.
I20260709 13:04:33.013948 21559 tablet_bootstrap.cc:492] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:33.014489 21559 ts_tablet_manager.cc:1403] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.000s
I20260709 13:04:33.014772 21483 leader_election.cc:304] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:33.015506 21443 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 1.
I20260709 13:04:33.015637 21564 raft_consensus.cc:2804] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.015903 21583 raft_consensus.cc:515] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.017604 21559 raft_consensus.cc:359] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.018378 21559 raft_consensus.cc:385] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.018684 21558 tablet_bootstrap.cc:492] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:33.018746 21559 raft_consensus.cc:740] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.019191 21558 ts_tablet_manager.cc:1403] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.072s	user 0.022s	sys 0.028s
I20260709 13:04:33.018472 21557 tablet_bootstrap.cc:492] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:33.019812 21557 ts_tablet_manager.cc:1403] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.022s
I20260709 13:04:33.020087 21583 leader_election.cc:290] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.020967 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efde3d922144fbd9d837360977e20b9" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.021301 21564 raft_consensus.cc:697] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:33.021535 21443 raft_consensus.cc:3060] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.021857 21559 consensus_queue.cc:260] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.022250 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efde3d922144fbd9d837360977e20b9" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.022209 21564 consensus_queue.cc:237] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
W20260709 13:04:33.023859 21331 leader_election.cc:343] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.024322 21559 ts_tablet_manager.cc:1434] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:04:33.025282 21559 tablet_bootstrap.cc:492] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:33.025189 21558 raft_consensus.cc:359] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.025995 21558 raft_consensus.cc:385] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.026365 21558 raft_consensus.cc:740] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.027225 21558 consensus_queue.cc:260] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.029361 21443 raft_consensus.cc:2468] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 1.
I20260709 13:04:33.029810 21558 ts_tablet_manager.cc:1434] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:33.030800 21558 tablet_bootstrap.cc:492] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:33.032239 21559 tablet_bootstrap.cc:654] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.030615 21332 leader_election.cc:304] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:33.039182 21558 tablet_bootstrap.cc:654] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.040879 21583 raft_consensus.cc:2804] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.042052 21557 raft_consensus.cc:359] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.042780 21557 raft_consensus.cc:385] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.043061 21557 raft_consensus.cc:740] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.043886 21557 consensus_queue.cc:260] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.047408 21557 ts_tablet_manager.cc:1434] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:04:33.048583 21557 tablet_bootstrap.cc:492] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:33.043053 21583 raft_consensus.cc:697] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:33.050451 21583 consensus_queue.cc:237] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.059729 21558 tablet_bootstrap.cc:492] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:33.060359 21558 ts_tablet_manager.cc:1403] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.006s
I20260709 13:04:33.062667 21557 tablet_bootstrap.cc:654] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.063822 21558 raft_consensus.cc:359] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.064754 21558 raft_consensus.cc:385] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.065128 21558 raft_consensus.cc:740] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.069487 21558 consensus_queue.cc:260] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.073547 21603 raft_consensus.cc:493] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.073429 21265 catalog_manager.cc:5697] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 0 to 1, leader changed from <none> to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 1 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.074084 21603 raft_consensus.cc:515] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.076822 21603 leader_election.cc:290] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.078078 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.078531 21558 ts_tablet_manager.cc:1434] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:04:33.078722 21368 raft_consensus.cc:2468] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 0.
I20260709 13:04:33.079468 21558 tablet_bootstrap.cc:492] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b: Bootstrap starting.
I20260709 13:04:33.080135 21483 leader_election.cc:304] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:33.080888 21603 raft_consensus.cc:2804] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.081275 21603 raft_consensus.cc:493] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.081324 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:33.081705 21603 raft_consensus.cc:3060] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.082188 21443 raft_consensus.cc:2468] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 0.
I20260709 13:04:33.093199 21557 tablet_bootstrap.cc:492] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:33.093114 21603 raft_consensus.cc:515] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.093804 21557 ts_tablet_manager.cc:1403] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.045s	user 0.024s	sys 0.001s
I20260709 13:04:33.096402 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.097069 21443 raft_consensus.cc:3060] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.096792 21557 raft_consensus.cc:359] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.097518 21557 raft_consensus.cc:385] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.097800 21557 raft_consensus.cc:740] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.098630 21558 tablet_bootstrap.cc:654] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.099409 21565 raft_consensus.cc:493] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.100201 21559 tablet_bootstrap.cc:492] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:33.099850 21565 raft_consensus.cc:515] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.101111 21603 leader_election.cc:290] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.102319 21565 leader_election.cc:290] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.103462 21559 ts_tablet_manager.cc:1403] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.078s	user 0.032s	sys 0.013s
I20260709 13:04:33.103627 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:33.104586 21557 consensus_queue.cc:260] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.104810 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.105652 21367 raft_consensus.cc:2468] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.106177 21559 raft_consensus.cc:359] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.106848 21559 raft_consensus.cc:385] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.102339 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.107368 21559 raft_consensus.cc:740] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.107775 21368 raft_consensus.cc:3060] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.108135 21559 consensus_queue.cc:260] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.108834 21557 ts_tablet_manager.cc:1434] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:04:33.109836 21557 tablet_bootstrap.cc:492] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d: Bootstrap starting.
I20260709 13:04:33.111876 21409 leader_election.cc:304] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.122747 21559 ts_tablet_manager.cc:1434] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:04:33.123709 21559 tablet_bootstrap.cc:492] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:33.127875 21517 raft_consensus.cc:2468] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.128051 21368 raft_consensus.cc:2468] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 1.
I20260709 13:04:33.130383 21584 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.130636 21263 catalog_manager.cc:5697] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.132223 21565 raft_consensus.cc:2804] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.134454 21443 raft_consensus.cc:2468] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 1.
I20260709 13:04:33.138993 21565 raft_consensus.cc:493] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.139417 21565 raft_consensus.cc:3060] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.139750 21481 leader_election.cc:304] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.137375 21584 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.142635 21584 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.143169 21603 raft_consensus.cc:2804] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.143890 21603 raft_consensus.cc:697] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:33.144492 21583 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.144549 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.144707 21603 consensus_queue.cc:237] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.145099 21583 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.146116 21367 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.149096 21558 tablet_bootstrap.cc:492] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b: No bootstrap required, opened a new log
I20260709 13:04:33.149605 21558 ts_tablet_manager.cc:1403] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.028s
I20260709 13:04:33.150866 21409 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.151937 21584 raft_consensus.cc:2804] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.152298 21584 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.152486 21559 tablet_bootstrap.cc:654] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.152558 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:33.152649 21584 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.155835 21583 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.155833 21557 tablet_bootstrap.cc:654] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.156246 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:33.157029 21558 raft_consensus.cc:359] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.157673 21558 raft_consensus.cc:385] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.158046 21558 raft_consensus.cc:740] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.157636 21516 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 0.
I20260709 13:04:33.158830 21558 consensus_queue.cc:260] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.160635 21558 ts_tablet_manager.cc:1434] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:04:33.161720 21331 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:33.162575 21583 raft_consensus.cc:2804] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.162919 21583 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.154701 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:33.163468 21583 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.165889 21517 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.169584 21565 raft_consensus.cc:515] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.170400 21584 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.171564 21443 raft_consensus.cc:2393] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30816458075e446eada4afc2f0e2796d in current term 1: Already voted for candidate c7f1455a61774c4fb7d085c735aa4c3b in this term.
I20260709 13:04:33.172506 21584 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.174599 21565 leader_election.cc:290] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.174270 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.177183 21367 raft_consensus.cc:3060] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.175288 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.179579 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.180589 21516 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.177886 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.186977 21583 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.189517 21368 raft_consensus.cc:2393] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7f1455a61774c4fb7d085c735aa4c3b in current term 1: Already voted for candidate 30816458075e446eada4afc2f0e2796d in this term.
I20260709 13:04:33.189119 21517 raft_consensus.cc:3060] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.195603 21583 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.196749 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.197726 21443 raft_consensus.cc:2393] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 30816458075e446eada4afc2f0e2796d in current term 1: Already voted for candidate c7f1455a61774c4fb7d085c735aa4c3b in this term.
I20260709 13:04:33.202445 21557 tablet_bootstrap.cc:492] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d: No bootstrap required, opened a new log
I20260709 13:04:33.202987 21557 ts_tablet_manager.cc:1403] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.003s
I20260709 13:04:33.204216 21515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.203347 21263 catalog_manager.cc:5697] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.205641 21559 tablet_bootstrap.cc:492] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:33.206131 21559 ts_tablet_manager.cc:1403] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.083s	user 0.015s	sys 0.003s
I20260709 13:04:33.206184 21367 raft_consensus.cc:2468] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.206664 21331 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30816458075e446eada4afc2f0e2796d; no voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:33.207556 21583 raft_consensus.cc:2749] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.207597 21409 leader_election.cc:304] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.208422 21565 raft_consensus.cc:2804] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.208973 21565 raft_consensus.cc:697] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:33.209944 21517 raft_consensus.cc:2468] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.210009 21516 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.209924 21565 consensus_queue.cc:237] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.213766 21406 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 30816458075e446eada4afc2f0e2796d
I20260709 13:04:33.215022 21584 raft_consensus.cc:2804] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.215528 21584 raft_consensus.cc:697] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:33.215921 21557 raft_consensus.cc:359] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.216698 21557 raft_consensus.cc:385] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.216421 21584 consensus_queue.cc:237] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.216974 21557 raft_consensus.cc:740] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.217895 21557 consensus_queue.cc:260] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.221289 21559 raft_consensus.cc:359] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.223511 21559 raft_consensus.cc:385] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.223806 21559 raft_consensus.cc:740] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.224018 21603 raft_consensus.cc:493] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.224466 21559 consensus_queue.cc:260] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.225454 21564 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.225939 21564 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.227298 21557 ts_tablet_manager.cc:1434] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.004s
I20260709 13:04:33.227975 21564 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.229099 21584 raft_consensus.cc:493] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.229387 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.229732 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:33.229671 21584 raft_consensus.cc:515] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.230274 21367 raft_consensus.cc:2393] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 344ed664705b49dfa97dee21a1b70411 in current term 1: Already voted for candidate c7f1455a61774c4fb7d085c735aa4c3b in this term.
I20260709 13:04:33.233045 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.233379 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:33.225479 21603 raft_consensus.cc:515] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.234091 21516 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.234128 21483 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411; no voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:33.234453 21368 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.235374 21564 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.235507 21406 leader_election.cc:304] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.232707 21263 catalog_manager.cc:5697] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 1 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.236439 21565 raft_consensus.cc:2804] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.236824 21565 raft_consensus.cc:493] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.237176 21565 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.238083 21368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.238276 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:33.238942 21443 raft_consensus.cc:2393] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 344ed664705b49dfa97dee21a1b70411 in current term 1: Already voted for candidate 30816458075e446eada4afc2f0e2796d in this term.
I20260709 13:04:33.240074 21481 leader_election.cc:304] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411; no voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:33.242964 21584 leader_election.cc:290] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.244136 21603 leader_election.cc:290] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.245177 21603 raft_consensus.cc:3060] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.249574 21565 raft_consensus.cc:515] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.252161 21559 ts_tablet_manager.cc:1434] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:04:33.255517 21266 catalog_manager.cc:5697] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 1 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.257742 21603 raft_consensus.cc:2749] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.257961 21559 tablet_bootstrap.cc:492] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411: Bootstrap starting.
I20260709 13:04:33.252298 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.258939 21367 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.253165 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.261241 21516 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.263763 21559 tablet_bootstrap.cc:654] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.265812 21367 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.253466 21565 leader_election.cc:290] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.268146 21516 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.268635 21409 leader_election.cc:304] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.255060 21564 raft_consensus.cc:2749] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.270407 21565 raft_consensus.cc:2804] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.271046 21565 raft_consensus.cc:697] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:33.272164 21565 consensus_queue.cc:237] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.275009 21559 tablet_bootstrap.cc:492] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411: No bootstrap required, opened a new log
I20260709 13:04:33.275481 21559 ts_tablet_manager.cc:1403] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.005s
I20260709 13:04:33.278026 21559 raft_consensus.cc:359] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.278519 21559 raft_consensus.cc:385] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.278749 21559 raft_consensus.cc:740] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.279440 21559 consensus_queue.cc:260] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.281499 21559 ts_tablet_manager.cc:1434] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:33.280848 21263 catalog_manager.cc:5697] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 1 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.282842 21565 raft_consensus.cc:493] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.283315 21565 raft_consensus.cc:515] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.285826 21565 leader_election.cc:290] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.286541 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.287156 21367 raft_consensus.cc:2468] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.288058 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411" is_pre_election: true
I20260709 13:04:33.288177 21409 leader_election.cc:304] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.288775 21516 raft_consensus.cc:2468] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 0.
I20260709 13:04:33.289710 21565 raft_consensus.cc:2804] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.290073 21565 raft_consensus.cc:493] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.290500 21565 raft_consensus.cc:3060] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.296480 21565 raft_consensus.cc:515] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.298266 21565 leader_election.cc:290] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.299108 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.299252 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.299790 21367 raft_consensus.cc:3060] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.299836 21516 raft_consensus.cc:3060] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.305048 21516 raft_consensus.cc:2468] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.304922 21367 raft_consensus.cc:2468] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 1.
I20260709 13:04:33.306308 21409 leader_election.cc:304] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.306977 21565 raft_consensus.cc:2804] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.307327 21565 raft_consensus.cc:697] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:33.308076 21565 consensus_queue.cc:237] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.316546 21266 catalog_manager.cc:5697] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 1 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.317585 21584 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.321271 21584 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.376462 21564 raft_consensus.cc:493] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.377080 21564 raft_consensus.cc:515] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.378794 21564 leader_election.cc:290] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.379714 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c264fe14fc1847eeb087cc1fef0c5aec" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d" is_pre_election: true
I20260709 13:04:33.379848 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c264fe14fc1847eeb087cc1fef0c5aec" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" is_pre_election: true
I20260709 13:04:33.380445 21367 raft_consensus.cc:2468] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 0.
I20260709 13:04:33.380542 21443 raft_consensus.cc:2468] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 0.
I20260709 13:04:33.381870 21483 leader_election.cc:304] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:33.382658 21564 raft_consensus.cc:2804] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.383064 21564 raft_consensus.cc:493] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.383368 21564 raft_consensus.cc:3060] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.388907 21564 raft_consensus.cc:515] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.390614 21564 leader_election.cc:290] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:33.391404 21367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c264fe14fc1847eeb087cc1fef0c5aec" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:33.391561 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c264fe14fc1847eeb087cc1fef0c5aec" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.391994 21367 raft_consensus.cc:3060] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.392021 21443 raft_consensus.cc:3060] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.399036 21367 raft_consensus.cc:2468] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 1.
I20260709 13:04:33.399047 21443 raft_consensus.cc:2468] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 1.
I20260709 13:04:33.400557 21483 leader_election.cc:304] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:33.401489 21564 raft_consensus.cc:2804] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.401963 21564 raft_consensus.cc:697] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:33.402781 21564 consensus_queue.cc:237] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.410032 21263 catalog_manager.cc:5697] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.415608 21583 consensus_queue.cc:1048] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.427094 21583 consensus_queue.cc:1048] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.464362 21516 raft_consensus.cc:3060] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.470371 21626 consensus_queue.cc:1048] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.480590 21626 consensus_queue.cc:1048] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.542042 21516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:49340
I20260709 13:04:33.542536 21516 raft_consensus.cc:606] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:33.555230 21564 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.557300 21443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:55860
I20260709 13:04:33.557912 21443 raft_consensus.cc:606] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:33.566203 21564 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.567521 21443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:55860
I20260709 13:04:33.568133 21443 raft_consensus.cc:606] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:33.581959 21442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:55860
I20260709 13:04:33.582491 21442 raft_consensus.cc:606] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:33.587502 21516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
mode: GRACEFUL
new_leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:49340
I20260709 13:04:33.588061 21516 raft_consensus.cc:606] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Received request to transfer leadership to TS c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:33.607565 21368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:57502
I20260709 13:04:33.608193 21368 raft_consensus.cc:606] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Received request to transfer leadership to TS c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:33.608770 21564 consensus_queue.cc:1048] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.612363 21565 consensus_queue.cc:1048] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.619062 21564 raft_consensus.cc:993] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.619532 21564 raft_consensus.cc:1081] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.622604 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57494
I20260709 13:04:33.626745 21366 raft_consensus.cc:493] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:33.627163 21366 raft_consensus.cc:3060] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.628346 21582 consensus_queue.cc:1048] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:33.631438 21615 consensus_queue.cc:1048] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:33.634380 21366 raft_consensus.cc:515] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.647790 21366 leader_election.cc:290] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.649369 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.650020 21516 raft_consensus.cc:3055] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:33.650120 21442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6183ad54ee054404aa4ac5d460cdfbe4" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.650425 21516 raft_consensus.cc:740] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:33.650751 21442 raft_consensus.cc:3060] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.651465 21516 consensus_queue.cc:260] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.652815 21516 raft_consensus.cc:3060] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.658052 21442 raft_consensus.cc:2468] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:33.659319 21516 raft_consensus.cc:2468] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:33.659201 21332 leader_election.cc:304] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.660069 21626 raft_consensus.cc:2804] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.660528 21626 raft_consensus.cc:697] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:33.661235 21626 consensus_queue.cc:237] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.666872 21582 consensus_queue.cc:1048] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.668076 21266 catalog_manager.cc:5697] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 1 to 2, leader changed from 344ed664705b49dfa97dee21a1b70411 (127.20.66.131) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 2 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.671690 21582 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.684453 21582 consensus_queue.cc:1048] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:33.690474 21584 raft_consensus.cc:993] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.690963 21584 raft_consensus.cc:1081] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.694303 21368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57484
I20260709 13:04:33.694959 21368 raft_consensus.cc:493] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:33.694954 21565 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:33.695538 21368 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.704285 21368 raft_consensus.cc:515] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.707449 21442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.708129 21442 raft_consensus.cc:3055] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:33.708448 21442 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:33.708612 21565 raft_consensus.cc:993] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.709590 21582 raft_consensus.cc:1081] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.709388 21442 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.710326 21442 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.713526 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57484
I20260709 13:04:33.714104 21366 raft_consensus.cc:493] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:33.714483 21366 raft_consensus.cc:3060] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.719504 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.720191 21368 leader_election.cc:290] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.720902 21516 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.721206 21442 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:33.721319 21366 raft_consensus.cc:515] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.722565 21332 leader_election.cc:304] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:33.723713 21366 leader_election.cc:290] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:33.723841 21626 raft_consensus.cc:2804] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.724284 21626 raft_consensus.cc:697] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:33.724905 21626 consensus_queue.cc:237] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.728631 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:33.728078 21516 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:33.729215 21517 raft_consensus.cc:3060] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.732844 21442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d62f7db1ffb4fdc850f526801d775be" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:33.733484 21442 raft_consensus.cc:3055] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:33.733803 21442 raft_consensus.cc:740] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:33.734365 21517 raft_consensus.cc:2468] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:33.734587 21442 consensus_queue.cc:260] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.735473 21331 leader_election.cc:304] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:33.735864 21442 raft_consensus.cc:3060] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:33.736137 21626 raft_consensus.cc:2804] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:33.736645 21626 raft_consensus.cc:697] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:33.737495 21626 consensus_queue.cc:237] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:33.738796 21264 catalog_manager.cc:5697] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 1 to 2, leader changed from c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 2 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.742657 21442 raft_consensus.cc:2468] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:33.752413 21264 catalog_manager.cc:5697] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 1 to 2, leader changed from c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 2 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.860083 21564 consensus_queue.cc:1048] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.860656 21584 consensus_queue.cc:1048] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.876255 21584 consensus_queue.cc:1048] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.881130 21584 raft_consensus.cc:993] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.882746 21582 raft_consensus.cc:1081] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:33.907969 21603 consensus_queue.cc:1048] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:04:34.005957 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57484
I20260709 13:04:34.006731 21366 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.007205 21366 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.015280 21366 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.018532 21366 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:34.025794 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:34.026870 21517 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.033226 21442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:34.033447 21517 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:34.033812 21442 raft_consensus.cc:3055] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.034132 21442 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:34.034601 21331 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:34.034983 21442 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.036192 21442 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.038012 21649 raft_consensus.cc:2804] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.038609 21649 raft_consensus.cc:697] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:34.039402 21649 consensus_queue.cc:237] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.047672 21603 raft_consensus.cc:993] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: : Instructing follower c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:34.048185 21603 raft_consensus.cc:1081] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Signalling peer c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:34.049113 21442 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:34.051048 21441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:55832
I20260709 13:04:34.051617 21441 raft_consensus.cc:493] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.052004 21441 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.058468 21650 raft_consensus.cc:993] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: : Instructing follower c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:34.059185 21651 raft_consensus.cc:1081] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Signalling peer c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:34.061208 21442 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:55844
I20260709 13:04:34.062117 21442 raft_consensus.cc:493] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.062078 21441 raft_consensus.cc:515] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.062697 21442 raft_consensus.cc:3060] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.064509 21441 leader_election.cc:290] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 2 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:34.065223 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:34.065704 21366 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.068609 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:34.069226 21517 raft_consensus.cc:3055] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.069535 21517 raft_consensus.cc:740] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:34.070310 21517 consensus_queue.cc:260] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.070721 21442 raft_consensus.cc:515] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.071411 21517 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.072844 21366 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 2.
I20260709 13:04:34.073351 21442 leader_election.cc:290] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 2 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:34.074311 21516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:34.075219 21516 raft_consensus.cc:3060] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.075810 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73d334b1ef14d248f3ae2145223dcf6" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:34.075941 21409 leader_election.cc:304] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:34.076437 21366 raft_consensus.cc:3055] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.076758 21366 raft_consensus.cc:740] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:34.078321 21366 consensus_queue.cc:260] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.078176 21648 raft_consensus.cc:2804] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.079736 21648 raft_consensus.cc:697] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:34.080492 21366 raft_consensus.cc:3060] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.080660 21648 consensus_queue.cc:237] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.088259 21517 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 2.
I20260709 13:04:34.090292 21366 raft_consensus.cc:2468] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 2.
I20260709 13:04:34.092760 21409 leader_election.cc:304] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:34.093924 21648 raft_consensus.cc:2804] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.094770 21648 raft_consensus.cc:697] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:34.101459 21648 consensus_queue.cc:237] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:34.113631 21516 raft_consensus.cc:2468] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 2.
I20260709 13:04:34.131709 21266 catalog_manager.cc:5697] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 1 to 2, leader changed from c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 2 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.142874 21263 catalog_manager.cc:5697] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 1 to 2, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 2 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.144142 21263 catalog_manager.cc:5697] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 1 to 2, leader changed from 344ed664705b49dfa97dee21a1b70411 (127.20.66.131) to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 2 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.162370 21443 raft_consensus.cc:1275] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.163774 21649 consensus_queue.cc:1048] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.171631 21516 raft_consensus.cc:1275] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.173204 21649 consensus_queue.cc:1048] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.209525 21443 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.210914 21649 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.219132 21516 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.220932 21626 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.233911 21443 raft_consensus.cc:1275] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.235184 21626 consensus_queue.cc:1048] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.243125 21516 raft_consensus.cc:1275] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.244865 21651 consensus_queue.cc:1048] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:34.432366 21517 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.433985 21651 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.442541 21443 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.444229 21651 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.601320 21366 raft_consensus.cc:1275] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.602731 21648 consensus_queue.cc:1048] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.614555 21368 raft_consensus.cc:1275] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.615664 21517 raft_consensus.cc:1275] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.616631 21584 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.617123 21648 consensus_queue.cc:1048] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:34.632400 21517 raft_consensus.cc:1275] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:34.634095 21584 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:34.918916 21297 debug-util.cc:398] Leaking SignalData structure 0x7b0800056d00 after lost signal to thread 20749
W20260709 13:04:34.919813 21297 debug-util.cc:398] Leaking SignalData structure 0x7b08000543e0 after lost signal to thread 21300
W20260709 13:04:34.920904 21297 debug-util.cc:398] Leaking SignalData structure 0x7b0800058540 after lost signal to thread 21392
W20260709 13:04:34.922111 21297 debug-util.cc:398] Leaking SignalData structure 0x7b080009b240 after lost signal to thread 21467
W20260709 13:04:34.923038 21297 debug-util.cc:398] Leaking SignalData structure 0x7b0800098460 after lost signal to thread 21541
I20260709 13:04:36.737049 20746 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:04:36.738066 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:36.763113 21366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:57508
I20260709 13:04:36.763818 21366 raft_consensus.cc:606] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Received request to transfer leadership to TS 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:36.766880 21366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:57508
I20260709 13:04:36.767549 21366 raft_consensus.cc:606] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Received request to transfer leadership to TS 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:36.768764 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:36.769394 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:36.872556 21659 raft_consensus.cc:993] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: : Instructing follower 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:36.873128 21659 raft_consensus.cc:1081] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Signalling peer 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:36.875222 21517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:49328
I20260709 13:04:36.875927 21517 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:36.876708 21517 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.884845 21517 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:36.887954 21517 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 3 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:36.888345 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:36.888458 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:36.889249 21366 raft_consensus.cc:3055] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:36.889461 21443 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.889750 21366 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:36.890815 21366 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:36.891916 21366 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:36.897192 21366 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 3.
I20260709 13:04:36.898835 21443 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 3.
I20260709 13:04:36.898864 21483 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:36.900164 21564 raft_consensus.cc:2804] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:36.901654 21564 raft_consensus.cc:697] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:36.902438 21564 consensus_queue.cc:237] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:36.911901 21263 catalog_manager.cc:5697] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 2 to 3, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.171257 21662 raft_consensus.cc:993] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: : Instructing follower 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:37.171742 21659 raft_consensus.cc:1081] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Signalling peer 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:37.173272 21517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:49328
I20260709 13:04:37.173781 21517 raft_consensus.cc:493] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:37.174083 21517 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.178896 21517 raft_consensus.cc:515] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.180567 21517 leader_election.cc:290] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 3 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:37.181458 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:37.181617 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:37.182046 21366 raft_consensus.cc:3055] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:37.182253 21443 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.182463 21366 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:37.183369 21366 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.184257 21366 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.188872 21366 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 3.
I20260709 13:04:37.189994 21483 leader_election.cc:304] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:37.190003 21443 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 3.
I20260709 13:04:37.190860 21564 raft_consensus.cc:2804] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:37.191424 21564 raft_consensus.cc:697] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:37.192260 21564 consensus_queue.cc:237] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.203578 21263 catalog_manager.cc:5697] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 2 to 3, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.338070 21366 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:37.339843 21564 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:37.349077 21443 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:37.352452 21671 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:37.685124 21443 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:37.686466 21671 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:37.702001 21366 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:37.703827 21564 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:37.770359 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:37.788168 21517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:49340
I20260709 13:04:37.788844 21517 raft_consensus.cc:606] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:37.801312 21517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:49340
I20260709 13:04:37.801858 21517 raft_consensus.cc:606] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:37.807288 21443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:55860
I20260709 13:04:37.807886 21443 raft_consensus.cc:606] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:37.813565 21443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:55860
I20260709 13:04:37.814217 21443 raft_consensus.cc:606] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:37.819958 21443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
mode: GRACEFUL
new_leader_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:55860
I20260709 13:04:37.820508 21443 raft_consensus.cc:606] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Received request to transfer leadership to TS 30816458075e446eada4afc2f0e2796d
I20260709 13:04:37.897655 21663 raft_consensus.cc:993] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:37.898175 21564 raft_consensus.cc:1081] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:37.899725 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57494
I20260709 13:04:37.900264 21366 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:37.900570 21366 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.907575 21366 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.910169 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:37.910341 21366 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 4 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:37.910571 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:37.910847 21443 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.911202 21517 raft_consensus.cc:3055] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:37.911935 21517 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:37.913133 21517 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.914661 21517 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.918322 21443 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 4.
I20260709 13:04:37.919483 21332 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:37.920396 21678 raft_consensus.cc:2804] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:37.920652 21517 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 4.
I20260709 13:04:37.920871 21678 raft_consensus.cc:697] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:37.921823 21678 consensus_queue.cc:237] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.929394 21263 catalog_manager.cc:5697] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 3 to 4, leader changed from 344ed664705b49dfa97dee21a1b70411 (127.20.66.131) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 4 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.930384 21584 raft_consensus.cc:993] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:37.930855 21676 raft_consensus.cc:1081] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:37.932549 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57484
I20260709 13:04:37.933107 21366 raft_consensus.cc:493] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:37.933470 21366 raft_consensus.cc:3060] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:37.940255 21366 raft_consensus.cc:515] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.943025 21366 leader_election.cc:290] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:37.943693 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:37.943739 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abdcd6cedd2745ab8091c067e79fee6a" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:37.944443 21443 raft_consensus.cc:3055] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:37.944478 21517 raft_consensus.cc:3060] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:37.944877 21443 raft_consensus.cc:740] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:37.945694 21443 consensus_queue.cc:260] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.946798 21443 raft_consensus.cc:3060] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:37.951713 21517 raft_consensus.cc:2468] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:37.953121 21331 leader_election.cc:304] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:37.953938 21678 raft_consensus.cc:2804] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:37.953920 21443 raft_consensus.cc:2468] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:37.954584 21678 raft_consensus.cc:697] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:37.955366 21678 consensus_queue.cc:237] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:37.964000 21263 catalog_manager.cc:5697] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 1 to 2, leader changed from c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 2 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.139147 21671 raft_consensus.cc:993] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:38.139674 21564 raft_consensus.cc:1081] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:38.141342 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57494
I20260709 13:04:38.141858 21366 raft_consensus.cc:493] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:38.142215 21366 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:38.147677 21366 raft_consensus.cc:515] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.149506 21366 leader_election.cc:290] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 4 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:38.150255 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:38.150460 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:38.150803 21443 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:38.150926 21517 raft_consensus.cc:3055] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:38.151278 21517 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:38.152041 21517 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.153389 21517 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:38.157410 21443 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 4.
I20260709 13:04:38.158586 21332 leader_election.cc:304] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:38.159276 21517 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 4.
I20260709 13:04:38.159384 21678 raft_consensus.cc:2804] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:38.159947 21678 raft_consensus.cc:697] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:38.160724 21678 consensus_queue.cc:237] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.166622 21263 catalog_manager.cc:5697] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 3 to 4, leader changed from 344ed664705b49dfa97dee21a1b70411 (127.20.66.131) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 4 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.180240 21584 raft_consensus.cc:993] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:38.180675 21676 raft_consensus.cc:1081] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:38.182369 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57484
I20260709 13:04:38.182891 21366 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:38.183233 21366 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:38.187695 21366 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.189347 21366 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:38.190062 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:38.190445 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:38.190609 21443 raft_consensus.cc:3055] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:38.190974 21517 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:38.190990 21443 raft_consensus.cc:740] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:38.191995 21443 consensus_queue.cc:260] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.192793 21443 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:38.196246 21517 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:38.196871 21443 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 2.
I20260709 13:04:38.197340 21331 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:38.198299 21678 raft_consensus.cc:2804] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:38.198734 21678 raft_consensus.cc:697] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:38.199641 21678 consensus_queue.cc:237] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.206612 21584 raft_consensus.cc:993] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: : Instructing follower 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:38.207069 21676 raft_consensus.cc:1081] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Signalling peer 30816458075e446eada4afc2f0e2796d to start an election
I20260709 13:04:38.207005 21263 catalog_manager.cc:5697] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 1 to 2, leader changed from c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 2 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.208935 21366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
 from {username='slave'} at 127.0.0.1:57484
I20260709 13:04:38.209478 21366 raft_consensus.cc:493] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:38.209822 21366 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:38.214485 21366 raft_consensus.cc:515] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.216136 21366 leader_election.cc:290] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 3 election: Requested vote from peers c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:38.217146 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:38.217311 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "30816458075e446eada4afc2f0e2796d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:38.217685 21517 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:38.217924 21443 raft_consensus.cc:3055] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:38.218314 21443 raft_consensus.cc:740] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:38.219089 21443 consensus_queue.cc:260] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.219939 21443 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:38.222179 21517 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 3.
I20260709 13:04:38.222859 21516 raft_consensus.cc:1240] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Rejecting Update request from peer c7f1455a61774c4fb7d085c735aa4c3b for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:38.223187 21331 leader_election.cc:304] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30816458075e446eada4afc2f0e2796d, 344ed664705b49dfa97dee21a1b70411; no voters: 
I20260709 13:04:38.224026 21678 raft_consensus.cc:2804] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:38.224475 21678 raft_consensus.cc:697] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 LEADER]: Becoming Leader. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:38.225677 21443 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30816458075e446eada4afc2f0e2796d in term 3.
I20260709 13:04:38.225415 21678 consensus_queue.cc:237] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:38.231930 21263 catalog_manager.cc:5697] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d reported cstate change: term changed from 2 to 3, leader changed from c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130) to 30816458075e446eada4afc2f0e2796d (127.20.66.129). New cstate: current_term: 3 leader_uuid: "30816458075e446eada4afc2f0e2796d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.358131 21443 raft_consensus.cc:1275] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.359455 21678 consensus_queue.cc:1048] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:38.369081 21516 raft_consensus.cc:1275] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.370729 21678 consensus_queue.cc:1048] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:38.489370 21516 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:38.490712 21678 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:38.500252 21443 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:38.502032 21692 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:38.625449 21443 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:38.626776 21692 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:38.639549 21516 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:38.641258 21659 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:38.698024 21516 raft_consensus.cc:1275] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.699398 21659 consensus_queue.cc:1048] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:38.708135 21443 raft_consensus.cc:1275] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.710511 21659 consensus_queue.cc:1048] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:38.783355 21443 raft_consensus.cc:1275] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:38.785760 21659 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:38.800642 21516 raft_consensus.cc:1275] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Refusing update from remote peer 30816458075e446eada4afc2f0e2796d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:38.803221 21694 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:40.830627 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:40.842530 21366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:57508
I20260709 13:04:40.843611 21366 raft_consensus.cc:606] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 LEADER]: Received request to transfer leadership to TS 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:40.846791 21366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:57508
I20260709 13:04:40.847322 21366 raft_consensus.cc:606] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Received request to transfer leadership to TS c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:40.849992 21366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:57508
I20260709 13:04:40.850466 21366 raft_consensus.cc:606] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Received request to transfer leadership to TS 344ed664705b49dfa97dee21a1b70411
I20260709 13:04:40.852803 21366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "30816458075e446eada4afc2f0e2796d"
mode: GRACEFUL
new_leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:57508
I20260709 13:04:40.853272 21366 raft_consensus.cc:606] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Received request to transfer leadership to TS c7f1455a61774c4fb7d085c735aa4c3b
I20260709 13:04:40.854281 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:40.854887 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:40.888188 21700 raft_consensus.cc:993] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: : Instructing follower c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:40.888859 21700 raft_consensus.cc:1081] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Signalling peer c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:40.890473 21443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:55844
I20260709 13:04:40.891150 21443 raft_consensus.cc:493] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:40.891528 21443 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.897693 21443 raft_consensus.cc:515] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:40.899360 21443 leader_election.cc:290] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 3 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:40.900507 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:40.900617 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442f9ad084ff4c8b8794e7291eae0a2b" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:40.901175 21366 raft_consensus.cc:3055] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:40.901276 21517 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.901480 21366 raft_consensus.cc:740] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:40.902276 21366 consensus_queue.cc:260] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:40.903322 21366 raft_consensus.cc:3060] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.910184 21517 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 3.
I20260709 13:04:40.910660 21366 raft_consensus.cc:2468] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 3.
I20260709 13:04:40.911444 21406 leader_election.cc:304] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:40.912361 21676 raft_consensus.cc:2804] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:40.913812 21676 raft_consensus.cc:697] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 3 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:40.914877 21676 consensus_queue.cc:237] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:40.923170 21263 catalog_manager.cc:5697] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 2 to 3, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 3 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.000340 21702 raft_consensus.cc:993] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: : Instructing follower 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:41.000795 21700 raft_consensus.cc:1081] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 LEADER]: Signalling peer 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:41.002257 21517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:49328
I20260709 13:04:41.002736 21517 raft_consensus.cc:493] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:41.003006 21517 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.007786 21517 raft_consensus.cc:515] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.009414 21517 leader_election.cc:290] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 4 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:41.010471 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:41.010397 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06afeced037f44c799edc35d5b8c4dcd" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:41.011054 21443 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.011080 21366 raft_consensus.cc:3055] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:41.011520 21366 raft_consensus.cc:740] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:41.012281 21366 consensus_queue.cc:260] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.013402 21366 raft_consensus.cc:3060] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.017771 21443 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 4.
I20260709 13:04:41.018877 21702 raft_consensus.cc:993] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: : Instructing follower 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:41.018978 21481 leader_election.cc:304] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:41.019374 21702 raft_consensus.cc:1081] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Signalling peer 344ed664705b49dfa97dee21a1b70411 to start an election
I20260709 13:04:41.019971 21704 raft_consensus.cc:2804] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:41.020090 21366 raft_consensus.cc:2468] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 4.
I20260709 13:04:41.021381 21704 raft_consensus.cc:697] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 4 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:41.022049 21517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e"
dest_uuid: "344ed664705b49dfa97dee21a1b70411"
 from {username='slave'} at 127.0.0.1:49328
I20260709 13:04:41.022167 21704 consensus_queue.cc:237] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.022563 21517 raft_consensus.cc:493] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:41.022905 21517 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.028266 21517 raft_consensus.cc:515] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.029699 21517 leader_election.cc:290] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 5 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130:40509)
I20260709 13:04:41.030046 21263 catalog_manager.cc:5697] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 3 to 4, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:41.030654 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
I20260709 13:04:41.030839 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "480c92b5319a4566b45b409b138eac0e" candidate_uuid: "344ed664705b49dfa97dee21a1b70411" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:41.031189 21443 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.031286 21366 raft_consensus.cc:3055] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:41.031616 21366 raft_consensus.cc:740] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:41.032318 21366 consensus_queue.cc:260] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.033486 21366 raft_consensus.cc:3060] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.037504 21443 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 5.
I20260709 13:04:41.038529 21481 leader_election.cc:304] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:41.039281 21704 raft_consensus.cc:2804] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:41.039527 21366 raft_consensus.cc:2468] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344ed664705b49dfa97dee21a1b70411 in term 5.
I20260709 13:04:41.039780 21704 raft_consensus.cc:697] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 5 LEADER]: Becoming Leader. State: Replica: 344ed664705b49dfa97dee21a1b70411, State: Running, Role: LEADER
I20260709 13:04:41.040664 21704 consensus_queue.cc:237] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.047570 21263 catalog_manager.cc:5697] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 reported cstate change: term changed from 4 to 5, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to 344ed664705b49dfa97dee21a1b70411 (127.20.66.131). New cstate: current_term: 5 leader_uuid: "344ed664705b49dfa97dee21a1b70411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:41.161530 21700 raft_consensus.cc:993] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: : Instructing follower c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:41.162041 21702 raft_consensus.cc:1081] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Signalling peer c7f1455a61774c4fb7d085c735aa4c3b to start an election
I20260709 13:04:41.163355 21443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5"
dest_uuid: "c7f1455a61774c4fb7d085c735aa4c3b"
 from {username='slave'} at 127.0.0.1:55844
I20260709 13:04:41.163846 21443 raft_consensus.cc:493] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:41.164135 21443 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.168540 21443 raft_consensus.cc:515] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.170045 21443 leader_election.cc:290] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 5 election: Requested vote from peers 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903), 344ed664705b49dfa97dee21a1b70411 (127.20.66.131:41469)
I20260709 13:04:41.171041 21517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "344ed664705b49dfa97dee21a1b70411"
I20260709 13:04:41.171041 21366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9e67348632445a92364fcca02e5ea5" candidate_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "30816458075e446eada4afc2f0e2796d"
I20260709 13:04:41.171798 21517 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.171928 21366 raft_consensus.cc:3055] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:41.172251 21366 raft_consensus.cc:740] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 30816458075e446eada4afc2f0e2796d, State: Running, Role: LEADER
I20260709 13:04:41.172940 21366 consensus_queue.cc:260] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.174199 21366 raft_consensus.cc:3060] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.177426 21517 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 5.
I20260709 13:04:41.178390 21406 leader_election.cc:304] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344ed664705b49dfa97dee21a1b70411, c7f1455a61774c4fb7d085c735aa4c3b; no voters: 
I20260709 13:04:41.179036 21676 raft_consensus.cc:2804] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:41.179486 21676 raft_consensus.cc:697] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 5 LEADER]: Becoming Leader. State: Replica: c7f1455a61774c4fb7d085c735aa4c3b, State: Running, Role: LEADER
I20260709 13:04:41.180043 21366 raft_consensus.cc:2468] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7f1455a61774c4fb7d085c735aa4c3b in term 5.
I20260709 13:04:41.180418 21676 consensus_queue.cc:237] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } }
I20260709 13:04:41.187004 21263 catalog_manager.cc:5697] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b reported cstate change: term changed from 4 to 5, leader changed from 30816458075e446eada4afc2f0e2796d (127.20.66.129) to c7f1455a61774c4fb7d085c735aa4c3b (127.20.66.130). New cstate: current_term: 5 leader_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.299381 21516 raft_consensus.cc:1275] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:41.300581 21676 consensus_queue.cc:1048] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:41.313426 21366 raft_consensus.cc:1275] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:41.314841 21676 consensus_queue.cc:1048] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:41.438508 21443 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:41.439742 21704 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:41.443426 21366 raft_consensus.cc:1275] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:41.445093 21704 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:41.451352 21368 raft_consensus.cc:1275] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:41.453450 21704 consensus_queue.cc:1048] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:41.471431 21443 raft_consensus.cc:1275] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Refusing update from remote peer 344ed664705b49dfa97dee21a1b70411: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:41.475821 21711 consensus_queue.cc:1048] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1455a61774c4fb7d085c735aa4c3b" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:41.651088 21516 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:41.652213 21676 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "344ed664705b49dfa97dee21a1b70411" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:41.661047 21366 raft_consensus.cc:1275] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Refusing update from remote peer c7f1455a61774c4fb7d085c735aa4c3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:41.662648 21728 consensus_queue.cc:1048] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30816458075e446eada4afc2f0e2796d" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:41.855820 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:41.861761 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
W20260709 13:04:41.883934 21409 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:04:41.887377 21409 consensus_peers.cc:597] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b -> Peer 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903): Couldn't send request to peer 30816458075e446eada4afc2f0e2796d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.897315 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:41.898471 20746 tablet_replica.cc:333] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.899474 20746 raft_consensus.cc:2243] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.900086 20746 raft_consensus.cc:2272] T 442f9ad084ff4c8b8794e7291eae0a2b P 30816458075e446eada4afc2f0e2796d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.902359 20746 tablet_replica.cc:333] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.902853 20746 raft_consensus.cc:2243] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.903759 20746 raft_consensus.cc:2272] T 6183ad54ee054404aa4ac5d460cdfbe4 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.905505 20746 tablet_replica.cc:333] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.905992 20746 raft_consensus.cc:2243] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.906644 20746 raft_consensus.cc:2272] T abdcd6cedd2745ab8091c067e79fee6a P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.908311 20746 tablet_replica.cc:333] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.908772 20746 raft_consensus.cc:2243] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.909219 20746 raft_consensus.cc:2272] T be9e67348632445a92364fcca02e5ea5 P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.911307 20746 tablet_replica.cc:333] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.911767 20746 raft_consensus.cc:2243] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.912106 20746 raft_consensus.cc:2272] T 06afeced037f44c799edc35d5b8c4dcd P 30816458075e446eada4afc2f0e2796d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.914425 20746 tablet_replica.cc:333] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.914904 20746 raft_consensus.cc:2243] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.915655 20746 raft_consensus.cc:2272] T 2efde3d922144fbd9d837360977e20b9 P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.917326 20746 tablet_replica.cc:333] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.917785 20746 raft_consensus.cc:2243] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.918165 20746 raft_consensus.cc:2272] T f73d334b1ef14d248f3ae2145223dcf6 P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.919763 20746 tablet_replica.cc:333] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.920184 20746 raft_consensus.cc:2243] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.920843 20746 raft_consensus.cc:2272] T 9d62f7db1ffb4fdc850f526801d775be P 30816458075e446eada4afc2f0e2796d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.922533 20746 tablet_replica.cc:333] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.922991 20746 raft_consensus.cc:2243] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.923383 20746 raft_consensus.cc:2272] T c264fe14fc1847eeb087cc1fef0c5aec P 30816458075e446eada4afc2f0e2796d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.925141 20746 tablet_replica.cc:333] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d: stopping tablet replica
I20260709 13:04:41.925637 20746 raft_consensus.cc:2243] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.926026 20746 raft_consensus.cc:2272] T 480c92b5319a4566b45b409b138eac0e P 30816458075e446eada4afc2f0e2796d [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:41.935561 21483 consensus_peers.cc:597] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 -> Peer 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903): Couldn't send request to peer 30816458075e446eada4afc2f0e2796d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.958843 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
I20260709 13:04:41.980444 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
W20260709 13:04:42.006728 21483 consensus_peers.cc:597] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 -> Peer 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903): Couldn't send request to peer 30816458075e446eada4afc2f0e2796d. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:42.011076 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:42.012094 20746 tablet_replica.cc:333] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.012776 20746 raft_consensus.cc:2243] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.013432 20746 raft_consensus.cc:2272] T 9d62f7db1ffb4fdc850f526801d775be P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.015333 20746 tablet_replica.cc:333] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.015872 20746 raft_consensus.cc:2243] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.016304 20746 raft_consensus.cc:2272] T abdcd6cedd2745ab8091c067e79fee6a P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.018103 20746 tablet_replica.cc:333] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.018600 20746 raft_consensus.cc:2243] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.019318 20746 raft_consensus.cc:2272] T be9e67348632445a92364fcca02e5ea5 P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.021539 20746 tablet_replica.cc:333] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.022046 20746 raft_consensus.cc:2243] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.022444 20746 raft_consensus.cc:2272] T 06afeced037f44c799edc35d5b8c4dcd P c7f1455a61774c4fb7d085c735aa4c3b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.024514 20746 tablet_replica.cc:333] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.025071 20746 raft_consensus.cc:2243] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.025530 20746 raft_consensus.cc:2272] T 2efde3d922144fbd9d837360977e20b9 P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.027144 20746 tablet_replica.cc:333] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.027617 20746 raft_consensus.cc:2243] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.028028 20746 raft_consensus.cc:2272] T 6183ad54ee054404aa4ac5d460cdfbe4 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.029654 20746 tablet_replica.cc:333] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.030144 20746 raft_consensus.cc:2243] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.030874 20746 raft_consensus.cc:2272] T f73d334b1ef14d248f3ae2145223dcf6 P c7f1455a61774c4fb7d085c735aa4c3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.032593 20746 tablet_replica.cc:333] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.033113 20746 raft_consensus.cc:2243] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.033911 20746 raft_consensus.cc:2272] T 442f9ad084ff4c8b8794e7291eae0a2b P c7f1455a61774c4fb7d085c735aa4c3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.036109 20746 tablet_replica.cc:333] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.036616 20746 raft_consensus.cc:2243] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.037261 20746 raft_consensus.cc:2272] T c264fe14fc1847eeb087cc1fef0c5aec P c7f1455a61774c4fb7d085c735aa4c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.039057 20746 tablet_replica.cc:333] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b: stopping tablet replica
I20260709 13:04:42.039520 20746 raft_consensus.cc:2243] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.040016 20746 raft_consensus.cc:2272] T 480c92b5319a4566b45b409b138eac0e P c7f1455a61774c4fb7d085c735aa4c3b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.070430 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
I20260709 13:04:42.088889 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:04:42.099159 21483 consensus_peers.cc:597] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 -> Peer 30816458075e446eada4afc2f0e2796d (127.20.66.129:32903): Couldn't send request to peer 30816458075e446eada4afc2f0e2796d. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:42.115497 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:42.116150 20746 tablet_replica.cc:333] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.116696 20746 raft_consensus.cc:2243] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.117247 20746 raft_consensus.cc:2272] T 9d62f7db1ffb4fdc850f526801d775be P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.118942 20746 tablet_replica.cc:333] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.119431 20746 raft_consensus.cc:2243] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.119845 20746 raft_consensus.cc:2272] T 6183ad54ee054404aa4ac5d460cdfbe4 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.121974 20746 tablet_replica.cc:333] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.122474 20746 raft_consensus.cc:2243] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.122882 20746 raft_consensus.cc:2272] T f73d334b1ef14d248f3ae2145223dcf6 P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.124480 20746 tablet_replica.cc:333] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.125093 20746 raft_consensus.cc:2243] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.125460 20746 raft_consensus.cc:2272] T abdcd6cedd2745ab8091c067e79fee6a P 344ed664705b49dfa97dee21a1b70411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.127063 20746 tablet_replica.cc:333] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.127525 20746 raft_consensus.cc:2243] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.127924 20746 raft_consensus.cc:2272] T be9e67348632445a92364fcca02e5ea5 P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.129802 20746 tablet_replica.cc:333] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.130293 20746 raft_consensus.cc:2243] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.130669 20746 raft_consensus.cc:2272] T 2efde3d922144fbd9d837360977e20b9 P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.132215 20746 tablet_replica.cc:333] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.132673 20746 raft_consensus.cc:2243] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.133392 20746 raft_consensus.cc:2272] T 06afeced037f44c799edc35d5b8c4dcd P 344ed664705b49dfa97dee21a1b70411 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.136072 20746 tablet_replica.cc:333] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.136611 20746 raft_consensus.cc:2243] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.137415 20746 raft_consensus.cc:2272] T c264fe14fc1847eeb087cc1fef0c5aec P 344ed664705b49dfa97dee21a1b70411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.138988 20746 tablet_replica.cc:333] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.139453 20746 raft_consensus.cc:2243] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.140126 20746 raft_consensus.cc:2272] T 480c92b5319a4566b45b409b138eac0e P 344ed664705b49dfa97dee21a1b70411 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.142026 20746 tablet_replica.cc:333] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411: stopping tablet replica
I20260709 13:04:42.142484 20746 raft_consensus.cc:2243] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:42.142879 20746 raft_consensus.cc:2272] T 442f9ad084ff4c8b8794e7291eae0a2b P 344ed664705b49dfa97dee21a1b70411 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.170504 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:04:42.188548 20746 master.cc:562] Master@127.20.66.190:41239 shutting down...
I20260709 13:04:42.208453 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:42.209132 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:42.209484 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9d859ca2ffb54b5ba41e19d9e39b16af: stopping tablet replica
I20260709 13:04:42.249826 20746 master.cc:584] Master@127.20.66.190:41239 shutdown complete.
30816458075e446eada4afc2f0e2796d  5
344ed664705b49dfa97dee21a1b70411  1
c7f1455a61774c4fb7d085c735aa4c3b  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10736 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:04:42.291167 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:46167
I20260709 13:04:42.292322 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.298341 21733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.298349 21734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.299371 21736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.299633 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:42.300513 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.300686 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.300844 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602282300827 us; error 0 us; skew 500 ppm
I20260709 13:04:42.301394 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.303891 20746 webserver.cc:533] Webserver started at http://127.20.66.190:38041/ using document root <none> and password file <none>
I20260709 13:04:42.304385 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.304567 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.304855 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.306071 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "85f464a9537546118c8c1817bd5574c1"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.310639 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:42.314059 21741 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.314838 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:42.315129 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "85f464a9537546118c8c1817bd5574c1"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.315431 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.325372 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.326540 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.364547 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:46167
I20260709 13:04:42.364671 21792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:46167 every 8 connection(s)
I20260709 13:04:42.368377 21793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.378530 21793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Bootstrap starting.
I20260709 13:04:42.383275 21793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.387566 21793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: No bootstrap required, opened a new log
I20260709 13:04:42.389778 21793 raft_consensus.cc:359] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f464a9537546118c8c1817bd5574c1" member_type: VOTER }
I20260709 13:04:42.390156 21793 raft_consensus.cc:385] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.390415 21793 raft_consensus.cc:740] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85f464a9537546118c8c1817bd5574c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.391010 21793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f464a9537546118c8c1817bd5574c1" member_type: VOTER }
I20260709 13:04:42.391449 21793 raft_consensus.cc:399] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:42.391671 21793 raft_consensus.cc:493] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:42.392084 21793 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.396453 21793 raft_consensus.cc:515] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f464a9537546118c8c1817bd5574c1" member_type: VOTER }
I20260709 13:04:42.397091 21793 leader_election.cc:304] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85f464a9537546118c8c1817bd5574c1; no voters: 
I20260709 13:04:42.398262 21793 leader_election.cc:290] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:42.398715 21796 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:42.400151 21796 raft_consensus.cc:697] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 1 LEADER]: Becoming Leader. State: Replica: 85f464a9537546118c8c1817bd5574c1, State: Running, Role: LEADER
I20260709 13:04:42.400915 21796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f464a9537546118c8c1817bd5574c1" member_type: VOTER }
I20260709 13:04:42.402406 21793 sys_catalog.cc:565] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:42.405927 21798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85f464a9537546118c8c1817bd5574c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f464a9537546118c8c1817bd5574c1" member_type: VOTER } }
I20260709 13:04:42.406206 21797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85f464a9537546118c8c1817bd5574c1. Latest consensus state: current_term: 1 leader_uuid: "85f464a9537546118c8c1817bd5574c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f464a9537546118c8c1817bd5574c1" member_type: VOTER } }
I20260709 13:04:42.406800 21798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:42.407020 21797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:42.412871 21802 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:42.420543 21802 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:42.421551 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:42.429507 21802 catalog_manager.cc:1383] Generated new cluster ID: 88b6cd9c6a984c3b82de9b374acb82b7
I20260709 13:04:42.429816 21802 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:42.447468 21802 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:42.448928 21802 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:42.467254 21802 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Generated new TSK 0
I20260709 13:04:42.467887 21802 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:42.488276 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.494165 21814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.495304 21815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.497141 21817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.497239 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:42.498273 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.498447 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.498567 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602282498556 us; error 0 us; skew 500 ppm
I20260709 13:04:42.499195 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.501578 20746 webserver.cc:533] Webserver started at http://127.20.66.129:33489/ using document root <none> and password file <none>
I20260709 13:04:42.502027 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.502180 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.502413 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.503466 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "8f3b35a35257416db3eaaf364cf6b52b"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.508072 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:04:42.511358 21822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.512118 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:04:42.512406 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "8f3b35a35257416db3eaaf364cf6b52b"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.512663 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.545437 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.546618 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.548074 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.550421 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.550637 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.550886 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.551074 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.591086 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:38927
I20260709 13:04:42.591202 21884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:38927 every 8 connection(s)
I20260709 13:04:42.595979 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.603144 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.604229 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.606971 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.607299 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:42.608462 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.608647 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.608776 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602282608765 us; error 0 us; skew 500 ppm
I20260709 13:04:42.609304 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.612428 20746 webserver.cc:533] Webserver started at http://127.20.66.130:44491/ using document root <none> and password file <none>
I20260709 13:04:42.612919 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.613168 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.613472 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.613780 21885 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46167
I20260709 13:04:42.614209 21885 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:42.615028 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "821b3831b8a348c0826bd3a20a857f00"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.615051 21885 heartbeater.cc:507] Master 127.20.66.190:46167 requested a full tablet report, sending...
I20260709 13:04:42.617851 21758 ts_manager.cc:194] Registered new tserver with Master: 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:04:42.619596 21758 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51604
I20260709 13:04:42.620436 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:04:42.624279 21897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.625270 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:42.625612 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "821b3831b8a348c0826bd3a20a857f00"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.625895 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.641506 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.642716 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.644487 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.647092 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.647349 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.647610 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.647807 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.689666 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:45191
I20260709 13:04:42.689779 21959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:45191 every 8 connection(s)
I20260709 13:04:42.694237 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.701418 21963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.703500 21964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.704083 20746 server_base.cc:1061] running on GCE node
W20260709 13:04:42.705907 21966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.706920 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.707194 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.707403 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602282707384 us; error 0 us; skew 500 ppm
I20260709 13:04:42.707790 21960 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46167
I20260709 13:04:42.708091 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.708190 21960 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:42.709157 21960 heartbeater.cc:507] Master 127.20.66.190:46167 requested a full tablet report, sending...
I20260709 13:04:42.711032 20746 webserver.cc:533] Webserver started at http://127.20.66.131:45113/ using document root <none> and password file <none>
I20260709 13:04:42.711483 21758 ts_manager.cc:194] Registered new tserver with Master: 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:42.711666 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.711920 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.712231 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.713322 21758 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51612
I20260709 13:04:42.713928 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.718863 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:42.722440 21971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.723393 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:42.723685 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-rgrd"
I20260709 13:04:42.724014 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.745572 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.746737 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.748131 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.750339 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.750556 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.750787 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.750954 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.789611 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:36521
I20260709 13:04:42.789716 22033 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:36521 every 8 connection(s)
I20260709 13:04:42.803197 22034 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46167
I20260709 13:04:42.803635 22034 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:42.804630 22034 heartbeater.cc:507] Master 127.20.66.190:46167 requested a full tablet report, sending...
I20260709 13:04:42.807664 21758 ts_manager.cc:194] Registered new tserver with Master: 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:42.911890 21758 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51626
I20260709 13:04:42.918180 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.125601763s
I20260709 13:04:43.011063 21758 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51636:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:43.017470 21758 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:43.233666 21998 tablet_service.cc:1511] Processing CreateTablet for tablet 1e33356352ee4f06be9c7f962467110c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:43.235121 21998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e33356352ee4f06be9c7f962467110c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.235450 21999 tablet_service.cc:1511] Processing CreateTablet for tablet 951fdd0c64f542cbab511fb32cfa91cb (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:43.236717 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 951fdd0c64f542cbab511fb32cfa91cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.238055 21997 tablet_service.cc:1511] Processing CreateTablet for tablet d2bcf30627084357a4745445324a2430 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:43.240252 21995 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee2529af5954101bde1189a694cafa5 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:43.242022 21996 tablet_service.cc:1511] Processing CreateTablet for tablet 8a76564a0ee84b74a3af01c79300d4b7 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:43.242312 21994 tablet_service.cc:1511] Processing CreateTablet for tablet 6f466d510c1b46d481ab83686b150d61 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:43.243227 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 1cf1fd218a894ed39170b8db4ecc6ee8 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:43.246366 21992 tablet_service.cc:1511] Processing CreateTablet for tablet 0103ac417f394657ac6cbe827df3b844 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:43.247334 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cf1fd218a894ed39170b8db4ecc6ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.247522 21991 tablet_service.cc:1511] Processing CreateTablet for tablet bc33f94111684d68a49eb643f8be2203 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:43.248848 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc33f94111684d68a49eb643f8be2203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.248632 21990 tablet_service.cc:1511] Processing CreateTablet for tablet c96543776baa4cdda2e6862202de4e52 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:43.249428 21995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee2529af5954101bde1189a694cafa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.250756 21990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96543776baa4cdda2e6862202de4e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.254388 21994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f466d510c1b46d481ab83686b150d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.255041 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bcf30627084357a4745445324a2430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.257300 21996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a76564a0ee84b74a3af01c79300d4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.265334 21992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0103ac417f394657ac6cbe827df3b844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.266142 21991 tablet_service.cc:1511] Processing CreateTablet for tablet 309ab90abff84c36a30bd5a2a0363d2c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:43.267460 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309ab90abff84c36a30bd5a2a0363d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.279035 22052 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:43.278782 21993 tablet_service.cc:1511] Processing CreateTablet for tablet ec2c60ea913b4b8a9b1cbfd733880203 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:43.280181 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec2c60ea913b4b8a9b1cbfd733880203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.286172 22052 tablet_bootstrap.cc:654] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.295814 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 1db8966fba10424a8df85ad63defc925 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:43.297178 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1db8966fba10424a8df85ad63defc925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.298699 21991 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2ed3e1870148589d1bba70ce75f160 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:43.299996 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2ed3e1870148589d1bba70ce75f160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.309707 21997 tablet_service.cc:1511] Processing CreateTablet for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:43.311023 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.319962 21850 tablet_service.cc:1511] Processing CreateTablet for tablet 951fdd0c64f542cbab511fb32cfa91cb (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:43.320464 21849 tablet_service.cc:1511] Processing CreateTablet for tablet 1e33356352ee4f06be9c7f962467110c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:43.321341 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 951fdd0c64f542cbab511fb32cfa91cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.322012 21849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e33356352ee4f06be9c7f962467110c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.322417 21995 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc728770e5c4aa1ac36a662cebb8598 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:43.325176 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 6622001ac78d44caa90b4ce7f5283aa1 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:43.325932 21995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc728770e5c4aa1ac36a662cebb8598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.330569 21848 tablet_service.cc:1511] Processing CreateTablet for tablet d2bcf30627084357a4745445324a2430 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:43.331590 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6622001ac78d44caa90b4ce7f5283aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.331938 21848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bcf30627084357a4745445324a2430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.342617 21999 tablet_service.cc:1511] Processing CreateTablet for tablet f7024293bd494d23afe75d0322f5445b (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:43.343926 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7024293bd494d23afe75d0322f5445b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.346938 22054 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:43.351471 21991 tablet_service.cc:1511] Processing CreateTablet for tablet f938584aad7a4b76be81dca03c9a647f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:43.352779 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f938584aad7a4b76be81dca03c9a647f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.352952 22054 tablet_bootstrap.cc:654] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.353996 21997 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:43.367760 21846 tablet_service.cc:1511] Processing CreateTablet for tablet 6f466d510c1b46d481ab83686b150d61 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:43.368611 21844 tablet_service.cc:1511] Processing CreateTablet for tablet 0103ac417f394657ac6cbe827df3b844 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:43.369197 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f466d510c1b46d481ab83686b150d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.369920 21844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0103ac417f394657ac6cbe827df3b844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.371096 21847 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee2529af5954101bde1189a694cafa5 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:43.372387 21847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee2529af5954101bde1189a694cafa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.373301 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.374083 21845 tablet_service.cc:1511] Processing CreateTablet for tablet 1cf1fd218a894ed39170b8db4ecc6ee8 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:43.375404 21845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cf1fd218a894ed39170b8db4ecc6ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.378741 21843 tablet_service.cc:1511] Processing CreateTablet for tablet bc33f94111684d68a49eb643f8be2203 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:43.380082 21843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc33f94111684d68a49eb643f8be2203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.381039 21850 tablet_service.cc:1511] Processing CreateTablet for tablet 8a76564a0ee84b74a3af01c79300d4b7 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:43.382296 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a76564a0ee84b74a3af01c79300d4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.385680 21995 tablet_service.cc:1511] Processing CreateTablet for tablet 958f1a2d78484a3bab2de506da912756 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:43.390400 21995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958f1a2d78484a3bab2de506da912756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.390730 21842 tablet_service.cc:1511] Processing CreateTablet for tablet c96543776baa4cdda2e6862202de4e52 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:43.386727 21999 tablet_service.cc:1511] Processing CreateTablet for tablet 97cf25498fcd4e4594d809b889d236d3 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:43.392033 21842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96543776baa4cdda2e6862202de4e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.392135 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cf25498fcd4e4594d809b889d236d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.394129 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 2167f46d5e1049268b9ce66dbf46f781 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:43.398708 21841 tablet_service.cc:1511] Processing CreateTablet for tablet 309ab90abff84c36a30bd5a2a0363d2c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:43.405314 21841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309ab90abff84c36a30bd5a2a0363d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.405387 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2167f46d5e1049268b9ce66dbf46f781. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:43.415117 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:43.420883 21997 tablet_service.cc:1511] Processing CreateTablet for tablet 7d337058bd1f4df1bd938c43960820be (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:43.422329 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d337058bd1f4df1bd938c43960820be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.429347 21994 tablet_service.cc:1511] Processing CreateTablet for tablet 90e34f5775b8400e8e9bb1a13ff27479 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:43.430670 21994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90e34f5775b8400e8e9bb1a13ff27479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.435048 22052 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:43.435534 22052 ts_tablet_manager.cc:1403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.157s	user 0.021s	sys 0.013s
I20260709 13:04:43.440860 21992 tablet_service.cc:1511] Processing CreateTablet for tablet b42af043129e482498af1e2c3882a042 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:43.442185 21992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42af043129e482498af1e2c3882a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.399467 21991 tablet_service.cc:1511] Processing CreateTablet for tablet 676bc9491c5a4083bdd0b754d904c1d6 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:43.446750 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676bc9491c5a4083bdd0b754d904c1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.450716 22052 raft_consensus.cc:359] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.451483 22052 raft_consensus.cc:385] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.451783 22052 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.453562 21925 tablet_service.cc:1511] Processing CreateTablet for tablet 951fdd0c64f542cbab511fb32cfa91cb (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:43.454871 21925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 951fdd0c64f542cbab511fb32cfa91cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.458251 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 2a91310e68464d64aa8324092182518f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:43.459538 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91310e68464d64aa8324092182518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.461443 21999 tablet_service.cc:1511] Processing CreateTablet for tablet 81cd5c4a1eef4004b3f1a9728e83a459 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:43.463163 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cd5c4a1eef4004b3f1a9728e83a459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.461443 21996 tablet_service.cc:1511] Processing CreateTablet for tablet 4458afcc40954b359ea857cddbfdd681 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:43.465355 21996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4458afcc40954b359ea857cddbfdd681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.452518 22052 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.475535 21847 tablet_service.cc:1511] Processing CreateTablet for tablet 7d337058bd1f4df1bd938c43960820be (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:43.479158 21844 tablet_service.cc:1511] Processing CreateTablet for tablet ec2c60ea913b4b8a9b1cbfd733880203 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:43.476846 21847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d337058bd1f4df1bd938c43960820be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.489590 22054 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:43.489745 21844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec2c60ea913b4b8a9b1cbfd733880203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.490165 22054 ts_tablet_manager.cc:1403] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.145s	user 0.013s	sys 0.003s
I20260709 13:04:43.489977 21998 tablet_service.cc:1511] Processing CreateTablet for tablet f6eae3eb9a4941fc88763a5da4ee7638 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:43.491446 21998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6eae3eb9a4941fc88763a5da4ee7638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.492971 21990 tablet_service.cc:1511] Processing CreateTablet for tablet a226c66b13a04eb389353b8e2deb175c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:43.493203 22054 raft_consensus.cc:359] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:43.493921 22054 raft_consensus.cc:385] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.494273 21990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226c66b13a04eb389353b8e2deb175c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.494227 22054 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.495154 22054 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:43.498907 21924 tablet_service.cc:1511] Processing CreateTablet for tablet 1e33356352ee4f06be9c7f962467110c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:43.500319 21924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e33356352ee4f06be9c7f962467110c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.499948 21922 tablet_service.cc:1511] Processing CreateTablet for tablet 8a76564a0ee84b74a3af01c79300d4b7 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:43.500736 21919 tablet_service.cc:1511] Processing CreateTablet for tablet 1cf1fd218a894ed39170b8db4ecc6ee8 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:43.501120 21923 tablet_service.cc:1511] Processing CreateTablet for tablet d2bcf30627084357a4745445324a2430 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:43.502097 21919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cf1fd218a894ed39170b8db4ecc6ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.502159 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee2529af5954101bde1189a694cafa5 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:43.505666 21920 tablet_service.cc:1511] Processing CreateTablet for tablet 6f466d510c1b46d481ab83686b150d61 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:43.523304 21918 tablet_service.cc:1511] Processing CreateTablet for tablet 0103ac417f394657ac6cbe827df3b844 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:43.525832 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 09efb42cfd6142bb9f816328042aea38 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:43.527089 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09efb42cfd6142bb9f816328042aea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.530328 21999 tablet_service.cc:1511] Processing CreateTablet for tablet 4021839a030d44ab886833307930ad33 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:43.531925 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4021839a030d44ab886833307930ad33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.533336 21846 tablet_service.cc:1511] Processing CreateTablet for tablet 1db8966fba10424a8df85ad63defc925 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:43.534658 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1db8966fba10424a8df85ad63defc925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.541237 21923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bcf30627084357a4745445324a2430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.543630 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 8594e4ce37964bf09c89143bed59f40e (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:43.545156 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8594e4ce37964bf09c89143bed59f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.546613 21917 tablet_service.cc:1511] Processing CreateTablet for tablet bc33f94111684d68a49eb643f8be2203 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:43.547123 21916 tablet_service.cc:1511] Processing CreateTablet for tablet c96543776baa4cdda2e6862202de4e52 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:43.548007 21917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc33f94111684d68a49eb643f8be2203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.555791 22052 ts_tablet_manager.cc:1434] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.120s	user 0.021s	sys 0.015s
I20260709 13:04:43.566115 21842 tablet_service.cc:1511] Processing CreateTablet for tablet 6622001ac78d44caa90b4ce7f5283aa1 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:43.598239 21842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6622001ac78d44caa90b4ce7f5283aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.600226 21919 tablet_service.cc:1511] Processing CreateTablet for tablet ec2c60ea913b4b8a9b1cbfd733880203 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:43.569324 21916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96543776baa4cdda2e6862202de4e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.603534 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee2529af5954101bde1189a694cafa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.561316 21997 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecc321cb1e14e99a3838b646f82e71a (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:43.610419 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecc321cb1e14e99a3838b646f82e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.570464 21845 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2ed3e1870148589d1bba70ce75f160 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:43.570564 21848 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:43.570837 21841 tablet_service.cc:1511] Processing CreateTablet for tablet f938584aad7a4b76be81dca03c9a647f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:43.613350 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f466d510c1b46d481ab83686b150d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.614337 21845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2ed3e1870148589d1bba70ce75f160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.617389 21918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0103ac417f394657ac6cbe827df3b844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.617784 21848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.619863 21922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a76564a0ee84b74a3af01c79300d4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.621543 21841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f938584aad7a4b76be81dca03c9a647f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.572554 21925 tablet_service.cc:1511] Processing CreateTablet for tablet 309ab90abff84c36a30bd5a2a0363d2c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:43.572554 21846 tablet_service.cc:1511] Processing CreateTablet for tablet 97cf25498fcd4e4594d809b889d236d3 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:43.622761 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cf25498fcd4e4594d809b889d236d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.625272 21885 heartbeater.cc:499] Master 127.20.66.190:46167 was elected leader, sending a full tablet report...
I20260709 13:04:43.628249 22034 heartbeater.cc:499] Master 127.20.66.190:46167 was elected leader, sending a full tablet report...
I20260709 13:04:43.628582 22052 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:43.569428 21849 tablet_service.cc:1511] Processing CreateTablet for tablet f7024293bd494d23afe75d0322f5445b (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:43.585160 22058 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:43.596319 21850 tablet_service.cc:1511] Processing CreateTablet for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:43.639322 21849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7024293bd494d23afe75d0322f5445b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.639108 22056 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.640128 22056 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.567768 21843 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc728770e5c4aa1ac36a662cebb8598 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:43.641705 21843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc728770e5c4aa1ac36a662cebb8598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.642560 22054 ts_tablet_manager.cc:1434] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.152s	user 0.015s	sys 0.025s
I20260709 13:04:43.642802 22056 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:43.643553 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.643671 22054 tablet_bootstrap.cc:492] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:43.652307 22058 tablet_bootstrap.cc:654] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.652829 21925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309ab90abff84c36a30bd5a2a0363d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.666124 22054 tablet_bootstrap.cc:654] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.680469 22052 tablet_bootstrap.cc:654] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.688938 21842 tablet_service.cc:1511] Processing CreateTablet for tablet 90e34f5775b8400e8e9bb1a13ff27479 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:43.717418 21960 heartbeater.cc:499] Master 127.20.66.190:46167 was elected leader, sending a full tablet report...
I20260709 13:04:43.721555 21923 tablet_service.cc:1511] Processing CreateTablet for tablet 1db8966fba10424a8df85ad63defc925 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:43.723157 21923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1db8966fba10424a8df85ad63defc925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.732999 21842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90e34f5775b8400e8e9bb1a13ff27479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.741142 21844 tablet_service.cc:1511] Processing CreateTablet for tablet 958f1a2d78484a3bab2de506da912756 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:43.753579 21844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958f1a2d78484a3bab2de506da912756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.762022 21919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec2c60ea913b4b8a9b1cbfd733880203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.798466 22058 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:43.798187 21917 tablet_service.cc:1511] Processing CreateTablet for tablet 6622001ac78d44caa90b4ce7f5283aa1 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:43.799063 22058 ts_tablet_manager.cc:1403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.239s	user 0.010s	sys 0.010s
I20260709 13:04:43.798803 21922 tablet_service.cc:1511] Processing CreateTablet for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:43.799580 21917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6622001ac78d44caa90b4ce7f5283aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.800194 21922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.801988 22058 raft_consensus.cc:359] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.804880 22054 tablet_bootstrap.cc:492] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:43.779748 21924 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2ed3e1870148589d1bba70ce75f160 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:43.836289 22054 ts_tablet_manager.cc:1403] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.193s	user 0.022s	sys 0.005s
I20260709 13:04:43.811798 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc728770e5c4aa1ac36a662cebb8598 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:43.836875 21924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2ed3e1870148589d1bba70ce75f160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.814137 22052 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:43.818030 21922 tablet_service.cc:1511] Processing CreateTablet for tablet f7024293bd494d23afe75d0322f5445b (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:43.826359 21999 tablet_service.cc:1511] Processing CreateTablet for tablet d41480ad8e164cf2aa88df180060345c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:43.837625 22052 ts_tablet_manager.cc:1403] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.209s	user 0.016s	sys 0.017s
I20260709 13:04:43.838274 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41480ad8e164cf2aa88df180060345c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.839365 22054 raft_consensus.cc:359] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.840099 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc728770e5c4aa1ac36a662cebb8598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.815035 21848 tablet_service.cc:1511] Processing CreateTablet for tablet 2167f46d5e1049268b9ce66dbf46f781 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:43.828604 21994 tablet_service.cc:1511] Processing CreateTablet for tablet be9ae799c8f043d7b4fc384aa1edf5ce (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:43.840417 22058 raft_consensus.cc:385] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.840721 22058 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.841707 21994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9ae799c8f043d7b4fc384aa1edf5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.841545 22058 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.826588 21998 tablet_service.cc:1511] Processing CreateTablet for tablet a5e5e01f84ee40afb9b688bf0b446c6c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:43.845868 21998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e5e01f84ee40afb9b688bf0b446c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.830703 21992 tablet_service.cc:1511] Processing CreateTablet for tablet 94e8056fad5248118e00e80f8a81538c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:43.847599 22052 raft_consensus.cc:359] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.848327 22052 raft_consensus.cc:385] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.848558 21992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e8056fad5248118e00e80f8a81538c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.848604 22052 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.850137 21848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2167f46d5e1049268b9ce66dbf46f781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.826884 21845 tablet_service.cc:1511] Processing CreateTablet for tablet 676bc9491c5a4083bdd0b754d904c1d6 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:43.831853 21850 tablet_service.cc:1511] Processing CreateTablet for tablet 2a91310e68464d64aa8324092182518f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:43.851819 21845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676bc9491c5a4083bdd0b754d904c1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.831038 21991 tablet_service.cc:1511] Processing CreateTablet for tablet 10076817fc0f46b4943de726ac0b508c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:43.853502 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91310e68464d64aa8324092182518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.853855 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10076817fc0f46b4943de726ac0b508c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.832864 21841 tablet_service.cc:1511] Processing CreateTablet for tablet 4458afcc40954b359ea857cddbfdd681 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:43.857089 21925 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:43.857862 21841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4458afcc40954b359ea857cddbfdd681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.858319 21925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.832403 21918 tablet_service.cc:1511] Processing CreateTablet for tablet f938584aad7a4b76be81dca03c9a647f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:43.834753 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 7e16208fe6d441998baf1cb8b8546d10 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:43.861688 21918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f938584aad7a4b76be81dca03c9a647f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.861819 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e16208fe6d441998baf1cb8b8546d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.864029 22052 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.836483 21990 tablet_service.cc:1511] Processing CreateTablet for tablet caafc8397a4f4963abeff030a28f5c47 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:43.865829 21990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caafc8397a4f4963abeff030a28f5c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.840034 22054 raft_consensus.cc:385] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.832981 21996 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed3a64e6b0242fa9fda4766cc68016b (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:43.869989 21996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3a64e6b0242fa9fda4766cc68016b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.870973 21849 tablet_service.cc:1511] Processing CreateTablet for tablet 09efb42cfd6142bb9f816328042aea38 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:43.872254 21849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09efb42cfd6142bb9f816328042aea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.877816 21922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7024293bd494d23afe75d0322f5445b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.881103 22054 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.882190 22054 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.884671 22054 ts_tablet_manager.cc:1434] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 13:04:43.887108 22054 tablet_bootstrap.cc:492] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:43.905741 22057 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.906255 22057 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:43.829717 21846 tablet_service.cc:1511] Processing CreateTablet for tablet b42af043129e482498af1e2c3882a042 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:43.910264 22057 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:43.915086 21916 tablet_service.cc:1511] Processing CreateTablet for tablet 97cf25498fcd4e4594d809b889d236d3 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:43.923918 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:43.924476 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:43.924672 21935 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:43.917703 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42af043129e482498af1e2c3882a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.857873 21847 tablet_service.cc:1511] Processing CreateTablet for tablet 81cd5c4a1eef4004b3f1a9728e83a459 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:43.927687 21847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cd5c4a1eef4004b3f1a9728e83a459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.928195 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:04:43.929227 21972 leader_election.cc:343] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:43.929261 22059 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:43.929860 22059 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.930253 22059 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:43.866246 22052 ts_tablet_manager.cc:1434] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:04:43.928238 21916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cf25498fcd4e4594d809b889d236d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.937698 21993 tablet_service.cc:1511] Processing CreateTablet for tablet f24c37d4e05d4b7eb71e234ac493bff5 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:43.942130 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c37d4e05d4b7eb71e234ac493bff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.864607 21843 tablet_service.cc:1511] Processing CreateTablet for tablet f6eae3eb9a4941fc88763a5da4ee7638 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:43.939996 22058 ts_tablet_manager.cc:1434] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.141s	user 0.008s	sys 0.004s
I20260709 13:04:43.947147 22056 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.947722 22056 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.949736 22056 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:43.953888 22059 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.956128 22059 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:43.912770 21920 tablet_service.cc:1511] Processing CreateTablet for tablet 958f1a2d78484a3bab2de506da912756 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
W20260709 13:04:43.964579 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08001eed60 after lost signal to thread 20749
I20260709 13:04:43.965049 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958f1a2d78484a3bab2de506da912756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.968274 22057 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.968788 22057 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:43.969707 21842 tablet_service.cc:1511] Processing CreateTablet for tablet d41480ad8e164cf2aa88df180060345c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:43.971122 22057 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:43.973646 22058 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:43.975363 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:43.978507 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:43.979058 21935 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:43.982669 21996 tablet_service.cc:1511] Processing CreateTablet for tablet 40a1bb4717e14331a67545a343ce1d87 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:43.983969 21996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40a1bb4717e14331a67545a343ce1d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.984292 22058 tablet_bootstrap.cc:654] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:43.985863 21972 leader_election.cc:343] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:43.986558 21997 tablet_service.cc:1511] Processing CreateTablet for tablet 0b8c4da9f5144e2b8007c8ffc563a584 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:43.987185 21999 tablet_service.cc:1511] Processing CreateTablet for tablet 5ef7ed88974a427b9cab09af73283182 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:43.987814 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8c4da9f5144e2b8007c8ffc563a584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.988544 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef7ed88974a427b9cab09af73283182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.999075 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 90e34f5775b8400e8e9bb1a13ff27479 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:44.000373 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90e34f5775b8400e8e9bb1a13ff27479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.001439 22052 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:44.011098 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:44.012284 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:44.014082 21843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6eae3eb9a4941fc88763a5da4ee7638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.017148 21842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41480ad8e164cf2aa88df180060345c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:44.021250 21972 leader_election.cc:343] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.021711 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:44.022517 22059 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:44.022867 21972 leader_election.cc:343] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.022745 21919 tablet_service.cc:1511] Processing CreateTablet for tablet 2167f46d5e1049268b9ce66dbf46f781 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:44.006677 22054 tablet_bootstrap.cc:654] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.038307 21923 tablet_service.cc:1511] Processing CreateTablet for tablet 676bc9491c5a4083bdd0b754d904c1d6 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:44.039616 21923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676bc9491c5a4083bdd0b754d904c1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.041047 21925 tablet_service.cc:1511] Processing CreateTablet for tablet 7d337058bd1f4df1bd938c43960820be (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:44.044399 21850 tablet_service.cc:1511] Processing CreateTablet for tablet 4021839a030d44ab886833307930ad33 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:44.064512 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4021839a030d44ab886833307930ad33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.065065 21918 tablet_service.cc:1511] Processing CreateTablet for tablet b42af043129e482498af1e2c3882a042 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:44.039628 21935 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:44.056135 21919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2167f46d5e1049268b9ce66dbf46f781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.070719 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:44.071528 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:44.062119 21841 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecc321cb1e14e99a3838b646f82e71a (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:44.072705 22059 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:44.086650 21925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d337058bd1f4df1bd938c43960820be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:44.085276 21823 leader_election.cc:343] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.082829 22052 tablet_bootstrap.cc:654] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.074942 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:44.090830 21920 tablet_service.cc:1511] Processing CreateTablet for tablet f6eae3eb9a4941fc88763a5da4ee7638 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:44.091081 21991 tablet_service.cc:1511] Processing CreateTablet for tablet 635368d86daf4188a9dcb9cc2f31ce10 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:44.073307 22059 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:44.085585 21917 tablet_service.cc:1511] Processing CreateTablet for tablet 4458afcc40954b359ea857cddbfdd681 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:04:44.097594 21823 leader_election.cc:343] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.098004 22059 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.102654 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:44.104326 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:44.105432 22058 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:44.105938 22058 ts_tablet_manager.cc:1403] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.133s	user 0.025s	sys 0.035s
W20260709 13:04:44.106221 21824 leader_election.cc:343] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.106716 21824 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:44.107621 22057 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:44.108004 21824 leader_election.cc:343] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.108511 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:44.109330 22057 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.102499 21922 tablet_service.cc:1511] Processing CreateTablet for tablet 81cd5c4a1eef4004b3f1a9728e83a459 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:44.073063 21841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecc321cb1e14e99a3838b646f82e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.113093 21918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42af043129e482498af1e2c3882a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.115121 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6eae3eb9a4941fc88763a5da4ee7638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.117671 22058 raft_consensus.cc:359] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:44.118427 22058 raft_consensus.cc:385] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.118705 22058 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.119478 22058 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:44.122570 21991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635368d86daf4188a9dcb9cc2f31ce10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.122720 21924 tablet_service.cc:1511] Processing CreateTablet for tablet 2a91310e68464d64aa8324092182518f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:44.121269 21917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4458afcc40954b359ea857cddbfdd681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.126420 21990 tablet_service.cc:1511] Processing CreateTablet for tablet 1d830a9119594381aa253bf173c99566 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:44.128082 21990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d830a9119594381aa253bf173c99566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.128811 22058 ts_tablet_manager.cc:1434] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:04:44.129837 22058 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:44.135259 21849 tablet_service.cc:1511] Processing CreateTablet for tablet a226c66b13a04eb389353b8e2deb175c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:44.136758 21849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226c66b13a04eb389353b8e2deb175c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.144938 21844 tablet_service.cc:1511] Processing CreateTablet for tablet 94e8056fad5248118e00e80f8a81538c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:44.146286 21844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e8056fad5248118e00e80f8a81538c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.147076 21924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91310e68464d64aa8324092182518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.148613 21922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cd5c4a1eef4004b3f1a9728e83a459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.141023 21847 tablet_service.cc:1511] Processing CreateTablet for tablet 8594e4ce37964bf09c89143bed59f40e (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:44.156297 21847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8594e4ce37964bf09c89143bed59f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.163043 22065 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.163333 21845 tablet_service.cc:1511] Processing CreateTablet for tablet 10076817fc0f46b4943de726ac0b508c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:44.163574 22065 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:44.171408 21916 tablet_service.cc:1511] Processing CreateTablet for tablet 09efb42cfd6142bb9f816328042aea38 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:44.173276 21916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09efb42cfd6142bb9f816328042aea38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.173349 22065 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:44.120071 21846 tablet_service.cc:1511] Processing CreateTablet for tablet a5e5e01f84ee40afb9b688bf0b446c6c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:44.183709 22054 tablet_bootstrap.cc:492] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:44.184226 22054 ts_tablet_manager.cc:1403] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.297s	user 0.017s	sys 0.006s
I20260709 13:04:44.183954 21923 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecc321cb1e14e99a3838b646f82e71a (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:44.185264 21923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecc321cb1e14e99a3838b646f82e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.187229 22054 raft_consensus.cc:359] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.187951 22054 raft_consensus.cc:385] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.188237 22054 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.169763 21845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10076817fc0f46b4943de726ac0b508c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.192071 22052 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:44.192586 22052 ts_tablet_manager.cc:1403] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.191s	user 0.011s	sys 0.005s
I20260709 13:04:44.195842 22052 raft_consensus.cc:359] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.196605 22052 raft_consensus.cc:385] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.196910 22052 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.214784 21842 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed3a64e6b0242fa9fda4766cc68016b (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:44.221287 21848 tablet_service.cc:1511] Processing CreateTablet for tablet be9ae799c8f043d7b4fc384aa1edf5ce (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:44.222131 21996 tablet_service.cc:1511] Processing CreateTablet for tablet 5f588420255042dda396ba6419abf6bc (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:44.223343 21996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f588420255042dda396ba6419abf6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.223299 21994 tablet_service.cc:1511] Processing CreateTablet for tablet 739311613bd748d68267b6f4478680c7 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:44.224733 21994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739311613bd748d68267b6f4478680c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.225175 21998 tablet_service.cc:1511] Processing CreateTablet for tablet d059242e679f4cbaab1cc83ba8353d60 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:44.225976 21997 tablet_service.cc:1511] Processing CreateTablet for tablet 430edce57310489986e4f739148461a6 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:44.227814 21993 tablet_service.cc:1511] Processing CreateTablet for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:44.235735 21990 tablet_service.cc:1511] Processing CreateTablet for tablet 634269b557d04cbe87cfab961aaf3f91 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:44.241252 21998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d059242e679f4cbaab1cc83ba8353d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.243655 21992 tablet_service.cc:1511] Processing CreateTablet for tablet 873e001f77374da4a034a00eaaea672f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:44.244972 21992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 873e001f77374da4a034a00eaaea672f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.236858 22058 tablet_bootstrap.cc:654] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.249886 21993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.208374 21999 tablet_service.cc:1511] Processing CreateTablet for tablet 8f3ae8d7d5514b8eb2514ea857b270c1 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:44.204631 22054 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.248545 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e5e01f84ee40afb9b688bf0b446c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.256222 21848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9ae799c8f043d7b4fc384aa1edf5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.258486 21918 tablet_service.cc:1511] Processing CreateTablet for tablet 4021839a030d44ab886833307930ad33 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:44.259728 21918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4021839a030d44ab886833307930ad33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.261161 21997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430edce57310489986e4f739148461a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.265313 21990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 634269b557d04cbe87cfab961aaf3f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.266281 21999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3ae8d7d5514b8eb2514ea857b270c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.265062 21850 tablet_service.cc:1511] Processing CreateTablet for tablet caafc8397a4f4963abeff030a28f5c47 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:44.265332 21842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3a64e6b0242fa9fda4766cc68016b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.252892 22054 ts_tablet_manager.cc:1434] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.000s
I20260709 13:04:44.269279 22054 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:44.273984 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caafc8397a4f4963abeff030a28f5c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.249079 21917 tablet_service.cc:1511] Processing CreateTablet for tablet a226c66b13a04eb389353b8e2deb175c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:44.288065 21757 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.264508 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 8594e4ce37964bf09c89143bed59f40e (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:44.291386 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8594e4ce37964bf09c89143bed59f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.293579 21917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226c66b13a04eb389353b8e2deb175c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.300611 22052 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.302371 21995 tablet_service.cc:1511] Processing CreateTablet for tablet 71a9145a148c4edebd5d8df1c9956298 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:44.303246 22052 ts_tablet_manager.cc:1434] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.110s	user 0.009s	sys 0.000s
I20260709 13:04:44.303536 21995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71a9145a148c4edebd5d8df1c9956298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.304446 22052 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:44.247856 21843 tablet_service.cc:1511] Processing CreateTablet for tablet 7e16208fe6d441998baf1cb8b8546d10 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:44.306061 21843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e16208fe6d441998baf1cb8b8546d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.308624 22054 tablet_bootstrap.cc:654] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.327014 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:44.352101 22059 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.352810 22059 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.355907 22059 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:44.356938 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:44.358507 21972 leader_election.cc:343] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.357020 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:44.359983 21972 leader_election.cc:343] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.360419 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:44.361536 22059 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.364974 21920 tablet_service.cc:1511] Processing CreateTablet for tablet d41480ad8e164cf2aa88df180060345c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:44.393229 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41480ad8e164cf2aa88df180060345c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.368770 21916 tablet_service.cc:1511] Processing CreateTablet for tablet be9ae799c8f043d7b4fc384aa1edf5ce (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:44.398087 21916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9ae799c8f043d7b4fc384aa1edf5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.403921 21922 tablet_service.cc:1511] Processing CreateTablet for tablet 10076817fc0f46b4943de726ac0b508c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:44.381353 22057 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.405558 22057 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.407670 22057 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:44.409478 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:44.384246 21845 tablet_service.cc:1511] Processing CreateTablet for tablet 5ef7ed88974a427b9cab09af73283182 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:44.384707 21841 tablet_service.cc:1511] Processing CreateTablet for tablet f24c37d4e05d4b7eb71e234ac493bff5 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:44.417662 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:44.417989 21845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef7ed88974a427b9cab09af73283182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.418351 21859 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
W20260709 13:04:44.418787 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:04:44.419554 21824 leader_election.cc:343] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.386812 21924 tablet_service.cc:1511] Processing CreateTablet for tablet 94e8056fad5248118e00e80f8a81538c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:04:44.373567 21899 leader_election.cc:343] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.420796 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:44.421607 22065 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:44.421969 22065 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:44.421854 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48497.4us
I20260709 13:04:44.422346 22065 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.425318 21841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c37d4e05d4b7eb71e234ac493bff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.383373 21923 tablet_service.cc:1511] Processing CreateTablet for tablet a5e5e01f84ee40afb9b688bf0b446c6c (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:44.428179 21846 tablet_service.cc:1511] Processing CreateTablet for tablet 40a1bb4717e14331a67545a343ce1d87 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:44.429438 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40a1bb4717e14331a67545a343ce1d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.432436 22065 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:44.435534 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:44.436124 21859 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.441495 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:44.442904 21823 leader_election.cc:343] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.443396 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:44.445533 22057 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.445308 21846 tablet_service.cc:1511] Processing CreateTablet for tablet 0b8c4da9f5144e2b8007c8ffc563a584 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:44.444964 22065 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:44.446679 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8c4da9f5144e2b8007c8ffc563a584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.448385 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:44.443411 21859 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:44.450078 21922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10076817fc0f46b4943de726ac0b508c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.450845 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:44.451205 21923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e5e01f84ee40afb9b688bf0b446c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.451727 22065 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:44.452207 21925 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed3a64e6b0242fa9fda4766cc68016b (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:44.411145 22052 tablet_bootstrap.cc:654] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:44.453356 21899 leader_election.cc:343] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.453632 21925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3a64e6b0242fa9fda4766cc68016b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.454670 21924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e8056fad5248118e00e80f8a81538c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.491087 21847 tablet_service.cc:1511] Processing CreateTablet for tablet 1d830a9119594381aa253bf173c99566 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:44.506983 21920 tablet_service.cc:1511] Processing CreateTablet for tablet caafc8397a4f4963abeff030a28f5c47 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:44.509025 21847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d830a9119594381aa253bf173c99566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.488453 21919 tablet_service.cc:1511] Processing CreateTablet for tablet 7e16208fe6d441998baf1cb8b8546d10 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:44.519299 22058 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:44.533661 22058 ts_tablet_manager.cc:1403] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.404s	user 0.047s	sys 0.047s
I20260709 13:04:44.537835 22058 raft_consensus.cc:359] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.551752 22058 raft_consensus.cc:385] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.552317 22058 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.560581 22058 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.565688 22065 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:44.572446 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caafc8397a4f4963abeff030a28f5c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.581555 21919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e16208fe6d441998baf1cb8b8546d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.582547 21849 tablet_service.cc:1511] Processing CreateTablet for tablet 635368d86daf4188a9dcb9cc2f31ce10 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:44.583902 21849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635368d86daf4188a9dcb9cc2f31ce10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.584548 21848 tablet_service.cc:1511] Processing CreateTablet for tablet 5f588420255042dda396ba6419abf6bc (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:44.585927 21848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f588420255042dda396ba6419abf6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.588757 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 5ef7ed88974a427b9cab09af73283182 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:44.590104 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef7ed88974a427b9cab09af73283182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.582532 21843 tablet_service.cc:1511] Processing CreateTablet for tablet 8f3ae8d7d5514b8eb2514ea857b270c1 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:44.591244 21843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3ae8d7d5514b8eb2514ea857b270c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.592351 22052 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:44.592808 22052 ts_tablet_manager.cc:1403] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.289s	user 0.056s	sys 0.050s
I20260709 13:04:44.594293 21842 tablet_service.cc:1511] Processing CreateTablet for tablet 430edce57310489986e4f739148461a6 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:44.595999 21842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430edce57310489986e4f739148461a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.597345 21917 tablet_service.cc:1511] Processing CreateTablet for tablet f24c37d4e05d4b7eb71e234ac493bff5 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:44.601516 22065 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:44.602686 21918 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33884 (request call id 34) took 1006 ms. Trace:
I20260709 13:04:44.610057 21850 tablet_service.cc:1511] Processing CreateTablet for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:44.614012 21850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.613104 21918 rpcz_store.cc:276] 0709 13:04:43.595992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.257563 (+661571us) service_pool.cc:224] Handling call
0709 13:04:44.259296 (+  1733us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:44.259386 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:44.391048 (+131662us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:44.577656 (+186608us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:44.602587 (+ 24931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.616281 22052 raft_consensus.cc:359] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.617178 21916 tablet_service.cc:1511] Processing CreateTablet for tablet 40a1bb4717e14331a67545a343ce1d87 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:44.618773 21845 tablet_service.cc:1511] Processing CreateTablet for tablet 739311613bd748d68267b6f4478680c7 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:44.628229 22052 raft_consensus.cc:385] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.632180 22058 ts_tablet_manager.cc:1434] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.013s
I20260709 13:04:44.637229 22052 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.645802 21918 tablet_service.cc:1511] Processing CreateTablet for tablet 1d830a9119594381aa253bf173c99566 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:44.647385 21844 tablet_service.cc:1511] Processing CreateTablet for tablet d059242e679f4cbaab1cc83ba8353d60 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:44.648185 22054 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:44.653419 22054 ts_tablet_manager.cc:1403] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.384s	user 0.015s	sys 0.006s
I20260709 13:04:44.653646 22058 tablet_bootstrap.cc:492] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:44.655586 21849 tablet_service.cc:1511] Processing CreateTablet for tablet 634269b557d04cbe87cfab961aaf3f91 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:44.651957 21758 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.656623 22054 raft_consensus.cc:359] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.662978 22054 raft_consensus.cc:385] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.663357 22054 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.664379 22052 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.667033 22052 ts_tablet_manager.cc:1434] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.000s
I20260709 13:04:44.668121 22052 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:44.664390 22054 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.669086 21845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739311613bd748d68267b6f4478680c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.669730 21844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d059242e679f4cbaab1cc83ba8353d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.678891 22054 ts_tablet_manager.cc:1434] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:04:44.685907 21923 tablet_service.cc:1511] Processing CreateTablet for tablet 0b8c4da9f5144e2b8007c8ffc563a584 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:44.690387 22057 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.678849 21849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 634269b557d04cbe87cfab961aaf3f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.691721 21917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c37d4e05d4b7eb71e234ac493bff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.693379 22057 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.705792 22057 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:44.706462 21916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40a1bb4717e14331a67545a343ce1d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.708444 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:44.709350 21918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d830a9119594381aa253bf173c99566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.709721 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:44.711129 21824 leader_election.cc:343] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:44.711614 21823 leader_election.cc:343] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.712028 21823 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:44.712666 22057 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.719534 21922 tablet_service.cc:1511] Processing CreateTablet for tablet 8f3ae8d7d5514b8eb2514ea857b270c1 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:44.720604 21922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3ae8d7d5514b8eb2514ea857b270c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.721467 22054 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:44.699064 21919 tablet_service.cc:1511] Processing CreateTablet for tablet 635368d86daf4188a9dcb9cc2f31ce10 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:04:44.703121 21972 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:44.725359 21919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635368d86daf4188a9dcb9cc2f31ce10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.728092 22058 tablet_bootstrap.cc:654] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.728005 22059 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:44.730579 22052 tablet_bootstrap.cc:654] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.734987 21923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8c4da9f5144e2b8007c8ffc563a584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.741443 21846 tablet_service.cc:1511] Processing CreateTablet for tablet 71a9145a148c4edebd5d8df1c9956298 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:44.742833 21846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71a9145a148c4edebd5d8df1c9956298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.744014 22059 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.744511 22059 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.756450 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:44.758675 21925 tablet_service.cc:1511] Processing CreateTablet for tablet 430edce57310489986e4f739148461a6 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:44.764437 21925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430edce57310489986e4f739148461a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:44.781214 21972 leader_election.cc:343] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.783030 22052 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:44.783579 22052 ts_tablet_manager.cc:1403] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.013s
I20260709 13:04:44.785449 22059 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:44.785660 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:44.744937 21920 tablet_service.cc:1511] Processing CreateTablet for tablet 5f588420255042dda396ba6419abf6bc (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:44.786427 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:44.787130 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f588420255042dda396ba6419abf6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.787703 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.788247 22052 raft_consensus.cc:359] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.789140 22052 raft_consensus.cc:385] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.789435 22052 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.790351 22054 tablet_bootstrap.cc:654] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:44.790251 21972 leader_election.cc:343] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.790858 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:44.791929 22059 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.744025 21848 tablet_service.cc:1511] Processing CreateTablet for tablet 873e001f77374da4a034a00eaaea672f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:44.793358 21848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 873e001f77374da4a034a00eaaea672f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.809067 22052 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.812096 22052 ts_tablet_manager.cc:1434] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.000s
I20260709 13:04:44.813150 22052 tablet_bootstrap.cc:492] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:44.818178 21924 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33884 (request call id 39) took 1069 ms. Trace:
I20260709 13:04:44.818651 21924 rpcz_store.cc:276] 0709 13:04:43.749091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.385934 (+636843us) service_pool.cc:224] Handling call
0709 13:04:44.420505 (+ 34571us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:44.420609 (+   104us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:44.618971 (+198362us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:44.817650 (+198679us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:44.818114 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.826694 22052 tablet_bootstrap.cc:654] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.820698 21924 tablet_service.cc:1511] Processing CreateTablet for tablet 739311613bd748d68267b6f4478680c7 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:44.828413 21924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739311613bd748d68267b6f4478680c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.827271 21916 tablet_service.cc:1511] Processing CreateTablet for tablet 634269b557d04cbe87cfab961aaf3f91 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:44.829730 21920 tablet_service.cc:1511] Processing CreateTablet for tablet d059242e679f4cbaab1cc83ba8353d60 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:44.835922 21921 tablet_service.cc:1511] Processing CreateTablet for tablet 873e001f77374da4a034a00eaaea672f (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:44.841284 21916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 634269b557d04cbe87cfab961aaf3f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.861181 21921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 873e001f77374da4a034a00eaaea672f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.866081 22058 tablet_bootstrap.cc:492] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:44.881577 22058 ts_tablet_manager.cc:1403] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.228s	user 0.021s	sys 0.021s
I20260709 13:04:44.881253 21919 tablet_service.cc:1511] Processing CreateTablet for tablet 71a9145a148c4edebd5d8df1c9956298 (DEFAULT_TABLE table=test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:44.867074 21917 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33884 (request call id 44) took 1037 ms. Trace:
I20260709 13:04:44.868706 22054 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:44.875555 22081 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.891613 21919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71a9145a148c4edebd5d8df1c9956298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.891752 22081 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.892525 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d059242e679f4cbaab1cc83ba8353d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.893944 22081 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:44.883601 21917 rpcz_store.cc:276] 0709 13:04:43.830100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.569249 (+739149us) service_pool.cc:224] Handling call
0709 13:04:44.627762 (+ 58513us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:44.638634 (+ 10872us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:44.825088 (+186454us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:44.866517 (+ 41429us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:44.867015 (+   498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.895323 22054 ts_tablet_manager.cc:1403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.174s	user 0.009s	sys 0.004s
I20260709 13:04:44.897428 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:44.898383 22054 raft_consensus.cc:359] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.898867 22059 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.899127 22054 raft_consensus.cc:385] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:44.899130 21898 leader_election.cc:343] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.899581 22054 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.899363 22059 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.902444 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:44.903950 21972 leader_election.cc:343] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.905718 22052 tablet_bootstrap.cc:492] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:44.905292 22054 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.906257 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:44.906862 21934 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:44.907687 22054 ts_tablet_manager.cc:1434] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:04:44.908236 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:44.909685 22059 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:44.909128 22052 ts_tablet_manager.cc:1403] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.006s
I20260709 13:04:44.917940 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:44.919778 22059 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:44.922598 22059 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.924744 22059 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.924686 22058 raft_consensus.cc:359] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.933168 22058 raft_consensus.cc:385] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.934017 22058 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.934958 22058 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.936468 22054 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:44.932312 22059 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.942111 22008 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 1: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:04:44.943400 22058 ts_tablet_manager.cc:1434] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.003s
I20260709 13:04:44.945355 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:44.946457 22059 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:44.946782 22081 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.947877 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:44.948760 22054 tablet_bootstrap.cc:654] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.950955 22058 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:44.949399 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:44.957242 22052 raft_consensus.cc:359] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.969216 22052 raft_consensus.cc:385] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.969614 22052 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
W20260709 13:04:44.969936 21972 leader_election.cc:343] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.970755 22052 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.972909 22081 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.986773 22058 tablet_bootstrap.cc:654] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.991375 22054 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:44.992000 22054 ts_tablet_manager.cc:1403] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.056s	user 0.027s	sys 0.005s
I20260709 13:04:44.995608 22054 raft_consensus.cc:359] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.996853 22054 raft_consensus.cc:385] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.997295 22054 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.998306 22054 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:44.985955 22052 ts_tablet_manager.cc:1434] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.002s
I20260709 13:04:45.006529 22052 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.010154 22054 ts_tablet_manager.cc:1434] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:04:45.019078 22081 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.019691 22081 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.022127 22081 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.023466 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.023640 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:45.025257 21898 leader_election.cc:343] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:45.025899 21899 leader_election.cc:343] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.026304 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.027724 22081 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.029633 22054 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.033989 21935 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:45.037376 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.037593 22052 tablet_bootstrap.cc:654] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.038911 22091 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.039477 22091 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:45.040325 22091 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.044767 22054 tablet_bootstrap.cc:654] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.048408 22058 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.050343 22052 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.051183 22052 ts_tablet_manager.cc:1403] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.000s
I20260709 13:04:45.054396 22052 raft_consensus.cc:359] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:45.055217 22052 raft_consensus.cc:385] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.055532 22052 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.055724 21758 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.056355 22052 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } }
I20260709 13:04:45.057595 22058 ts_tablet_manager.cc:1403] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.107s	user 0.019s	sys 0.055s
I20260709 13:04:45.060317 22058 raft_consensus.cc:359] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.060933 22058 raft_consensus.cc:385] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.061321 22058 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.062186 22058 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.063344 22052 ts_tablet_manager.cc:1434] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:04:45.064546 22052 tablet_bootstrap.cc:492] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.064640 22058 ts_tablet_manager.cc:1434] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:45.068439 22054 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.073624 22054 ts_tablet_manager.cc:1403] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.003s
I20260709 13:04:45.075007 22052 tablet_bootstrap.cc:654] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.077325 22054 raft_consensus.cc:359] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.078218 22054 raft_consensus.cc:385] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.078684 22054 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.079478 22058 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:45.079481 22054 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.086026 22058 tablet_bootstrap.cc:654] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.088126 22052 tablet_bootstrap.cc:492] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.088738 22052 ts_tablet_manager.cc:1403] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.004s
I20260709 13:04:45.091938 22054 ts_tablet_manager.cc:1434] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:04:45.092177 22052 raft_consensus.cc:359] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.093118 22052 raft_consensus.cc:385] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.093430 22052 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.094290 22052 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.096451 22052 ts_tablet_manager.cc:1434] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:45.100566 22052 tablet_bootstrap.cc:492] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.102345 21859 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.106370 22052 tablet_bootstrap.cc:654] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.125545 22052 tablet_bootstrap.cc:492] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.129139 22054 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.130555 22052 ts_tablet_manager.cc:1403] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 13:04:45.132839 22081 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:45.137342 22057 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.137948 22057 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.141918 22057 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.142128 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.142490 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.142127 22008 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.143601 22054 tablet_bootstrap.cc:654] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.147567 22058 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.148092 22058 ts_tablet_manager.cc:1403] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.069s	user 0.043s	sys 0.019s
W20260709 13:04:45.145828 21824 leader_election.cc:343] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:45.149938 21823 leader_election.cc:343] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.151091 22058 raft_consensus.cc:359] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.151868 22058 raft_consensus.cc:385] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.152171 22058 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.152064 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:45.157281 22057 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.162195 22052 raft_consensus.cc:359] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.163247 22052 raft_consensus.cc:385] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.163640 22052 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.164601 22052 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.165802 22058 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.177707 22081 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:45.179144 22052 ts_tablet_manager.cc:1434] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:04:45.181931 22052 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.195904 22052 tablet_bootstrap.cc:654] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.196729 22058 ts_tablet_manager.cc:1434] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.001s
I20260709 13:04:45.196921 22054 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.197461 22054 ts_tablet_manager.cc:1403] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.069s	user 0.024s	sys 0.012s
I20260709 13:04:45.198004 22058 tablet_bootstrap.cc:492] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:45.200209 22054 raft_consensus.cc:359] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.200917 22054 raft_consensus.cc:385] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.201223 22054 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.202174 22054 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.204743 22054 ts_tablet_manager.cc:1434] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:45.205868 22054 tablet_bootstrap.cc:492] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.212245 22054 tablet_bootstrap.cc:654] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.225986 22058 tablet_bootstrap.cc:654] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.227165 22052 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.227690 22052 ts_tablet_manager.cc:1403] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.011s
I20260709 13:04:45.230522 22052 raft_consensus.cc:359] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.231489 22052 raft_consensus.cc:385] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.231824 22052 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.236450 22091 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.236976 22091 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.239253 22091 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:45.241621 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.242350 21859 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:45.243366 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.245250 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:04:45.246204 21972 leader_election.cc:343] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.247292 22095 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.247714 22095 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.248057 22095 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.268630 22054 tablet_bootstrap.cc:492] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.269164 22054 ts_tablet_manager.cc:1403] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.024s
I20260709 13:04:45.269557 22052 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.271971 22054 raft_consensus.cc:359] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.270870 22095 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.272666 22054 raft_consensus.cc:385] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.273016 22054 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.273430 22052 ts_tablet_manager.cc:1434] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.001s
I20260709 13:04:45.273871 22054 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.276757 22052 tablet_bootstrap.cc:492] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.276868 22058 tablet_bootstrap.cc:492] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.277491 22058 ts_tablet_manager.cc:1403] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.017s
I20260709 13:04:45.280401 22058 raft_consensus.cc:359] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.281143 22058 raft_consensus.cc:385] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.281417 22058 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.282176 22091 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.282421 22052 tablet_bootstrap.cc:654] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.282729 22091 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.285838 22054 ts_tablet_manager.cc:1434] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:04:45.285532 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:45.286394 21860 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.286725 22054 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.290239 22058 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.291679 22106 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.292230 22106 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.296221 22058 ts_tablet_manager.cc:1434] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:04:45.300598 22054 tablet_bootstrap.cc:654] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.296918 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.301512 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.299757 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.297407 22058 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:45.306016 21823 leader_election.cc:343] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:45.309423 21824 leader_election.cc:343] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.309947 21824 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:45.310941 22058 tablet_bootstrap.cc:654] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.301056 22057 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.311713 22057 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:45.313941 21972 leader_election.cc:343] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.315798 22057 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.316524 22057 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.318499 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.319229 21935 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:45.320741 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:45.323778 22091 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:45.327829 22058 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.328325 22058 ts_tablet_manager.cc:1403] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.003s
I20260709 13:04:45.299971 22106 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.329926 22106 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.330518 22106 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.331079 22106 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.331164 22058 raft_consensus.cc:359] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.328824 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.338565 22058 raft_consensus.cc:385] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.338992 22058 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
W20260709 13:04:45.339254 21972 leader_election.cc:343] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.339776 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.339812 22058 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.341226 22091 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.343824 22058 ts_tablet_manager.cc:1434] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:04:45.346685 22058 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:45.354039 22058 tablet_bootstrap.cc:654] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.368352 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:45.369807 21935 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.374927 22058 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.375448 22058 ts_tablet_manager.cc:1403] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.002s
I20260709 13:04:45.378360 22058 raft_consensus.cc:359] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.379115 22058 raft_consensus.cc:385] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.379443 22058 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.386281 22058 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.389174 21860 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:45.403889 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:45.405547 22059 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.406225 22059 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.410986 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:45.413408 21972 leader_election.cc:343] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.393345 22058 ts_tablet_manager.cc:1434] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:04:45.415644 22059 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:45.414222 22091 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.416802 22091 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:45.418013 22091 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.415923 22058 tablet_bootstrap.cc:492] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:45.401821 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 92426.7us
W20260709 13:04:45.422900 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:45.426051 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.426997 22008 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:45.438371 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:45.443816 21972 leader_election.cc:343] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.444725 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.445942 22059 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.390133 22095 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:45.447759 22058 tablet_bootstrap.cc:654] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.452987 21935 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:45.469831 22108 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.470598 22108 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.475991 22108 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.478173 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:45.480448 21899 leader_election.cc:343] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.484506 22108 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.485203 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.485526 22108 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.486095 21860 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:45.488113 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:45.488953 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.489646 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:45.489705 22081 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.490295 22081 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.490772 22081 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.490984 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:45.494735 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.495463 22108 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.495977 22108 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.496327 22108 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.496649 22108 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:45.496922 21898 leader_election.cc:343] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.487363 21754 catalog_manager.cc:5697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.516606 22108 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.519977 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:45.520620 22008 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.520804 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:04:45.522949 21898 leader_election.cc:343] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.529086 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:45.530719 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.533151 22121 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.535104 22121 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.542321 22121 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.544752 22121 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.545097 22108 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.543160 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.546564 22121 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:45.548977 22121 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:45.549775 21898 leader_election.cc:343] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.554960 22054 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.555845 22116 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.556429 22116 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.560103 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:45.561497 21824 leader_election.cc:343] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.562255 22116 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.562999 22054 ts_tablet_manager.cc:1403] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.276s	user 0.050s	sys 0.089s
I20260709 13:04:45.570550 22058 tablet_bootstrap.cc:492] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.571305 22058 ts_tablet_manager.cc:1403] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.156s	user 0.011s	sys 0.001s
I20260709 13:04:45.574357 22054 raft_consensus.cc:359] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.579819 22054 raft_consensus.cc:385] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.580785 22054 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.582317 22054 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.581777 21756 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:45.584267 21972 consensus_peers.cc:597] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.588581 22054 ts_tablet_manager.cc:1434] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:04:45.586589 22095 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:04:45.614573 22054 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.459725 22106 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.621855 22054 tablet_bootstrap.cc:654] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.622567 22106 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.620428 22052 tablet_bootstrap.cc:492] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.625111 22052 ts_tablet_manager.cc:1403] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.349s	user 0.106s	sys 0.177s
I20260709 13:04:45.536480 22081 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.623663 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:45.628484 22008 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.629062 22058 raft_consensus.cc:359] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.629770 22058 raft_consensus.cc:385] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.630019 22058 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.630668 22058 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.632727 22058 ts_tablet_manager.cc:1434] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.061s	user 0.003s	sys 0.000s
I20260709 13:04:45.633930 22058 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:45.636505 22008 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:45.638298 22052 raft_consensus.cc:359] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.640740 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:45.642117 22052 raft_consensus.cc:385] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.642323 21860 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.642434 22052 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.643231 22052 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.645323 22052 ts_tablet_manager.cc:1434] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:04:45.644986 22058 tablet_bootstrap.cc:654] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.646699 22052 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.640897 22081 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.651422 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:45.652236 22106 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.652697 22106 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:45.653519 22106 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.654672 22052 tablet_bootstrap.cc:654] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.658846 21860 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:45.660319 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:45.661224 22081 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.661666 22081 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:45.661849 22054 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.662372 22054 ts_tablet_manager.cc:1403] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.000s
I20260709 13:04:45.662473 22081 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.665251 22054 raft_consensus.cc:359] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.665951 22054 raft_consensus.cc:385] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.666235 22054 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.667412 22054 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.669515 22054 ts_tablet_manager.cc:1434] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 13:04:45.641530 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 110899us
I20260709 13:04:45.670439 22054 tablet_bootstrap.cc:492] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.671664 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.672513 22008 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:45.675124 22122 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.675614 22122 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.677269 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:45.677561 22054 tablet_bootstrap.cc:654] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.679143 22122 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.679217 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.679898 22052 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.679932 21934 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:45.680601 22052 ts_tablet_manager.cc:1403] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 13:04:45.681715 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:45.682291 21934 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.681578 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.683427 22052 raft_consensus.cc:359] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.684170 22052 raft_consensus.cc:385] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.684473 22052 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.685348 22052 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.684633 21860 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:45.688002 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:45.689157 22122 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.689643 22122 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.690094 22122 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.691331 22052 ts_tablet_manager.cc:1434] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 13:04:45.692327 22052 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.697847 22094 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.702157 21899 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.703169 22081 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:45.703783 21899 leader_election.cc:343] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.704059 22081 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.704903 22081 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.704922 22094 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.701619 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:45.708890 22094 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:45.709878 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:45.711778 21972 leader_election.cc:343] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.716691 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.717949 22106 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.724069 22106 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.724447 22106 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:45.725080 21899 leader_election.cc:343] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.716789 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:45.728271 21972 leader_election.cc:343] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.730080 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.731197 22094 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.731771 22081 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.722611 21756 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.734140 22081 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.735107 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:45.729056 22122 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.729162 21934 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
W20260709 13:04:45.737042 21898 leader_election.cc:343] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.738442 22122 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.740885 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:45.741724 21860 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.742497 22058 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.742966 22058 ts_tablet_manager.cc:1403] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.007s
I20260709 13:04:45.742966 22052 tablet_bootstrap.cc:654] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.747313 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:45.746042 22058 raft_consensus.cc:359] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.748656 22058 raft_consensus.cc:385] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:45.748804 21899 leader_election.cc:343] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.748966 22058 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.749591 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:45.750147 22008 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.750113 22058 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.752357 22058 ts_tablet_manager.cc:1434] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:04:45.752310 21860 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:45.753656 22058 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:45.754082 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:45.754896 22122 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.755440 22122 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:45.756606 22122 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.762547 22008 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:45.756860 22106 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.766906 21899 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:45.768347 22106 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.769109 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:45.771848 21934 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.772728 22054 tablet_bootstrap.cc:492] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.773346 22054 ts_tablet_manager.cc:1403] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.036s
I20260709 13:04:45.769507 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:45.769956 22081 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.774748 22081 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:04:45.776887 21824 leader_election.cc:343] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.775743 22081 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.778395 22054 raft_consensus.cc:359] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.779259 22054 raft_consensus.cc:385] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.782624 22058 tablet_bootstrap.cc:654] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.779564 22054 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.783458 22054 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.786866 22054 ts_tablet_manager.cc:1434] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.004s
I20260709 13:04:45.787001 21934 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:45.788795 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:45.790897 22116 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.791411 22116 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:45.792712 22116 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.790462 22054 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.806969 22052 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.807503 22052 ts_tablet_manager.cc:1403] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.115s	user 0.007s	sys 0.008s
I20260709 13:04:45.815692 21754 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.819530 21755 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.820668 22052 raft_consensus.cc:359] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.821483 22052 raft_consensus.cc:385] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.821764 22052 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.822045 21755 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.825105 22052 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.829280 22054 tablet_bootstrap.cc:654] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.835533 22052 ts_tablet_manager.cc:1434] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:04:45.836777 22052 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.844524 22052 tablet_bootstrap.cc:654] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.849313 22094 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:45.850421 22054 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.850857 22054 ts_tablet_manager.cc:1403] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.004s
I20260709 13:04:45.854210 22054 raft_consensus.cc:359] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.854868 22054 raft_consensus.cc:385] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.855116 22054 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.855686 22054 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.857733 22054 ts_tablet_manager.cc:1434] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:45.858129 22094 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:45.864593 22054 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.871534 22054 tablet_bootstrap.cc:654] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.872924 22052 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.873569 22052 ts_tablet_manager.cc:1403] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.023s
I20260709 13:04:45.881399 22052 raft_consensus.cc:359] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.882424 22052 raft_consensus.cc:385] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.882772 22052 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.883647 22052 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.887435 22052 ts_tablet_manager.cc:1434] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:04:45.890029 22052 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.894256 22116 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.894799 22116 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.896868 22116 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.897769 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.898012 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.898711 22009 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:45.901295 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:45.902841 22116 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.905297 22052 tablet_bootstrap.cc:654] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.911327 22058 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:45.911791 22058 ts_tablet_manager.cc:1403] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.158s	user 0.038s	sys 0.029s
I20260709 13:04:45.915146 22054 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.915607 22054 ts_tablet_manager.cc:1403] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.002s
I20260709 13:04:45.918131 22054 raft_consensus.cc:359] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.918812 22054 raft_consensus.cc:385] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.919093 22054 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.920765 22116 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.919682 22054 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.924832 22058 raft_consensus.cc:359] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.925585 22058 raft_consensus.cc:385] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.925843 22058 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.926927 22054 ts_tablet_manager.cc:1434] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:04:45.930276 22054 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.930548 22052 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.931015 22052 ts_tablet_manager.cc:1403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 13:04:45.933748 22052 raft_consensus.cc:359] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.934425 22052 raft_consensus.cc:385] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.934710 22052 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.935348 22052 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.936636 22054 tablet_bootstrap.cc:654] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.926749 22058 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.927497 22116 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.937603 22116 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.927872 21758 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.940867 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:45.941735 22052 ts_tablet_manager.cc:1434] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:04:45.946882 22052 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:45.943312 22116 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.944396 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:45.954749 21824 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:45.956723 22052 tablet_bootstrap.cc:654] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.960304 22122 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:45.943079 22058 ts_tablet_manager.cc:1434] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:04:45.963939 22058 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:45.950418 22081 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.969340 22081 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.973909 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:45.975869 21899 leader_election.cc:343] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.978050 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:45.978921 21859 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:45.980597 21898 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:45.981830 22122 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.982264 22122 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.982677 22122 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.985213 22081 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:45.987087 22106 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.987959 22054 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:45.988411 22054 ts_tablet_manager.cc:1403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.007s
I20260709 13:04:45.991087 22052 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:45.991057 22054 raft_consensus.cc:359] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.992264 22054 raft_consensus.cc:385] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.992321 22052 ts_tablet_manager.cc:1403] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.009s
I20260709 13:04:45.994685 22054 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.995544 22054 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.997195 21859 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.997820 22054 ts_tablet_manager.cc:1434] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 13:04:45.997998 22052 raft_consensus.cc:359] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:45.998793 22054 tablet_bootstrap.cc:492] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:45.998843 22052 raft_consensus.cc:385] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.999415 22052 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.000284 22052 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.002535 22052 ts_tablet_manager.cc:1434] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:04:46.003847 22052 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:46.005740 22054 tablet_bootstrap.cc:654] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.010495 22052 tablet_bootstrap.cc:654] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.017511 22054 tablet_bootstrap.cc:492] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:46.018003 22054 ts_tablet_manager.cc:1403] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:04:46.022805 22054 raft_consensus.cc:359] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.023494 22054 raft_consensus.cc:385] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.023772 22054 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.025882 22054 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.027091 22052 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:46.027647 22052 ts_tablet_manager.cc:1403] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.009s
I20260709 13:04:46.029990 22091 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.030400 22091 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.030717 22052 raft_consensus.cc:359] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.031441 22052 raft_consensus.cc:385] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.031736 22052 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.032255 22091 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.027777 22054 ts_tablet_manager.cc:1434] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:46.033468 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:46.032425 22052 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.035727 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.036269 22052 ts_tablet_manager.cc:1434] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:04:46.037233 22052 tablet_bootstrap.cc:492] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
W20260709 13:04:46.037308 21972 leader_election.cc:343] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:46.038012 21972 leader_election.cc:343] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.038375 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.033660 22054 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:46.043409 22106 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.043799 22052 tablet_bootstrap.cc:654] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.043951 22091 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.043951 22106 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.044922 22116 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.045558 22116 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.047740 22116 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.047091 22106 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.049640 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:46.050222 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:46.052765 21823 leader_election.cc:343] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.050973 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.053589 22009 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:46.051045 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.054176 22054 tablet_bootstrap.cc:654] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.054499 22008 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:46.059517 22052 tablet_bootstrap.cc:492] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:46.059873 22054 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:46.060003 22052 ts_tablet_manager.cc:1403] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.013s
I20260709 13:04:46.060341 22054 ts_tablet_manager.cc:1403] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
W20260709 13:04:46.061857 21823 leader_election.cc:343] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.063654 21824 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.064307 22052 raft_consensus.cc:359] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.064512 22054 raft_consensus.cc:359] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.064965 22052 raft_consensus.cc:385] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.065290 22054 raft_consensus.cc:385] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.065333 22052 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.065464 22116 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.065724 22054 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.066108 22116 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.066541 22116 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.066356 22052 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.066628 22054 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.067453 22059 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.067891 22059 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.068640 22052 ts_tablet_manager.cc:1434] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:04:46.069235 22054 ts_tablet_manager.cc:1434] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:04:46.069569 22052 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:46.070158 22059 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.070297 22054 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:46.073079 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.073753 21858 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:46.074740 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:46.076504 22052 tablet_bootstrap.cc:654] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.076586 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:46.077670 22054 tablet_bootstrap.cc:654] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:46.077894 21972 leader_election.cc:343] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.078958 22059 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.079416 22059 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.080282 22059 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.064817 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.085621 22106 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.086062 22106 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.086436 22106 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.094014 22116 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.094159 22106 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.096958 22106 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.098064 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:46.098623 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.098991 22008 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.100672 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.102689 22052 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:46.103147 22052 ts_tablet_manager.cc:1403] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.001s
I20260709 13:04:46.104679 22122 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.106305 22052 raft_consensus.cc:359] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.106988 22052 raft_consensus.cc:385] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.107273 22052 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.107867 22058 tablet_bootstrap.cc:654] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.109320 22081 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.107929 22052 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:46.112421 21823 leader_election.cc:343] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.114758 22052 ts_tablet_manager.cc:1434] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:04:46.110983 22116 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.112814 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.110194 22122 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.115885 22052 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:46.117278 21858 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.121148 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:46.123171 21823 leader_election.cc:343] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.124349 22007 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.128564 22059 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.131562 22008 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:46.138378 22052 tablet_bootstrap.cc:654] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.134826 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.144212 21857 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.145478 22091 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.145972 22091 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.142623 22059 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:04:46.148532 21899 leader_election.cc:343] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.152663 21824 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.154592 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:04:46.156944 21972 leader_election.cc:343] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.157143 22091 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.158915 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:46.160418 21972 leader_election.cc:343] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.163617 22054 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:46.163774 22106 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.164111 22054 ts_tablet_manager.cc:1403] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.006s
I20260709 13:04:46.164976 22116 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.165542 22116 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:46.166357 22116 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.167376 22054 raft_consensus.cc:359] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.168037 22054 raft_consensus.cc:385] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.168305 22054 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.169102 22054 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.172154 22054 ts_tablet_manager.cc:1434] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:46.173115 22054 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:46.178490 21857 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:46.174878 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.184314 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:46.185192 22006 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.190771 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.192377 22091 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.193033 22091 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:46.194069 22091 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.201599 22058 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:46.202327 22058 ts_tablet_manager.cc:1403] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.239s	user 0.014s	sys 0.017s
I20260709 13:04:46.199496 22121 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:04:46.212265 22059 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.212841 22059 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:46.212313 21972 leader_election.cc:343] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.214725 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.216691 21858 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:46.217133 22094 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.216805 22059 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.218382 21898 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:46.215572 22058 raft_consensus.cc:359] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.220420 22058 raft_consensus.cc:385] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.221221 22121 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.222554 22121 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:46.223428 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:46.224062 22121 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:46.224941 21972 leader_election.cc:343] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.234037 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.235146 22116 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.235785 22116 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.220822 22058 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.239527 22116 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.252390 22058 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.235176 21860 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:46.241205 22106 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.242363 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.245959 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:46.250568 22057 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.255707 22054 tablet_bootstrap.cc:654] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.261248 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.260612 22008 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:46.261942 21755 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.264278 22059 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.264716 22059 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.264777 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:46.265142 22059 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.268273 22058 ts_tablet_manager.cc:1434] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:04:46.269551 22058 tablet_bootstrap.cc:492] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:46.272269 22106 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.273439 22081 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.286962 21860 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:46.272296 21823 leader_election.cc:343] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.287628 22116 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.318240 22116 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.320798 22052 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:46.322189 22052 ts_tablet_manager.cc:1403] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.207s	user 0.024s	sys 0.012s
I20260709 13:04:46.331969 22006 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:46.336624 22164 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.299487 22094 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.321573 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:46.299281 21824 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:46.290836 22106 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.291162 22081 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.346287 22094 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.350467 22116 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:04:46.359516 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53455.6us
I20260709 13:04:46.356179 22052 raft_consensus.cc:359] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.360456 22052 raft_consensus.cc:385] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.360885 22052 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.362046 22052 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.351081 22106 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:46.347700 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48538.3us
I20260709 13:04:46.377283 22052 ts_tablet_manager.cc:1434] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.001s
I20260709 13:04:46.378067 22052 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:46.388649 22094 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.391572 22122 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.391973 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.379380 22059 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.395108 22081 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.395888 22059 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.396948 22054 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:46.397593 22054 ts_tablet_manager.cc:1403] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.225s	user 0.019s	sys 0.014s
I20260709 13:04:46.398548 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.399106 21858 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.399514 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.399710 22106 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.400115 22006 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:46.401197 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.401804 22009 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:46.401908 21755 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.402582 22106 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.403564 22058 tablet_bootstrap.cc:654] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.404930 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:04:46.389493 21823 leader_election.cc:343] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.405318 22122 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.406634 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.407837 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:46.408062 21972 leader_election.cc:343] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.408475 21860 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:46.409322 22009 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.409432 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:46.409951 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.410211 22006 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:46.414402 21823 leader_election.cc:343] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.397662 22116 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.415460 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:46.418074 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:46.413893 22164 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.419440 22164 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.419772 22164 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.422384 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.441373 22057 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.441974 22057 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.442430 22057 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.442662 22052 tablet_bootstrap.cc:654] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.421038 22091 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.425058 22116 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:46.465945 22091 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.488371 22091 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.489507 22166 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.490099 22166 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.501820 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.502785 21859 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:46.508716 21755 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.514783 22054 raft_consensus.cc:359] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.515511 22054 raft_consensus.cc:385] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.515774 22054 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.516389 22054 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.540071 22054 ts_tablet_manager.cc:1434] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.142s	user 0.007s	sys 0.000s
I20260709 13:04:46.544301 22054 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:46.542853 22095 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.544407 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 139282us
I20260709 13:04:46.559952 22164 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.555873 22116 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.551599 22006 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:46.557855 22095 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.566085 22058 tablet_bootstrap.cc:492] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:46.574533 22095 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.578598 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:46.580304 21972 leader_election.cc:343] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.578478 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:46.582834 21972 leader_election.cc:343] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.583287 21972 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.584522 22095 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.585834 21934 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:46.588263 22058 ts_tablet_manager.cc:1403] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.319s	user 0.053s	sys 0.084s
I20260709 13:04:46.593233 22166 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.598126 22164 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:04:46.601781 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:46.610061 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.610675 21856 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.608350 22058 raft_consensus.cc:359] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.611771 22058 raft_consensus.cc:385] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.612020 22058 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.612545 22058 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.613844 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.614420 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.615963 22006 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:46.614861 22164 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:04:46.617360 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.629470 21858 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
W20260709 13:04:46.629884 21823 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:46.630170 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:46.631276 22164 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.631170 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.631667 22164 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.632009 22164 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.632570 22094 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.632259 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:46.633416 21898 leader_election.cc:343] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.633611 22094 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:46.624866 22171 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.634557 22171 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:46.634827 22094 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.637046 22058 ts_tablet_manager.cc:1434] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.002s
I20260709 13:04:46.638514 22058 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:46.635722 22171 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.637274 22091 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.640748 22006 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 1: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:04:46.643998 22054 tablet_bootstrap.cc:654] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.646108 22057 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.647050 21856 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:46.652842 22059 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.653395 22052 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:46.653990 22052 ts_tablet_manager.cc:1403] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.276s	user 0.018s	sys 0.028s
I20260709 13:04:46.653849 22059 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.654325 22091 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:46.655884 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.655984 22166 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.657259 21858 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:46.657464 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.658228 21935 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:46.657929 22052 raft_consensus.cc:359] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.658687 22052 raft_consensus.cc:385] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.659008 22052 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.654243 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.659857 21898 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:46.660459 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:46.661571 22009 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.661718 22165 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.663040 22165 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:04:46.664495 21823 leader_election.cc:343] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.665215 22165 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.663245 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.668557 22091 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.659631 22057 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.682351 22052 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.689286 22052 ts_tablet_manager.cc:1434] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 13:04:46.666718 22164 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.691923 22164 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.692480 22059 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:04:46.689306 22176 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:46.702421 22116 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.703006 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.703725 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:46.706853 21972 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:46.707850 22091 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.708105 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.709162 22059 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:46.718050 22009 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:46.722404 22052 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:46.718528 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.734618 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:46.735997 22171 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:46.736078 21898 leader_election.cc:343] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.737411 22171 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:46.741367 22171 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.746201 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:46.756173 22009 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.756791 22166 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.757422 22166 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.759588 22166 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.760957 22166 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.770097 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.770817 22005 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:46.773840 22058 tablet_bootstrap.cc:654] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.763471 21755 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.789552 22009 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:46.819468 22054 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:46.819949 22054 ts_tablet_manager.cc:1403] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.276s	user 0.022s	sys 0.011s
I20260709 13:04:46.827121 22052 tablet_bootstrap.cc:654] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.829061 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:46.830901 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.834390 22164 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.834946 22164 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:46.835916 22164 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.841943 22168 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.843007 21898 leader_election.cc:343] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.843317 22168 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.851895 22168 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.853503 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:46.854370 22009 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:46.856113 21824 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:46.857115 22168 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.857446 22168 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.857757 22168 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.847802 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:46.867434 21755 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.886545 21755 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.877625 22081 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.915982 22054 raft_consensus.cc:359] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.917495 22054 raft_consensus.cc:385] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.917824 22054 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.908717 22081 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.920396 22054 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.923018 22081 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.923918 22054 ts_tablet_manager.cc:1434] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.000s
I20260709 13:04:46.925141 22054 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:46.932658 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:46.936094 21823 leader_election.cc:343] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.939256 22007 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.952899 22058 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:46.953442 22058 ts_tablet_manager.cc:1403] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.315s	user 0.020s	sys 0.014s
I20260709 13:04:46.959854 22058 raft_consensus.cc:359] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.960877 22058 raft_consensus.cc:385] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.961316 22058 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.962267 22058 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.965772 22058 ts_tablet_manager.cc:1434] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:04:46.968799 22058 tablet_bootstrap.cc:492] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:46.971554 22081 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.973100 22168 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.976382 22168 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:46.975872 21758 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.981022 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:46.982129 22052 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:46.982594 22052 ts_tablet_manager.cc:1403] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.260s	user 0.016s	sys 0.026s
W20260709 13:04:46.982834 21823 leader_election.cc:343] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.985208 22054 tablet_bootstrap.cc:654] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.990193 22165 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.992669 22052 raft_consensus.cc:359] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:46.993402 22052 raft_consensus.cc:385] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.993669 22052 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.994535 22052 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:46.995970 21823 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:46.997184 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:46.998965 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:46.999588 22007 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:47.000255 21898 leader_election.cc:343] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.008354 22058 tablet_bootstrap.cc:654] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.009371 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:47.014288 22006 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.013219 22007 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:47.016670 21824 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.017828 22168 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.018302 22168 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:47.019269 22168 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.021097 22081 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.021808 22052 ts_tablet_manager.cc:1434] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:04:47.022939 22052 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.028474 22006 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:47.028659 21755 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.029685 22054 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.030164 22054 ts_tablet_manager.cc:1403] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.105s	user 0.017s	sys 0.019s
I20260709 13:04:47.029989 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:47.032483 22054 raft_consensus.cc:359] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.032977 22054 raft_consensus.cc:385] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.033246 22054 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.033840 22054 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.034363 22081 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.035115 22081 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:47.036047 22054 ts_tablet_manager.cc:1434] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:04:47.036103 22081 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.037070 22054 tablet_bootstrap.cc:492] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.037829 22058 tablet_bootstrap.cc:492] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.038267 22058 ts_tablet_manager.cc:1403] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.001s
I20260709 13:04:47.040966 22171 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.041442 22171 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.041303 21758 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.043241 22052 tablet_bootstrap.cc:654] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.043576 22171 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.045575 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:47.046886 21824 leader_election.cc:343] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.047184 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.049213 22059 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.049539 22058 raft_consensus.cc:359] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.050205 22058 raft_consensus.cc:385] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.050472 22058 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.050990 22054 tablet_bootstrap.cc:654] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:47.051715 21823 leader_election.cc:343] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.052156 21823 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.052939 22168 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.053781 22058 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.056222 22058 ts_tablet_manager.cc:1434] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:04:47.060276 22058 tablet_bootstrap.cc:492] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.064306 22054 tablet_bootstrap.cc:492] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
W20260709 13:04:47.064827 21972 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:47.065407 22054 ts_tablet_manager.cc:1403] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.001s
I20260709 13:04:47.067988 22054 raft_consensus.cc:359] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.068809 22054 raft_consensus.cc:385] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.069149 22054 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.069854 22054 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.071807 22058 tablet_bootstrap.cc:654] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.072252 22054 ts_tablet_manager.cc:1434] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:04:47.073151 22054 tablet_bootstrap.cc:492] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.080392 21758 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.096150 22054 tablet_bootstrap.cc:654] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.096791 22052 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:47.097389 22052 ts_tablet_manager.cc:1403] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.075s	user 0.006s	sys 0.008s
I20260709 13:04:47.099982 22058 tablet_bootstrap.cc:492] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.100420 22058 ts_tablet_manager.cc:1403] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.010s
I20260709 13:04:47.101948 21755 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.102890 22058 raft_consensus.cc:359] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.103091 22052 raft_consensus.cc:359] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.103608 22058 raft_consensus.cc:385] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.103901 22052 raft_consensus.cc:385] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.103994 22058 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.104420 22052 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.104809 22058 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.109226 22058 ts_tablet_manager.cc:1434] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:47.105329 22052 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.110224 22058 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.117045 22058 tablet_bootstrap.cc:654] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.112753 22052 ts_tablet_manager.cc:1434] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:04:47.119586 22052 tablet_bootstrap.cc:492] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.126240 22054 tablet_bootstrap.cc:492] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.126752 22054 ts_tablet_manager.cc:1403] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.009s
I20260709 13:04:47.129479 22054 raft_consensus.cc:359] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.130152 22054 raft_consensus.cc:385] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.130414 22054 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.131012 22054 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.135248 22052 tablet_bootstrap.cc:654] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.140828 22168 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.145715 22058 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.146190 22058 ts_tablet_manager.cc:1403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.003s
I20260709 13:04:47.147133 22054 ts_tablet_manager.cc:1434] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:04:47.148595 22054 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.149245 22058 raft_consensus.cc:359] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.149905 22058 raft_consensus.cc:385] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.150170 22058 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.150954 22058 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.152433 22052 tablet_bootstrap.cc:492] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:47.152864 22052 ts_tablet_manager.cc:1403] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.003s
I20260709 13:04:47.154590 22054 tablet_bootstrap.cc:654] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.155519 22052 raft_consensus.cc:359] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.156190 22052 raft_consensus.cc:385] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.156457 22052 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.158836 21935 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.163127 22058 ts_tablet_manager.cc:1434] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:04:47.169597 22106 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.171147 22106 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.157181 22052 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.181998 21933 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.186038 22052 ts_tablet_manager.cc:1434] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.000s
I20260709 13:04:47.186250 22058 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.187006 22052 tablet_bootstrap.cc:492] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.209110 22052 tablet_bootstrap.cc:654] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.212585 22054 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.213611 22054 ts_tablet_manager.cc:1403] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.065s	user 0.029s	sys 0.028s
I20260709 13:04:47.216243 22054 raft_consensus.cc:359] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.216915 22054 raft_consensus.cc:385] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.217250 22054 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.217890 22054 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.220108 22054 ts_tablet_manager.cc:1434] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260709 13:04:47.221035 22054 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.227313 22054 tablet_bootstrap.cc:654] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.235383 22052 tablet_bootstrap.cc:492] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:47.235872 22052 ts_tablet_manager.cc:1403] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.001s
I20260709 13:04:47.255353 22052 raft_consensus.cc:359] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.256090 22058 tablet_bootstrap.cc:654] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.256088 22052 raft_consensus.cc:385] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.256620 22052 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.257421 22052 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.258700 22054 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.259171 22054 ts_tablet_manager.cc:1403] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.006s
I20260709 13:04:47.259723 22052 ts_tablet_manager.cc:1434] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:04:47.260620 22052 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.265929 22054 raft_consensus.cc:359] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.266467 22054 raft_consensus.cc:385] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.266721 22054 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.267742 22058 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.268203 22058 ts_tablet_manager.cc:1403] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.082s	user 0.007s	sys 0.007s
I20260709 13:04:47.270900 22052 tablet_bootstrap.cc:654] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.270588 22058 raft_consensus.cc:359] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.271219 22058 raft_consensus.cc:385] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.271469 22058 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.272526 22058 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.278680 22058 ts_tablet_manager.cc:1434] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:04:47.279888 22058 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.282641 22081 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.284194 22106 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.267354 22054 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.284664 22106 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.286756 22054 ts_tablet_manager.cc:1434] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 13:04:47.287425 22058 tablet_bootstrap.cc:654] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.287717 22054 tablet_bootstrap.cc:492] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.290308 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:47.291743 21823 leader_election.cc:343] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.288648 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.293340 22164 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:47.294240 21824 leader_election.cc:343] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.294436 22054 tablet_bootstrap.cc:654] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.294714 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.295516 22057 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.286691 22106 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.305220 22058 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.306385 22058 ts_tablet_manager.cc:1403] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.009s
I20260709 13:04:47.311520 22057 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.313112 22058 raft_consensus.cc:359] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.313856 22058 raft_consensus.cc:385] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.313892 22005 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.323758 22058 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.324522 22058 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.329095 22054 tablet_bootstrap.cc:492] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.328994 22081 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.329617 22054 ts_tablet_manager.cc:1403] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.004s
I20260709 13:04:47.332291 22054 raft_consensus.cc:359] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.332962 22054 raft_consensus.cc:385] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.333274 22054 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.334035 22054 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.340180 22059 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.340736 22059 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.343569 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:47.345160 21972 leader_election.cc:343] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.345352 22058 ts_tablet_manager.cc:1434] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
I20260709 13:04:47.346201 22059 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:47.347074 21933 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.350097 22054 ts_tablet_manager.cc:1434] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:04:47.350097 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.351192 22054 tablet_bootstrap.cc:492] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
W20260709 13:04:47.351728 21972 leader_election.cc:343] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.352144 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:47.354301 22058 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.356199 22059 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.356107 22106 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.359436 22054 tablet_bootstrap.cc:654] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.363574 22052 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:47.364745 22052 ts_tablet_manager.cc:1403] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.104s	user 0.014s	sys 0.011s
I20260709 13:04:47.367476 22058 tablet_bootstrap.cc:654] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.367919 22091 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.369400 22057 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.384738 22091 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.384982 22057 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.386511 22054 tablet_bootstrap.cc:492] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.386981 22054 ts_tablet_manager.cc:1403] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.000s
I20260709 13:04:47.387115 22057 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.387073 22091 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:47.388806 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.389328 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:47.389533 22054 raft_consensus.cc:359] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.390102 21860 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:47.390362 22054 raft_consensus.cc:385] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.390642 22054 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.370831 22106 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.391316 22054 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:47.391871 21824 leader_election.cc:343] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.392119 22106 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.393041 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.394395 22106 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:04:47.395764 21823 leader_election.cc:343] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.396409 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.397444 22171 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.398986 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.401088 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:47.402185 22054 ts_tablet_manager.cc:1434] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:04:47.401932 22095 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.401743 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.403064 22095 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.403451 22054 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.403505 22007 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:47.406010 22173 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.406503 22173 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.406585 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:47.407015 22173 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.407303 21856 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:47.408839 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:47.409801 22081 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.410110 22095 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:47.410440 22081 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.413134 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.414826 22081 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.415287 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:47.415776 21860 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.418355 22054 tablet_bootstrap.cc:654] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.421319 22095 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.421815 22095 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.422242 22095 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.427381 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.430054 22057 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.432767 21935 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.435953 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.438614 21899 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:47.438746 22170 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.440394 21932 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:47.442073 21972 leader_election.cc:343] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.384609 22052 raft_consensus.cc:359] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.456169 22052 raft_consensus.cc:385] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.456619 22052 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.457540 22052 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.453418 22164 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.474675 22081 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.460861 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.499571 22052 ts_tablet_manager.cc:1434] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.117s	user 0.009s	sys 0.001s
I20260709 13:04:47.500761 22052 tablet_bootstrap.cc:492] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.506969 22168 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.507885 22058 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.508338 22058 ts_tablet_manager.cc:1403] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.154s	user 0.038s	sys 0.023s
I20260709 13:04:47.512517 22058 raft_consensus.cc:359] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.513230 22058 raft_consensus.cc:385] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.513648 22058 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.507463 22168 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.498626 22057 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.513273 22095 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.515022 22058 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.516717 22095 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:47.517246 22058 ts_tablet_manager.cc:1434] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:04:47.518129 22058 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:47.514273 21972 leader_election.cc:343] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.519397 22173 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.521759 22168 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.525851 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:47.525743 22164 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.526481 21860 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.532732 21860 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:47.533563 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.542765 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.545094 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:04:47.547117 21972 leader_election.cc:343] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.561959 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.552557 22170 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.565284 22171 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.566227 22170 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.568012 22170 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.569866 22054 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.569415 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.570329 22054 ts_tablet_manager.cc:1403] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.167s	user 0.018s	sys 0.035s
I20260709 13:04:47.564077 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:04:47.578030 21824 leader_election.cc:343] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.586263 22054 raft_consensus.cc:359] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.586989 22054 raft_consensus.cc:385] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.587262 22054 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.587926 22054 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.589742 22058 tablet_bootstrap.cc:654] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.590457 22054 ts_tablet_manager.cc:1434] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:04:47.592410 22054 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.583359 22173 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:47.595211 22052 tablet_bootstrap.cc:654] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.609210 21934 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.602967 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.596918 22059 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:47.607254 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48989.7us
I20260709 13:04:47.647774 22106 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:47.655905 22059 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:04:47.670729 21823 leader_election.cc:343] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.671263 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.671660 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:47.672241 21856 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.673345 22231 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.677030 22054 tablet_bootstrap.cc:654] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.677363 22059 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.678702 22164 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:04:47.684989 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:47.697564 22095 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.698390 22095 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.700964 22095 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:47.728601 22106 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.729224 22106 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.731338 22106 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.733378 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.734524 21856 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
W20260709 13:04:47.737329 21823 leader_election.cc:343] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.740397 21754 catalog_manager.cc:5697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.749701 22058 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.750211 22058 ts_tablet_manager.cc:1403] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.232s	user 0.025s	sys 0.022s
I20260709 13:04:47.755038 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.758348 22009 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:47.762025 22058 raft_consensus.cc:359] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.762887 22058 raft_consensus.cc:385] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.763279 22058 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.764766 22058 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.771502 22058 ts_tablet_manager.cc:1434] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:04:47.772573 22058 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.776917 21824 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.778110 22106 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.778523 22106 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.778879 22106 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.788498 22054 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.789168 22054 ts_tablet_manager.cc:1403] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.197s	user 0.024s	sys 0.017s
I20260709 13:04:47.792557 22106 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.792671 22054 raft_consensus.cc:359] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.794306 22054 raft_consensus.cc:385] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.794651 22054 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.795328 22054 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.796726 22168 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.794224 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:47.798281 22168 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.810576 22168 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.800799 22106 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.801258 22052 tablet_bootstrap.cc:492] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:47.811965 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.812244 22052 ts_tablet_manager.cc:1403] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.312s	user 0.020s	sys 0.039s
I20260709 13:04:47.812366 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:47.813330 21824 leader_election.cc:343] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.814065 22166 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.814194 22095 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.814759 22095 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:47.814584 22166 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.815732 22095 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.818728 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:47.820657 21823 leader_election.cc:343] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.821215 21823 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.816917 22166 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.823015 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:47.802146 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:47.804565 22054 ts_tablet_manager.cc:1434] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:04:47.824260 21860 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:04:47.824648 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.804730 22058 tablet_bootstrap.cc:654] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.808089 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:47.825138 22054 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.825923 22009 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.830850 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.833539 21899 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:47.834906 22166 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.836316 22106 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.816359 22052 raft_consensus.cc:359] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.837638 22052 raft_consensus.cc:385] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.838037 22052 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.840309 22052 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.840766 22168 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.844573 22168 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.843973 22009 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
W20260709 13:04:47.844918 21823 leader_election.cc:343] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.850162 22052 ts_tablet_manager.cc:1434] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.004s
I20260709 13:04:47.851598 22052 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.853121 21824 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.857681 22106 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:47.858819 21972 leader_election.cc:343] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.859031 22106 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:47.859447 22168 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.859704 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:47.860072 22106 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:47.861127 21823 leader_election.cc:343] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.861711 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.864279 22009 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:47.865360 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.866106 22230 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.866487 22230 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.866881 22230 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.867583 22054 tablet_bootstrap.cc:654] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.871004 22058 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.871640 22058 ts_tablet_manager.cc:1403] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.099s	user 0.010s	sys 0.004s
W20260709 13:04:47.877152 21972 leader_election.cc:343] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.879974 21972 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:47.879771 22230 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.880182 22058 raft_consensus.cc:359] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.880668 22095 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.880901 22058 raft_consensus.cc:385] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.881250 22058 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.883467 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:47.884903 22009 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.883777 22230 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.887674 22052 tablet_bootstrap.cc:654] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.883467 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:47.890424 22054 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.890976 22054 ts_tablet_manager.cc:1403] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.001s
W20260709 13:04:47.891597 21823 leader_election.cc:343] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.887460 22058 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.895714 22058 ts_tablet_manager.cc:1434] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:04:47.897090 22058 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:47.899410 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:47.898368 22054 raft_consensus.cc:359] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.900779 22054 raft_consensus.cc:385] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.901093 22054 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.901331 21934 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.902621 22009 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:47.904599 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:47.905025 22054 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.905764 22230 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.906234 22230 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:47.907169 22054 ts_tablet_manager.cc:1434] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.003s
I20260709 13:04:47.907130 22230 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.908281 22054 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.915688 22058 tablet_bootstrap.cc:654] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.926221 22054 tablet_bootstrap.cc:654] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.926676 22166 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.927160 22166 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.929191 22166 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.934033 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:47.934789 22009 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:47.934954 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:47.935554 21860 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:47.935957 21899 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:47.936823 22081 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.937294 22081 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.937661 22081 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.944380 22058 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.944890 22058 ts_tablet_manager.cc:1403] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.000s
I20260709 13:04:47.946539 22081 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.948819 22058 raft_consensus.cc:359] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.949573 22058 raft_consensus.cc:385] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.949735 22081 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:47.949747 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:47.950496 22052 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:47.950448 21860 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.951007 22052 ts_tablet_manager.cc:1403] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.006s
I20260709 13:04:47.949869 22058 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.952541 22058 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.954818 22058 ts_tablet_manager.cc:1434] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:04:47.956022 22058 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.960928 22052 raft_consensus.cc:359] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.961674 22052 raft_consensus.cc:385] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.961934 22052 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.963461 22058 tablet_bootstrap.cc:654] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.965575 22054 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:47.966048 22054 ts_tablet_manager.cc:1403] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.007s
I20260709 13:04:47.968822 22054 raft_consensus.cc:359] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.969538 22054 raft_consensus.cc:385] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.969800 22054 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.970988 21758 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.972339 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:47.973734 22009 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.974689 22058 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:47.973188 21860 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:47.975170 22058 ts_tablet_manager.cc:1403] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:04:47.978112 22058 raft_consensus.cc:359] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.978803 22058 raft_consensus.cc:385] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.979074 22058 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.979775 22058 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.980629 22054 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.981479 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:47.981918 22058 ts_tablet_manager.cc:1434] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:04:47.981739 22052 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.982676 22166 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.982774 22058 tablet_bootstrap.cc:492] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:47.983664 22166 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:47.984573 22166 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:47.985597 22054 ts_tablet_manager.cc:1434] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:04:47.986598 22054 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:47.983846 22052 ts_tablet_manager.cc:1434] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
I20260709 13:04:47.988867 22052 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:47.990343 22058 tablet_bootstrap.cc:654] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.995430 21754 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.997170 21754 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.000326 22009 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.009739 22054 tablet_bootstrap.cc:654] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.024294 22052 tablet_bootstrap.cc:654] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.029059 22237 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.029589 22237 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.038484 22058 tablet_bootstrap.cc:492] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:48.038971 22058 ts_tablet_manager.cc:1403] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.056s	user 0.006s	sys 0.025s
I20260709 13:04:48.041819 22237 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.042164 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.045096 22058 raft_consensus.cc:359] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.046166 22058 raft_consensus.cc:385] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.045972 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.046509 22058 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.047312 22058 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:48.048074 21972 leader_election.cc:343] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.048810 22058 ts_tablet_manager.cc:1434] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:04:48.049657 22058 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:48.051558 21972 leader_election.cc:343] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.052172 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:48.053797 22237 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.063736 22058 tablet_bootstrap.cc:654] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.089973 22052 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.090338 22052 ts_tablet_manager.cc:1403] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.102s	user 0.007s	sys 0.009s
I20260709 13:04:48.096585 21757 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.099792 22054 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:48.100289 22054 ts_tablet_manager.cc:1403] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.114s	user 0.026s	sys 0.019s
I20260709 13:04:48.102357 22164 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.103389 22054 raft_consensus.cc:359] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.102851 22164 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.104046 22054 raft_consensus.cc:385] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.104461 22054 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.105098 22054 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.106498 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.107110 21860 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:48.107669 22054 ts_tablet_manager.cc:1434] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:48.108541 22054 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:48.108230 22052 raft_consensus.cc:359] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.108917 22052 raft_consensus.cc:385] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.109233 22052 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.109485 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:48.110349 22166 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.110694 22166 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.111011 22166 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.111342 22052 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.111790 22058 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:48.112226 22058 ts_tablet_manager.cc:1403] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.014s
I20260709 13:04:48.114148 22058 raft_consensus.cc:359] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.114791 22058 raft_consensus.cc:385] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.115056 22058 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.115651 22058 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.117558 22058 ts_tablet_manager.cc:1434] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:04:48.119423 22052 ts_tablet_manager.cc:1434] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.002s
I20260709 13:04:48.120347 22058 tablet_bootstrap.cc:492] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:48.123873 22081 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.125241 22052 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:48.124802 22081 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.126888 22054 tablet_bootstrap.cc:654] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.126991 22081 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.129611 22237 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.129468 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.130126 22237 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.130326 22007 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:48.132107 22237 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.132355 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:48.134290 22121 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.134687 22121 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.135015 22121 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.137573 22058 tablet_bootstrap.cc:654] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.138824 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.139451 21856 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:48.141987 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:48.143107 22095 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.143726 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.144847 22095 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.145323 22095 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.145732 22095 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.150141 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.150981 21860 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:48.152456 22164 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.168948 22052 tablet_bootstrap.cc:654] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.199210 22121 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.201361 22121 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.184777 22166 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.205461 22166 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.206789 22164 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.207300 22164 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:48.209062 21972 leader_election.cc:343] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.206053 22166 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.211457 22164 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.216007 22095 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:48.217353 21972 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:48.218174 22095 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.218914 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:48.218817 22166 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.219480 21856 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.225144 22166 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.229641 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:48.230261 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:48.230518 21859 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.231813 21856 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
W20260709 13:04:48.232363 21972 leader_election.cc:343] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.233134 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:48.234951 22237 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.235450 22237 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:48.238257 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.236316 22237 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.239554 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:48.240043 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:48.242143 22007 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.253533 22058 tablet_bootstrap.cc:492] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:48.254264 22058 ts_tablet_manager.cc:1403] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.134s	user 0.012s	sys 0.001s
I20260709 13:04:48.255667 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:48.255977 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.256350 21856 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.258594 22052 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.259071 22052 ts_tablet_manager.cc:1403] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.134s	user 0.009s	sys 0.003s
I20260709 13:04:48.259840 22007 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.262322 22052 raft_consensus.cc:359] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.263063 22052 raft_consensus.cc:385] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.263334 22052 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.262729 21859 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.264936 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.265643 22054 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:48.266076 22054 ts_tablet_manager.cc:1403] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.158s	user 0.031s	sys 0.032s
I20260709 13:04:48.265672 21858 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:48.268093 22052 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.275898 22054 raft_consensus.cc:359] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.276285 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.276134 22058 raft_consensus.cc:359] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.276587 22054 raft_consensus.cc:385] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.276851 22058 raft_consensus.cc:385] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.276891 22054 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.277284 22058 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.277674 22054 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:48.279132 21899 leader_election.cc:343] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.278066 22058 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.279779 22054 ts_tablet_manager.cc:1434] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.004s
I20260709 13:04:48.280681 22054 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:48.280550 21757 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:48.282869 21899 leader_election.cc:343] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:48.283591 21899 leader_election.cc:343] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.284219 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:48.273403 22237 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:04:48.285691 22121 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.286211 22121 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:48.287063 22121 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.288199 22052 ts_tablet_manager.cc:1434] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:04:48.289119 22052 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
W20260709 13:04:48.285686 21899 leader_election.cc:343] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.312438 22058 ts_tablet_manager.cc:1434] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.001s
I20260709 13:04:48.313916 22058 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:48.297820 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.320981 21858 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:48.325233 22052 tablet_bootstrap.cc:654] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.334450 21856 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.347731 21757 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.359509 22054 tablet_bootstrap.cc:654] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.363584 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:48.364513 22121 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.364897 22121 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.364796 21898 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:48.365301 22121 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.365698 22164 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.366904 22164 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.368808 22164 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.374111 22230 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.374594 22230 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.377413 22166 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.377978 22230 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.377899 22166 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.378209 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.379976 22166 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:04:48.380136 21823 leader_election.cc:343] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.366626 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:48.384748 22081 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:48.395342 21972 consensus_peers.cc:597] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:48.395422 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.399216 22164 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.406011 22121 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.408087 22121 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.410326 22164 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.411711 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.412371 21859 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:48.420632 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:48.420684 21899 leader_election.cc:343] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.422047 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:48.421836 22058 tablet_bootstrap.cc:654] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.422613 22009 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.437906 22054 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:48.424489 22081 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:48.431550 21898 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:48.431684 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:48.439735 21856 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.438725 22054 ts_tablet_manager.cc:1403] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.158s	user 0.013s	sys 0.010s
I20260709 13:04:48.436686 22052 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.441489 22052 ts_tablet_manager.cc:1403] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.153s	user 0.019s	sys 0.036s
I20260709 13:04:48.427155 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:04:48.426380 21824 leader_election.cc:343] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.425338 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:48.434433 22009 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.443985 22166 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.444880 21859 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.445439 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:04:48.451225 21899 leader_election.cc:343] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.451886 22230 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.455921 21856 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.456111 22168 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.457804 22052 raft_consensus.cc:359] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.458524 22052 raft_consensus.cc:385] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.458789 22052 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.459584 22052 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.449299 22166 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.460753 22166 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.462456 22052 ts_tablet_manager.cc:1434] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.002s
I20260709 13:04:48.463584 22052 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:48.463380 22230 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.474867 21898 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:48.475683 22121 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.476078 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:48.473356 22095 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.477702 22095 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.441916 22081 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.480016 22095 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.491845 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.525831 22121 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:48.523391 21859 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.533821 22121 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.535969 22054 raft_consensus.cc:359] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.540355 22054 raft_consensus.cc:385] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.540694 22054 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
W20260709 13:04:48.536275 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60248.9us
W20260709 13:04:48.534894 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 60185.9us
I20260709 13:04:48.542934 22054 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.547216 22058 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:48.547890 22058 ts_tablet_manager.cc:1403] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.234s	user 0.007s	sys 0.011s
I20260709 13:04:48.550931 22054 ts_tablet_manager.cc:1434] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.110s	user 0.005s	sys 0.004s
I20260709 13:04:48.552388 22054 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
W20260709 13:04:48.557076 21823 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:48.559556 22058 raft_consensus.cc:359] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.560544 22058 raft_consensus.cc:385] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.550197 22164 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.560879 22058 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.560824 22052 tablet_bootstrap.cc:654] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.568007 22058 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.564049 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.587740 21859 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:48.576575 22166 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.583314 22164 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:48.589568 21934 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:48.590013 22164 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.593765 22166 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.596313 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:48.597743 22264 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.598096 22264 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.598410 22264 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.593858 22058 ts_tablet_manager.cc:1434] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:04:48.604877 22058 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:48.638082 21757 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:48.638803 21823 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:48.642030 22054 tablet_bootstrap.cc:654] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.647385 22264 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.649474 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:48.650321 22005 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 1: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:04:48.659726 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:48.660499 21934 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:48.661252 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:48.662010 21859 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.667388 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:48.669495 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:48.671360 22095 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.672124 22081 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.677593 22280 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.678694 22052 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.678349 22280 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.679349 22052 ts_tablet_manager.cc:1403] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.216s	user 0.014s	sys 0.006s
I20260709 13:04:48.657095 22264 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.680866 22280 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.690816 22058 tablet_bootstrap.cc:654] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.683413 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.692498 22081 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.687703 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.708415 21858 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 1: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:04:48.709933 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:48.686129 21754 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.710839 22280 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.684871 22052 raft_consensus.cc:359] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.713984 22052 raft_consensus.cc:385] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.714339 22052 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.715138 22052 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.719024 22280 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.711777 21754 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.721359 22052 ts_tablet_manager.cc:1434] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:04:48.722589 22052 tablet_bootstrap.cc:492] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:48.684361 21859 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:48.729300 21898 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:48.738626 22052 tablet_bootstrap.cc:654] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.741616 22058 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:48.742580 22058 ts_tablet_manager.cc:1403] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.138s	user 0.023s	sys 0.013s
I20260709 13:04:48.745988 22052 tablet_bootstrap.cc:492] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.746294 22054 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:48.754284 22054 ts_tablet_manager.cc:1403] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.202s	user 0.010s	sys 0.007s
I20260709 13:04:48.746536 22052 ts_tablet_manager.cc:1403] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.001s
I20260709 13:04:48.755625 22164 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.756121 22164 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:48.757906 22052 raft_consensus.cc:359] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.758056 22054 raft_consensus.cc:359] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.758087 22164 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.758745 22052 raft_consensus.cc:385] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.759229 22052 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.758831 22054 raft_consensus.cc:385] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.760006 22054 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.760041 22052 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.760829 22054 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.761989 22052 ts_tablet_manager.cc:1434] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:48.763113 22054 ts_tablet_manager.cc:1434] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:04:48.763319 22052 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:48.767874 22264 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.770048 22231 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.770617 22231 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.773250 22231 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.773083 22058 raft_consensus.cc:359] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.773906 22058 raft_consensus.cc:385] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.774245 22058 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.774994 22058 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.773368 22054 tablet_bootstrap.cc:492] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:48.781328 22058 ts_tablet_manager.cc:1434] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:04:48.781392 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.782333 22058 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:48.787878 21823 leader_election.cc:343] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.791556 22237 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.794035 22052 tablet_bootstrap.cc:654] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.797637 22054 tablet_bootstrap.cc:654] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.799402 22058 tablet_bootstrap.cc:654] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.801425 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.807725 22054 tablet_bootstrap.cc:492] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:48.808274 22054 ts_tablet_manager.cc:1403] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.002s
I20260709 13:04:48.811137 22054 raft_consensus.cc:359] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.811971 22054 raft_consensus.cc:385] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.812243 22054 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.812414 22052 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.812868 22052 ts_tablet_manager.cc:1403] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.004s
I20260709 13:04:48.812796 22054 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.814982 22054 ts_tablet_manager.cc:1434] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
W20260709 13:04:48.815739 21824 leader_election.cc:343] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.816267 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:48.816395 22052 raft_consensus.cc:359] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.817211 22052 raft_consensus.cc:385] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.817507 22052 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.817399 22231 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.820494 22054 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:48.823081 22052 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.824932 22058 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:48.825547 22058 ts_tablet_manager.cc:1403] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.000s
I20260709 13:04:48.826097 22052 ts_tablet_manager.cc:1434] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:04:48.827014 22054 tablet_bootstrap.cc:654] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.827041 22052 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:48.885233 22264 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.888247 22237 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.889554 22237 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.889674 21754 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.868077 22058 raft_consensus.cc:359] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.894328 22264 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.901600 22264 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.899053 22237 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.903577 22058 raft_consensus.cc:385] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.907655 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.909479 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.911026 21859 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:48.912397 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:48.915494 21972 leader_election.cc:343] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:48.922633 21972 leader_election.cc:343] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.923321 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:48.904481 22058 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.919433 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.926142 22081 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.927718 22081 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.927386 22058 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.933629 22058 ts_tablet_manager.cc:1434] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.108s	user 0.011s	sys 0.014s
I20260709 13:04:48.939891 22052 tablet_bootstrap.cc:654] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.936203 22237 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.941568 22266 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.944275 22266 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.946821 22058 tablet_bootstrap.cc:492] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:48.951360 22266 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:48.955318 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.949469 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:48.966073 22237 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.967163 22081 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.971086 22054 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:48.964836 22280 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:04:48.972153 22054 ts_tablet_manager.cc:1403] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.006s
I20260709 13:04:48.972894 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:48.973066 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:48.979563 22121 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:48.980144 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:48.973569 22164 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.985270 22121 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.985302 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.974668 22231 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.979120 22052 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:48.986207 22231 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.987322 22052 ts_tablet_manager.cc:1403] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.160s	user 0.010s	sys 0.007s
I20260709 13:04:48.981547 21855 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:48.990594 22054 raft_consensus.cc:359] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.991542 22054 raft_consensus.cc:385] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.991923 22054 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.992511 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:48.994206 21934 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:48.994402 22054 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.995096 22231 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:48.993522 22058 tablet_bootstrap.cc:654] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.990840 22052 raft_consensus.cc:359] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:48.996296 22052 raft_consensus.cc:385] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.996609 22052 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.997166 22121 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:49.001308 21823 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:49.001667 22237 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.003059 22230 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.003463 22230 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.003809 22230 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.007607 22054 ts_tablet_manager.cc:1434] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.002s
I20260709 13:04:49.012216 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:49.013836 21824 leader_election.cc:343] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.010284 22052 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.022396 22054 tablet_bootstrap.cc:492] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:49.019496 22052 ts_tablet_manager.cc:1434] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:04:49.040381 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:49.041811 22266 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.040037 22237 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.047480 22052 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:49.047125 22237 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.043692 21932 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.051498 22280 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.099996 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:49.049551 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47244.6us
I20260709 13:04:49.106534 22266 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.103500 21858 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:49.111001 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:49.112010 22237 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:49.112547 22237 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.112905 22237 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.113262 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:49.115105 21972 leader_election.cc:343] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.113261 22237 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.137262 22054 tablet_bootstrap.cc:654] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.137617 22058 tablet_bootstrap.cc:492] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:49.139760 22058 ts_tablet_manager.cc:1403] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.200s	user 0.008s	sys 0.012s
I20260709 13:04:49.131353 22237 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.124891 22230 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.148010 22230 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.150563 22279 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.150826 22058 raft_consensus.cc:359] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.151444 22058 raft_consensus.cc:385] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.151731 22058 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.152418 22058 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.154990 22058 ts_tablet_manager.cc:1434] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:04:49.155297 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:49.155989 22237 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:04:49.157582 21824 leader_election.cc:343] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.157869 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:49.158643 21856 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.162391 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:04:49.163663 21972 leader_election.cc:343] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.181757 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:49.182364 21934 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.184527 22058 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:49.190748 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:49.191594 21856 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:49.193049 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:49.194854 22266 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.195365 22266 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:49.196343 22279 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:49.196233 22266 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.201543 22052 tablet_bootstrap.cc:654] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.207762 22279 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:49.228549 21934 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:49.228521 22271 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.253825 21823 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:49.279296 22271 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.287787 22271 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.288889 22230 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.329109 22230 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:49.333910 22230 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.362489 22300 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.363142 22300 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.356793 22280 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 13:04:49.423696 22058 tablet_bootstrap.cc:654] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.426951 22230 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.434007 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:49.440925 22052 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:49.441411 22052 ts_tablet_manager.cc:1403] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.407s	user 0.023s	sys 0.004s
I20260709 13:04:49.442204 22054 tablet_bootstrap.cc:492] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:49.442703 22054 ts_tablet_manager.cc:1403] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.421s	user 0.004s	sys 0.027s
W20260709 13:04:49.457206 21823 leader_election.cc:343] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.461295 22054 raft_consensus.cc:359] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.462073 22054 raft_consensus.cc:385] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.462360 22054 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.463028 22054 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.472870 22054 ts_tablet_manager.cc:1434] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:04:49.437340 22230 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.469694 22081 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.490715 22052 raft_consensus.cc:359] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.491286 22052 raft_consensus.cc:385] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.491498 22052 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
W20260709 13:04:49.492241 22310 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.132s	user 0.002s	sys 0.002s
I20260709 13:04:49.485520 22081 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.502931 22081 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.504739 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.507339 22230 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.509569 22054 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:49.514031 22052 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.516350 22052 ts_tablet_manager.cc:1434] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.001s
I20260709 13:04:49.516362 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.517997 22052 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:49.536429 21934 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.537519 22054 tablet_bootstrap.cc:654] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.537694 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:49.542233 22058 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:49.542693 22058 ts_tablet_manager.cc:1403] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.358s	user 0.013s	sys 0.015s
I20260709 13:04:49.543563 22300 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.546655 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:49.548556 21824 leader_election.cc:343] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.549072 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:49.550172 22052 tablet_bootstrap.cc:654] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.549952 21758 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:49.551087 21823 leader_election.cc:343] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.552742 22230 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.555523 22058 raft_consensus.cc:359] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.556201 22058 raft_consensus.cc:385] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.556469 22058 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.560611 22058 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.562453 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.563062 22058 ts_tablet_manager.cc:1434] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 13:04:49.561146 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:49.564213 22058 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:49.567121 22271 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:49.565770 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.577549 21899 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:04:49.581880 21824 leader_election.cc:343] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.582432 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:49.586210 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:49.586309 22300 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.596660 22081 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.601696 22058 tablet_bootstrap.cc:654] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.614988 22054 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:49.615522 22054 ts_tablet_manager.cc:1403] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.125s	user 0.011s	sys 0.014s
I20260709 13:04:49.615255 22230 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.618312 22230 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.622905 21931 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.624929 22121 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.624723 22054 raft_consensus.cc:359] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.625785 22054 raft_consensus.cc:385] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.626155 22054 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.636125 22052 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:49.645211 22230 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.676374 22054 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.677502 22052 ts_tablet_manager.cc:1403] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.160s	user 0.014s	sys 0.014s
I20260709 13:04:49.679596 22054 ts_tablet_manager.cc:1434] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.064s	user 0.023s	sys 0.021s
I20260709 13:04:49.683290 22054 tablet_bootstrap.cc:492] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:49.683523 21935 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.680487 22052 raft_consensus.cc:359] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.684976 22052 raft_consensus.cc:385] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.681473 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:49.680984 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.687323 22052 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.688103 22052 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.689846 22054 tablet_bootstrap.cc:654] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.690410 22052 ts_tablet_manager.cc:1434] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:04:49.691417 22052 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:49.694247 22058 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:49.694887 22058 ts_tablet_manager.cc:1403] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.131s	user 0.015s	sys 0.000s
I20260709 13:04:49.696969 21754 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:49.699182 21824 leader_election.cc:343] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:49.713271 21823 leader_election.cc:343] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.701716 22054 tablet_bootstrap.cc:492] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
W20260709 13:04:49.703006 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:49.718452 22081 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:04:49.719053 22054 ts_tablet_manager.cc:1403] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.004s
I20260709 13:04:49.719152 22052 tablet_bootstrap.cc:654] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.719133 22081 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.723169 22054 raft_consensus.cc:359] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.723893 22054 raft_consensus.cc:385] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.724154 22054 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.724758 22054 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.726109 22081 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.726358 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:49.727095 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.729588 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:49.731781 22231 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.729583 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:49.730458 22054 ts_tablet_manager.cc:1434] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:04:49.736232 22081 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:49.708791 22058 raft_consensus.cc:359] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.737593 22058 raft_consensus.cc:385] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.737852 22058 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.740971 22054 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:49.739174 22058 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.744069 22058 ts_tablet_manager.cc:1434] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:04:49.752297 22058 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:49.762955 22058 tablet_bootstrap.cc:654] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.768513 22231 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.769272 22231 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.773128 22052 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:49.773671 22052 ts_tablet_manager.cc:1403] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.082s	user 0.024s	sys 0.021s
I20260709 13:04:49.774331 22231 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.777199 22054 tablet_bootstrap.cc:654] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.780441 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:49.781119 22007 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:49.798446 22300 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:49.800426 22264 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.800911 22264 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.803743 22264 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:49.798766 22237 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:04:49.805912 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:49.806843 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:49.808802 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:49.809687 22006 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.813475 22300 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.813860 22300 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.814220 22300 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.816028 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:49.798853 22052 raft_consensus.cc:359] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.820591 22052 raft_consensus.cc:385] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.821017 22052 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.783522 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:49.828042 22052 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.855317 22052 ts_tablet_manager.cc:1434] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
W20260709 13:04:49.856122 21823 leader_election.cc:343] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.866622 22052 tablet_bootstrap.cc:492] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:49.893177 22058 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:49.893966 22058 ts_tablet_manager.cc:1403] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.142s	user 0.031s	sys 0.045s
I20260709 13:04:49.898013 22264 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.905411 22264 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.907377 22054 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:49.908034 22054 ts_tablet_manager.cc:1403] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.167s	user 0.008s	sys 0.024s
I20260709 13:04:49.919152 22266 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.919656 22266 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.918241 22058 raft_consensus.cc:359] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.923036 22058 raft_consensus.cc:385] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.921769 22266 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:49.923410 22058 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.919864 22054 raft_consensus.cc:359] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.923996 22054 raft_consensus.cc:385] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.924239 22054 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
W20260709 13:04:49.924748 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 68692.4us
I20260709 13:04:49.925384 22264 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:49.928345 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:49.929436 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:49.930094 21932 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:49.930253 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:49.931589 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:49.931828 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:49.932420 21972 leader_election.cc:343] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.933112 22266 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.933660 22266 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.934104 22266 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.934643 22058 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:49.937501 21972 leader_election.cc:343] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.937922 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:49.925323 22054 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.933108 21860 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:49.945693 22059 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.945086 22300 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.946200 22054 ts_tablet_manager.cc:1434] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.002s
I20260709 13:04:49.954247 22237 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.956889 22300 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:49.958158 22054 tablet_bootstrap.cc:492] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:49.958271 22058 ts_tablet_manager.cc:1434] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.064s	user 0.004s	sys 0.004s
I20260709 13:04:49.959431 22058 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:49.965276 22266 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:49.968642 22052 tablet_bootstrap.cc:654] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.972492 22058 tablet_bootstrap.cc:654] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.975170 22320 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.998236 22266 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:49.999111 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:49.999905 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:50.007210 21860 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.007931 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:50.008682 22007 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.020166 22054 tablet_bootstrap.cc:654] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.001736 22052 tablet_bootstrap.cc:492] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:50.001724 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:50.020869 22007 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:50.021365 21932 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.031081 21932 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:50.032709 22052 ts_tablet_manager.cc:1403] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.166s	user 0.017s	sys 0.012s
I20260709 13:04:50.033555 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:50.036868 22059 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.020287 21860 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:50.037400 22059 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:50.038342 22059 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.042165 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.046041 22231 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.046517 22231 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:50.047340 22231 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.047953 22058 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.048596 22058 ts_tablet_manager.cc:1403] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.089s	user 0.016s	sys 0.019s
I20260709 13:04:50.067219 22058 raft_consensus.cc:359] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.068168 22058 raft_consensus.cc:385] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.068516 22058 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.069419 22058 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.073201 22058 ts_tablet_manager.cc:1434] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 13:04:50.074689 22058 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.079015 22052 raft_consensus.cc:359] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.083037 22052 raft_consensus.cc:385] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.084007 22058 tablet_bootstrap.cc:654] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:50.080760 21823 leader_election.cc:343] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.084214 22052 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.105168 22052 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:50.094679 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51265.8us
I20260709 13:04:50.112795 22058 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.113854 22058 ts_tablet_manager.cc:1403] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.016s
I20260709 13:04:50.116511 22058 raft_consensus.cc:359] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.117205 22058 raft_consensus.cc:385] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.117408 22058 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.118217 22058 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.121387 22058 ts_tablet_manager.cc:1434] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:04:50.122409 22058 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.112922 21756 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.133533 22058 tablet_bootstrap.cc:654] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.145067 22052 ts_tablet_manager.cc:1434] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
I20260709 13:04:50.157621 22231 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.158463 22231 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.161989 22231 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.167865 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:50.177603 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:50.180608 22279 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.176498 22052 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:50.181135 22279 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.181381 22059 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.181444 21754 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.182335 22059 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.208343 22059 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.209964 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:50.211416 21972 leader_election.cc:343] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.213966 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:50.207376 21824 leader_election.cc:343] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:50.220525 21972 leader_election.cc:343] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.218482 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:50.221108 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:50.223495 22054 tablet_bootstrap.cc:492] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
W20260709 13:04:50.227193 21898 leader_election.cc:343] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.227393 22059 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.229748 22279 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.230605 22054 ts_tablet_manager.cc:1403] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.266s	user 0.047s	sys 0.068s
I20260709 13:04:50.231781 22052 tablet_bootstrap.cc:654] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.243692 22054 raft_consensus.cc:359] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.244266 22054 raft_consensus.cc:385] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.244451 22054 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.246942 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:50.247651 22054 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.251192 22054 ts_tablet_manager.cc:1434] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.002s
I20260709 13:04:50.253598 22054 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
W20260709 13:04:50.257302 21823 leader_election.cc:343] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.257759 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:04:50.258245 21899 leader_election.cc:343] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.259274 21899 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:50.265184 22271 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.259037 22300 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.271831 22058 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.272375 22058 ts_tablet_manager.cc:1403] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.150s	user 0.031s	sys 0.007s
I20260709 13:04:50.278884 22058 raft_consensus.cc:359] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.279686 22058 raft_consensus.cc:385] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.284500 22054 tablet_bootstrap.cc:654] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.284662 22237 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.285189 22237 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.287310 22237 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.287937 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:50.288585 21856 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:50.289779 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.290755 22059 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.291149 22059 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.291487 22059 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.295014 22052 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:50.295580 22052 ts_tablet_manager.cc:1403] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.147s	user 0.019s	sys 0.009s
I20260709 13:04:50.297078 22058 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.297854 22058 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.298148 22052 raft_consensus.cc:359] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.298750 22052 raft_consensus.cc:385] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.299017 22052 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.300081 22052 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.302248 22052 ts_tablet_manager.cc:1434] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:50.303227 22052 tablet_bootstrap.cc:492] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
W20260709 13:04:50.304150 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:50.308840 22052 tablet_bootstrap.cc:654] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.311311 22237 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.311591 22058 ts_tablet_manager.cc:1434] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.004s
I20260709 13:04:50.314770 22058 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.314567 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:50.315498 21931 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:50.318626 21935 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.338299 22058 tablet_bootstrap.cc:654] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.343941 22054 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:50.353345 22054 ts_tablet_manager.cc:1403] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.100s	user 0.021s	sys 0.007s
I20260709 13:04:50.356249 22054 raft_consensus.cc:359] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.357125 22054 raft_consensus.cc:385] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.357496 22054 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.312033 22237 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.362285 22058 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.362890 22058 ts_tablet_manager.cc:1403] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.000s
I20260709 13:04:50.364610 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:50.366894 21855 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:50.368147 22237 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.368583 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.369902 22317 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.370841 22317 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.371382 22317 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.358795 22054 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.376210 22054 ts_tablet_manager.cc:1434] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:04:50.366429 22058 raft_consensus.cc:359] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.377444 22058 raft_consensus.cc:385] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.377810 22058 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.378494 22058 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.380865 22058 ts_tablet_manager.cc:1434] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:04:50.381765 22054 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:50.381846 22058 tablet_bootstrap.cc:492] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.388010 22058 tablet_bootstrap.cc:654] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.311336 22059 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.392975 22054 tablet_bootstrap.cc:654] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.394537 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:50.395321 21855 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.403663 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:50.404412 21935 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:50.407436 22317 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.403097 21855 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:50.413707 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.414502 22237 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.415892 22237 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:50.416558 22059 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.416786 22237 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.419564 22058 tablet_bootstrap.cc:492] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.445677 22266 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.461748 22058 ts_tablet_manager.cc:1403] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.006s
I20260709 13:04:50.464474 22058 raft_consensus.cc:359] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.465170 22058 raft_consensus.cc:385] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.465443 22058 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.466156 22230 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.466141 22058 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.466883 22230 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.468400 22058 ts_tablet_manager.cc:1434] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:04:50.469456 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:50.469885 22058 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.470072 21935 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.468815 22230 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.474162 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:50.474861 21931 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:50.476562 22058 tablet_bootstrap.cc:654] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.477540 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.478843 22230 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.479225 22230 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.480068 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:50.480093 22230 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.485271 22006 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:50.510216 22266 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.513504 21931 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.518369 21935 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:50.551457 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:50.552094 21855 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.552700 22317 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.554277 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:50.555076 21934 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.564158 21934 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:50.568725 22300 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.569201 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:50.569351 22300 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.571830 22300 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.572162 22230 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.575822 22052 tablet_bootstrap.cc:492] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:50.580699 22052 ts_tablet_manager.cc:1403] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.278s	user 0.073s	sys 0.132s
I20260709 13:04:50.581254 22231 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.582852 21756 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.583420 22052 raft_consensus.cc:359] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.584046 22052 raft_consensus.cc:385] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.584299 22052 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.584934 22052 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.577270 22317 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.586223 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:50.587472 21933 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.587472 22052 ts_tablet_manager.cc:1434] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:50.588601 22052 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:50.589973 22230 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.591255 22317 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:50.617036 22317 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.591529 21855 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:50.592589 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:50.595690 22052 tablet_bootstrap.cc:654] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.622840 21934 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:50.615468 21933 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:50.624161 21823 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.604957 22058 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.625641 22058 ts_tablet_manager.cc:1403] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.156s	user 0.036s	sys 0.030s
I20260709 13:04:50.631817 22058 raft_consensus.cc:359] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.632557 22058 raft_consensus.cc:385] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.632856 22058 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.633483 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.634301 22231 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.634346 22344 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.634799 22231 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.635035 22344 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:50.635246 22231 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.636023 22344 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.637215 22054 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:50.637670 22054 ts_tablet_manager.cc:1403] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.256s	user 0.026s	sys 0.045s
I20260709 13:04:50.640415 22054 raft_consensus.cc:359] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.641196 22054 raft_consensus.cc:385] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.641508 22054 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.643422 22231 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.644215 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:50.645776 22058 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.646347 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:50.645712 21754 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.646934 21934 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.647024 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:50.647697 22006 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.653774 22058 ts_tablet_manager.cc:1434] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.003s
I20260709 13:04:50.654938 22058 tablet_bootstrap.cc:492] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.655637 22006 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:50.657368 21934 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:50.658118 22231 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.659518 21823 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:50.660308 22231 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.660763 22231 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:50.663331 22052 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:50.663902 22052 ts_tablet_manager.cc:1403] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.076s	user 0.011s	sys 0.001s
I20260709 13:04:50.666209 22320 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.661664 22231 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.666819 22052 raft_consensus.cc:359] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.667464 22052 raft_consensus.cc:385] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.667806 22052 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
W20260709 13:04:50.668304 21824 leader_election.cc:343] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.671343 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:50.694041 21754 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.677735 22054 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.697654 22300 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.697919 22052 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.698376 22300 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.702751 22009 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.704093 22052 ts_tablet_manager.cc:1434] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:04:50.705224 22052 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:50.705770 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:50.707239 21823 leader_election.cc:343] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.710896 22052 tablet_bootstrap.cc:654] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.713553 22351 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.714022 22351 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.718086 22300 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.718190 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:50.718714 22351 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.720515 22058 tablet_bootstrap.cc:654] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:50.720531 21898 leader_election.cc:343] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.721485 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:50.721864 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:50.722152 22007 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:50.712956 22009 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
W20260709 13:04:50.723136 21899 leader_election.cc:343] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.723706 21899 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:50.724766 22351 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.725880 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:50.726889 22054 ts_tablet_manager.cc:1434] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.000s
I20260709 13:04:50.726938 22300 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.727834 22300 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.728304 22300 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.739748 22054 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:50.740458 22058 tablet_bootstrap.cc:492] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.742187 22058 ts_tablet_manager.cc:1403] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.001s
I20260709 13:04:50.741046 21756 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.748098 22058 raft_consensus.cc:359] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.748852 22058 raft_consensus.cc:385] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.749198 22058 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.750103 22058 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.755410 22300 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.758098 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:04:50.760865 21823 leader_election.cc:343] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.769550 22300 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.770470 22052 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:50.770991 22052 ts_tablet_manager.cc:1403] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.004s
I20260709 13:04:50.771026 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:50.771947 22006 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.773820 22052 raft_consensus.cc:359] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.774482 22052 raft_consensus.cc:385] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.774747 22052 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.779083 22052 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.781857 22052 ts_tablet_manager.cc:1434] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:04:50.782652 22058 ts_tablet_manager.cc:1434] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 13:04:50.791321 22058 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.798358 22054 tablet_bootstrap.cc:654] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.802652 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:50.809455 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:50.810979 22300 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.811618 22300 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:50.812674 22300 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.817855 22271 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.818408 22271 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.821655 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:50.823240 21899 leader_election.cc:343] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.825199 22271 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.826478 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:50.827142 21856 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:50.828487 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:50.829599 22271 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.829977 22271 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.830332 22271 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.782946 22052 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:50.845062 22052 tablet_bootstrap.cc:654] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.850026 22058 tablet_bootstrap.cc:654] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.851382 21756 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.858855 22359 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.852813 22271 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.859347 22359 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.861459 22359 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.861555 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:50.862146 21856 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.864473 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:50.865368 21855 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:50.865857 22271 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:50.866888 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:50.868229 21899 leader_election.cc:343] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.868351 21856 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:50.869334 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:50.869648 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:50.870807 22271 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.871297 22271 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:50.872150 22271 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.873772 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:04:50.874644 21972 leader_election.cc:343] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.875672 22359 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.876062 22359 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.876372 22359 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.882244 22052 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:50.882752 22052 ts_tablet_manager.cc:1403] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.100s	user 0.007s	sys 0.004s
I20260709 13:04:50.884435 22054 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:50.883594 22059 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.887912 22054 ts_tablet_manager.cc:1403] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.148s	user 0.026s	sys 0.025s
I20260709 13:04:50.890098 22052 raft_consensus.cc:359] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.898888 22052 raft_consensus.cc:385] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.899187 22052 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.899909 22052 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.902956 21756 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.904407 22359 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.907940 22359 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:50.912354 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:04:50.914040 21972 leader_election.cc:343] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.917883 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:50.918624 21856 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.921602 22054 raft_consensus.cc:359] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.922559 22054 raft_consensus.cc:385] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.922914 22054 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.923851 22054 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.929222 22058 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.929721 22058 ts_tablet_manager.cc:1403] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.139s	user 0.022s	sys 0.036s
I20260709 13:04:50.932643 22058 raft_consensus.cc:359] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.933291 22058 raft_consensus.cc:385] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.933529 22058 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.934300 22058 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.936759 22058 ts_tablet_manager.cc:1434] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:50.937822 22058 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.939513 22054 ts_tablet_manager.cc:1434] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:04:50.942173 22054 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:50.944185 22058 tablet_bootstrap.cc:654] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.947518 22359 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.960947 22058 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:50.961414 22058 ts_tablet_manager.cc:1403] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.008s
I20260709 13:04:50.969950 22058 raft_consensus.cc:359] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.970696 22058 raft_consensus.cc:385] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.971009 22058 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.971642 22058 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:50.973742 22058 ts_tablet_manager.cc:1434] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:04:50.975288 22058 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:50.981673 22058 tablet_bootstrap.cc:654] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.982986 22052 ts_tablet_manager.cc:1434] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.000s
I20260709 13:04:50.994278 22054 tablet_bootstrap.cc:654] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.008569 22058 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.009069 22058 ts_tablet_manager.cc:1403] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.006s
I20260709 13:04:51.010565 22052 tablet_bootstrap.cc:492] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:51.012013 22058 raft_consensus.cc:359] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.013286 22054 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.013922 22054 ts_tablet_manager.cc:1403] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.072s	user 0.008s	sys 0.004s
I20260709 13:04:51.014513 21856 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:51.015950 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:51.012851 22058 raft_consensus.cc:385] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.017361 22266 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.017362 22058 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.017865 22266 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:51.016448 22054 raft_consensus.cc:359] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.018730 22266 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.018769 22058 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.019375 22054 raft_consensus.cc:385] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.019840 22054 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.020646 22054 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.021092 22058 ts_tablet_manager.cc:1434] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:04:51.023479 22059 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.028435 22058 tablet_bootstrap.cc:492] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:51.035918 22054 ts_tablet_manager.cc:1434] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:04:51.037256 22054 tablet_bootstrap.cc:492] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:51.039194 22058 tablet_bootstrap.cc:654] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.045998 22344 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.046494 22344 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.047173 22052 tablet_bootstrap.cc:654] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.048609 22344 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.051244 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.051664 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.051951 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:51.052215 22006 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:51.053524 21824 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:51.055975 22300 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.056329 22300 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.056638 22300 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.063333 22058 tablet_bootstrap.cc:492] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.062420 21758 catalog_manager.cc:5697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.068241 22058 ts_tablet_manager.cc:1403] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.005s
I20260709 13:04:51.069597 22300 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.072535 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:51.073079 21935 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.073822 22054 tablet_bootstrap.cc:654] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.077517 22059 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.078123 22059 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.080122 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:51.080703 22059 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:51.081341 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:51.096678 22266 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.097478 22344 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.097986 22344 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:51.098587 22052 tablet_bootstrap.cc:492] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.099086 22052 ts_tablet_manager.cc:1403] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.089s	user 0.023s	sys 0.013s
I20260709 13:04:51.098846 22344 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.083526 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.092630 22054 tablet_bootstrap.cc:492] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.101929 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:51.102488 22006 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.102643 22300 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.102921 22054 ts_tablet_manager.cc:1403] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.066s	user 0.004s	sys 0.008s
I20260709 13:04:51.082268 22058 raft_consensus.cc:359] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.085434 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:51.104732 22058 raft_consensus.cc:385] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:51.104828 21972 leader_election.cc:343] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.105101 22058 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
W20260709 13:04:51.105474 21972 leader_election.cc:343] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.105842 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:51.109280 22351 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.122031 22351 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.108709 22054 raft_consensus.cc:359] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.123445 22054 raft_consensus.cc:385] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.120783 22230 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.124752 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:51.124974 22230 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.110538 22231 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:04:51.109097 22059 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.106822 22058 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:51.126180 21898 leader_election.cc:343] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.135324 22006 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:51.141233 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.138015 22052 raft_consensus.cc:359] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.143736 22052 raft_consensus.cc:385] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:51.144899 21824 leader_election.cc:343] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.138034 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.146046 21935 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:51.147146 21823 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:51.144897 22052 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.148262 22052 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.148751 22320 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.157150 22320 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.125560 22351 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.127609 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.155870 22054 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.157634 22320 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.159554 22052 ts_tablet_manager.cc:1434] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.000s
W20260709 13:04:51.159837 21899 leader_election.cc:343] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.162829 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:51.161264 22058 ts_tablet_manager.cc:1434] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.092s	user 0.013s	sys 0.000s
I20260709 13:04:51.164795 22351 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.141235 22230 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.166337 22058 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:51.175678 22320 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.178390 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:51.178918 21935 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.184183 22054 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.185858 22320 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.167165 22052 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:51.187808 22054 ts_tablet_manager.cc:1434] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.085s	user 0.001s	sys 0.005s
I20260709 13:04:51.188759 22054 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:51.191043 21935 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:51.191303 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:51.193122 21824 leader_election.cc:343] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.198441 21823 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:51.201117 22058 tablet_bootstrap.cc:654] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.202819 22230 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.204588 22230 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:51.207002 22052 tablet_bootstrap.cc:654] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.205590 22230 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.202819 21756 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.213546 22271 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.214015 22271 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.211719 22231 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:04:51.217540 22271 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.221194 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.222481 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:51.223223 21855 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 1: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:04:51.225224 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:51.226529 22231 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.227833 22054 tablet_bootstrap.cc:654] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.229921 22351 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.231159 21933 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.240599 22059 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.241240 22059 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.243481 22059 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:51.250042 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.259963 22320 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.260512 22052 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.260975 22052 ts_tablet_manager.cc:1403] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.094s	user 0.009s	sys 0.018s
I20260709 13:04:51.258810 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:51.263613 21855 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:51.264743 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:51.265645 22266 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.265122 22052 raft_consensus.cc:359] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.273722 22052 raft_consensus.cc:385] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.275671 22052 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.281098 21758 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.287501 22052 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.285218 22351 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.304728 22052 ts_tablet_manager.cc:1434] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.003s
I20260709 13:04:51.320395 22266 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.328341 22271 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.328913 22271 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.333884 22058 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.334532 22058 ts_tablet_manager.cc:1403] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.168s	user 0.014s	sys 0.004s
W20260709 13:04:51.352015 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72685.5us
I20260709 13:04:51.357084 22320 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:51.366124 22058 raft_consensus.cc:359] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.366855 22058 raft_consensus.cc:385] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.367105 22058 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.367758 22058 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.368780 22054 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.369549 22054 ts_tablet_manager.cc:1403] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.181s	user 0.013s	sys 0.014s
W20260709 13:04:51.369302 22396 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:04:51.381103 22058 ts_tablet_manager.cc:1434] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260709 13:04:51.382872 22052 tablet_bootstrap.cc:492] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
W20260709 13:04:51.383749 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:51.387444 22271 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.387212 22054 raft_consensus.cc:359] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.387935 22054 raft_consensus.cc:385] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.388195 22054 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.388700 22054 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.393702 22058 tablet_bootstrap.cc:492] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:51.400581 22121 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.406783 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.407475 22008 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:51.408918 22058 tablet_bootstrap.cc:654] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.409441 22054 ts_tablet_manager.cc:1434] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:04:51.410595 22054 tablet_bootstrap.cc:492] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:51.421736 22052 tablet_bootstrap.cc:654] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.425513 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:51.425858 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:51.431872 22054 tablet_bootstrap.cc:654] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.433100 22230 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.439033 22052 tablet_bootstrap.cc:492] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.439528 22052 ts_tablet_manager.cc:1403] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.057s	user 0.005s	sys 0.005s
I20260709 13:04:51.442643 22230 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.443377 22230 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.446187 22230 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.446750 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:51.448500 21823 leader_election.cc:343] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.449769 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.453763 22271 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:51.454895 22271 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:04:51.455297 22271 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.442631 22052 raft_consensus.cc:359] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.457691 22052 raft_consensus.cc:385] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.457964 22052 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.458668 22052 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.460345 22009 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:51.460965 22052 ts_tablet_manager.cc:1434] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:04:51.461941 22052 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:51.462558 22271 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.466169 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:51.469116 22230 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.469524 22230 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.469918 22230 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.475162 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:51.475626 22271 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.477066 22230 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.478331 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:51.480216 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:51.481125 22271 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:51.481910 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:51.482115 22402 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.482514 22009 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.484418 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:51.485265 22230 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:04:51.487669 21823 leader_election.cc:343] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.488783 22052 tablet_bootstrap.cc:654] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.490839 22009 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:51.504195 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:51.495807 22230 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.500064 21935 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:51.508131 22231 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.516139 22231 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:51.518851 22059 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.519333 22059 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.522243 22059 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:51.522468 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:51.530069 21972 leader_election.cc:343] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.516680 22300 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:04:51.536877 22320 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.537242 22320 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:51.538000 22320 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.539181 22320 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.518550 22231 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.548651 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.553114 22054 tablet_bootstrap.cc:492] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.562855 22054 ts_tablet_manager.cc:1403] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.153s	user 0.031s	sys 0.009s
W20260709 13:04:51.559957 21972 leader_election.cc:343] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.549305 22317 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:04:51.565256 21972 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:51.561599 22058 tablet_bootstrap.cc:492] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.566335 22058 ts_tablet_manager.cc:1403] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.173s	user 0.015s	sys 0.012s
I20260709 13:04:51.567417 22059 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.566735 22230 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.580204 22058 raft_consensus.cc:359] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.580946 22058 raft_consensus.cc:385] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.581310 22058 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.584548 22230 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.587827 22054 raft_consensus.cc:359] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.588240 22230 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.588578 22054 raft_consensus.cc:385] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.588944 22054 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.589718 22054 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.594635 22058 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.595585 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.599995 22058 ts_tablet_manager.cc:1434] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.001s
W20260709 13:04:51.604380 21824 leader_election.cc:343] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.611413 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.611891 22054 ts_tablet_manager.cc:1434] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.003s
I20260709 13:04:51.612185 21932 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:51.609038 21756 catalog_manager.cc:5697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.616947 21823 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:51.618350 22054 tablet_bootstrap.cc:492] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:51.627538 22058 tablet_bootstrap.cc:492] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:51.619240 22230 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.632258 22230 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.632714 22230 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.605705 22231 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:04:51.634680 22300 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.636320 22317 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.636505 22300 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.643304 22300 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.645995 22391 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.646476 22391 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.650928 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.651372 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:51.652650 21823 leader_election.cc:343] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:51.653138 21824 leader_election.cc:343] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.653554 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:51.654879 22300 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.657968 22391 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.661409 22052 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.661350 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:51.663427 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:04:51.668254 21824 leader_election.cc:343] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:51.674002 21823 leader_election.cc:343] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.674455 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:51.664939 22052 ts_tablet_manager.cc:1403] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.203s	user 0.012s	sys 0.013s
I20260709 13:04:51.675606 22300 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.677430 22054 tablet_bootstrap.cc:654] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.673615 22230 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:51.690199 22035 tablet.cc:2406] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:51.701566 22230 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.699239 22052 raft_consensus.cc:359] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.703163 22052 raft_consensus.cc:385] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.703522 22052 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.704214 22052 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.704538 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:51.706142 21824 leader_election.cc:343] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.710254 22052 ts_tablet_manager.cc:1434] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:04:51.710299 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:51.711004 21935 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.727115 22058 tablet_bootstrap.cc:654] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.741436 21935 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:51.742583 21823 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:51.742836 22052 tablet_bootstrap.cc:492] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:51.743877 22391 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.744894 22391 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:51.745172 22414 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:04:51.745908 22391 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.749442 22054 tablet_bootstrap.cc:492] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.750625 22054 ts_tablet_manager.cc:1403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.009s
I20260709 13:04:51.792059 22052 tablet_bootstrap.cc:654] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.796622 22058 tablet_bootstrap.cc:492] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.799474 22058 ts_tablet_manager.cc:1403] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.180s	user 0.017s	sys 0.002s
I20260709 13:04:51.803377 22058 raft_consensus.cc:359] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.809336 22058 raft_consensus.cc:385] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.809620 22058 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.812091 22008 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.809074 21756 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.819255 22054 raft_consensus.cc:359] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.820065 22054 raft_consensus.cc:385] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.820345 22054 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.821326 22054 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.823164 22391 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.811525 22058 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.829653 22058 ts_tablet_manager.cc:1434] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:04:51.846637 22058 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:51.832370 22052 tablet_bootstrap.cc:492] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.849193 22052 ts_tablet_manager.cc:1403] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.121s	user 0.013s	sys 0.006s
I20260709 13:04:51.851610 22052 raft_consensus.cc:359] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.852120 22052 raft_consensus.cc:385] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.852351 22052 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.853327 22052 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.857213 22058 tablet_bootstrap.cc:654] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.823836 22054 ts_tablet_manager.cc:1434] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.000s
I20260709 13:04:51.862372 22054 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:51.864298 22052 ts_tablet_manager.cc:1434] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:04:51.868865 22054 tablet_bootstrap.cc:654] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.870499 22052 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:51.878067 22052 tablet_bootstrap.cc:654] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.903239 22058 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.903784 22058 ts_tablet_manager.cc:1403] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.057s	user 0.006s	sys 0.006s
I20260709 13:04:51.904083 22052 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.904533 22052 ts_tablet_manager.cc:1403] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:04:51.906955 22058 raft_consensus.cc:359] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.907219 22052 raft_consensus.cc:359] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.907716 22058 raft_consensus.cc:385] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.907889 22052 raft_consensus.cc:385] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.908062 22058 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.908255 22052 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.908825 22058 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.908942 22052 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.911119 22052 ts_tablet_manager.cc:1434] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:51.911221 22058 ts_tablet_manager.cc:1434] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:51.912405 22058 tablet_bootstrap.cc:492] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:51.913599 22054 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.914053 22054 ts_tablet_manager.cc:1403] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.007s
I20260709 13:04:51.920318 22052 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:51.924873 22054 raft_consensus.cc:359] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.925616 22054 raft_consensus.cc:385] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.925889 22054 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.926508 22054 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.928696 22054 ts_tablet_manager.cc:1434] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.001s
I20260709 13:04:51.931159 22052 tablet_bootstrap.cc:654] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.931308 22058 tablet_bootstrap.cc:654] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.933666 22054 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:51.941092 22054 tablet_bootstrap.cc:654] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.943642 22058 tablet_bootstrap.cc:492] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:51.944248 22058 ts_tablet_manager.cc:1403] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.004s
I20260709 13:04:51.950228 22058 raft_consensus.cc:359] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.951144 22058 raft_consensus.cc:385] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.951473 22058 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.972162 21935 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.975476 22052 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:51.975977 22052 ts_tablet_manager.cc:1403] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.004s
I20260709 13:04:51.978658 22052 raft_consensus.cc:359] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.979326 22052 raft_consensus.cc:385] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.979581 22052 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.980216 22414 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.981357 22058 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.980218 22052 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.986302 22414 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.986941 22414 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.989445 22054 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:51.989904 22054 ts_tablet_manager.cc:1403] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.056s	user 0.007s	sys 0.024s
I20260709 13:04:51.990887 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:51.994393 21824 leader_election.cc:343] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.995118 22414 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:51.996021 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:51.992532 22054 raft_consensus.cc:359] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:51.997028 22054 raft_consensus.cc:385] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.997324 22054 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
W20260709 13:04:51.997479 21823 leader_election.cc:343] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.997895 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:51.999198 22414 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.001830 22054 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.003960 22054 ts_tablet_manager.cc:1434] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:04:52.007443 22052 ts_tablet_manager.cc:1434] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:04:52.008576 22052 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:52.014472 22052 tablet_bootstrap.cc:654] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.020112 22391 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:52.024721 22058 ts_tablet_manager.cc:1434] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 13:04:52.028534 22058 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:52.072101 22121 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.072635 22121 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.074708 22121 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.078998 22054 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:52.081081 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.081739 21855 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:52.084587 21898 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.085430 22399 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.085789 22399 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.086099 22399 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.094883 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:52.095801 22009 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:52.098570 22399 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.101284 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:52.101871 21858 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.110373 22054 tablet_bootstrap.cc:654] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.118783 22058 tablet_bootstrap.cc:654] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.118624 21858 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:52.120091 22399 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.126549 21898 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.127513 22402 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.127689 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:52.128129 22402 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:52.128636 22008 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.129236 22402 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.129933 22052 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:52.132121 22052 ts_tablet_manager.cc:1403] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.124s	user 0.038s	sys 0.077s
I20260709 13:04:52.132870 22391 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.134232 22391 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.136281 22391 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.136672 22052 raft_consensus.cc:359] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.138126 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.138525 22052 raft_consensus.cc:385] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.139611 22052 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
W20260709 13:04:52.140179 21823 leader_election.cc:343] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.141139 22052 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.145392 22052 ts_tablet_manager.cc:1434] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:52.148623 22052 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:52.162420 22052 tablet_bootstrap.cc:654] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.165403 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:52.166234 22006 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:52.172634 22054 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:52.172574 22008 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:52.173197 22054 ts_tablet_manager.cc:1403] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.095s	user 0.019s	sys 0.015s
I20260709 13:04:52.175823 22054 raft_consensus.cc:359] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.176463 21824 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:52.177505 22391 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.177881 22391 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.178126 22052 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:52.178211 22391 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.178587 22052 ts_tablet_manager.cc:1403] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.005s
I20260709 13:04:52.181324 22052 raft_consensus.cc:359] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.181941 22399 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.182587 22399 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.181972 22052 raft_consensus.cc:385] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.183655 22052 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.176467 22054 raft_consensus.cc:385] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.185308 22054 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.185947 22054 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.186610 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.187278 21855 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:52.187995 22054 ts_tablet_manager.cc:1434] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:04:52.188477 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.189285 22121 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.189680 22121 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.190056 22121 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.184286 22052 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.194129 22052 ts_tablet_manager.cc:1434] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:04:52.188921 22054 tablet_bootstrap.cc:492] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:52.196183 22399 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.196708 22399 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.199277 22391 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.199553 22399 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.200827 22054 tablet_bootstrap.cc:654] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.201388 22317 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.201905 22317 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.205991 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.206952 22052 tablet_bootstrap.cc:492] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
W20260709 13:04:52.207655 21972 leader_election.cc:343] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.209263 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.210016 21855 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:52.205544 22317 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:52.210184 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.211460 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:04:52.213587 21972 leader_election.cc:343] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.214098 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:52.214385 22399 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.212905 22052 tablet_bootstrap.cc:654] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.216122 22317 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.219154 22121 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.223414 22317 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.223982 22317 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.226586 22391 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.228847 22317 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:52.231323 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.238806 21756 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.239975 21935 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:52.241302 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:52.236092 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.242223 22413 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.237309 22402 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.242611 22413 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.242985 22413 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.238224 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:52.245283 22008 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.255031 22008 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:52.256225 21824 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.239068 22058 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:52.259634 22058 ts_tablet_manager.cc:1403] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.231s	user 0.009s	sys 0.023s
W20260709 13:04:52.261551 21972 leader_election.cc:343] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.266920 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:52.243002 22402 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.264227 22300 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.269328 21858 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.273068 22052 tablet_bootstrap.cc:492] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:52.273488 22052 ts_tablet_manager.cc:1403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.067s	user 0.021s	sys 0.008s
I20260709 13:04:52.276100 22052 raft_consensus.cc:359] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.276757 22052 raft_consensus.cc:385] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.277032 22052 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.277624 22052 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.288017 21858 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:52.288461 22317 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.289049 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.288914 22317 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.290270 22439 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.290711 22439 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:52.291570 22317 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:52.292197 22300 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:52.292791 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.294543 22052 ts_tablet_manager.cc:1434] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:04:52.295167 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.295821 21856 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 1: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:52.298170 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:52.298887 22052 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:52.299994 22411 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.294389 22300 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.270957 22402 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:52.314697 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 122372us
I20260709 13:04:52.316398 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:52.319226 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:52.321058 22121 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.329135 22052 tablet_bootstrap.cc:654] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.297374 22439 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.334939 22058 raft_consensus.cc:359] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.335718 22058 raft_consensus.cc:385] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.336057 22058 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.336942 22054 tablet_bootstrap.cc:492] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:52.337456 22054 ts_tablet_manager.cc:1403] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.149s	user 0.025s	sys 0.020s
I20260709 13:04:52.341779 22054 raft_consensus.cc:359] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.342581 22054 raft_consensus.cc:385] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.342913 22054 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.344110 22054 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.318274 22391 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:04:52.345808 22411 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.333420 22413 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.352470 22006 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:52.354856 22054 ts_tablet_manager.cc:1434] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:04:52.360047 22006 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.359598 22058 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.362739 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:52.365991 22054 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:52.365975 21756 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.386052 22230 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:52.388872 22391 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.399196 22391 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.413646 22359 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.414207 22359 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.414649 22058 ts_tablet_manager.cc:1434] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.154s	user 0.007s	sys 0.002s
I20260709 13:04:52.416106 22359 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:52.417604 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.418336 21858 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:52.419618 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.419788 22391 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.422016 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:52.422606 22006 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:52.425836 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.425709 21824 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.428793 22230 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.429188 22230 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.429437 22230 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.437770 22317 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.438246 22317 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.438617 22317 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.438701 22052 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:52.439769 22052 ts_tablet_manager.cc:1403] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.009s
I20260709 13:04:52.442858 22058 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
W20260709 13:04:52.447044 21972 leader_election.cc:343] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.450070 22230 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.452950 22230 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.454104 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:52.454672 22006 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:52.456861 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:52.460175 22058 tablet_bootstrap.cc:654] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.458959 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.466568 22052 raft_consensus.cc:359] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.467260 22052 raft_consensus.cc:385] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.467521 22052 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.468150 22052 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.469605 22054 tablet_bootstrap.cc:654] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.470620 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:52.472544 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:52.473533 22052 ts_tablet_manager.cc:1434] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:04:52.474617 22052 tablet_bootstrap.cc:492] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:52.477308 22006 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:52.480449 21824 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.475600 21754 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.484763 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:52.485460 21934 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:52.488512 21972 leader_election.cc:343] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.471685 21931 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
W20260709 13:04:52.489425 21899 leader_election.cc:343] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.479255 22413 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:52.487641 22391 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.468940 22317 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.498306 21934 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:52.499565 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:52.493348 22391 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:52.494692 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:52.495272 21823 leader_election.cc:343] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.505959 22391 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.485813 22402 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.529747 22402 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:04:52.531176 22444 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.003s
I20260709 13:04:52.501403 22413 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.534137 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:52.534869 21856 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.536027 22006 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:52.532006 22413 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:52.554833 22413 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.561978 22317 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:04:52.568876 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60962.2us
I20260709 13:04:52.577394 21856 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:52.578969 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:52.580084 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:04:52.590780 21899 leader_election.cc:343] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.608191 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:52.594616 22443 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.630038 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:52.630674 21856 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.637902 21856 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:52.634766 22443 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:04:52.652731 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70659.9us
I20260709 13:04:52.654774 22121 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:52.662438 22058 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:52.662902 22058 ts_tablet_manager.cc:1403] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.229s	user 0.020s	sys 0.013s
I20260709 13:04:52.686960 22052 tablet_bootstrap.cc:654] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.683789 22443 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.695389 22439 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:04:52.710610 22058 raft_consensus.cc:359] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.711355 22058 raft_consensus.cc:385] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.711627 22058 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.712839 22058 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.697167 22413 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.726372 22459 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:52.742851 22058 ts_tablet_manager.cc:1434] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 13:04:52.778720 22058 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:52.794499 22054 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:52.732246 22413 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.795097 22054 ts_tablet_manager.cc:1403] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.435s	user 0.016s	sys 0.019s
W20260709 13:04:52.802327 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 209349us
I20260709 13:04:52.816622 22052 tablet_bootstrap.cc:492] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:52.817168 22052 ts_tablet_manager.cc:1403] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.343s	user 0.013s	sys 0.022s
I20260709 13:04:52.822633 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:04:52.825603 21972 leader_election.cc:343] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.821740 22462 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.842681 22462 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.871333 22413 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:52.880277 22450 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.879321 22052 raft_consensus.cc:359] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.881475 21930 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.881182 22052 raft_consensus.cc:385] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.883224 22052 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
W20260709 13:04:52.883620 22466 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:04:52.884004 22052 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.886255 22450 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:52.882622 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.892745 22052 ts_tablet_manager.cc:1434] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.075s	user 0.003s	sys 0.005s
I20260709 13:04:52.893803 22052 tablet_bootstrap.cc:492] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:52.894997 22058 tablet_bootstrap.cc:654] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:52.898782 21972 leader_election.cc:343] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.899608 22462 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:52.900121 22054 raft_consensus.cc:359] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.900799 22054 raft_consensus.cc:385] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.901093 22054 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.900641 21758 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.901626 22054 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.902066 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:04:52.903576 21824 leader_election.cc:343] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.903683 22054 ts_tablet_manager.cc:1434] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.108s	user 0.000s	sys 0.004s
I20260709 13:04:52.887128 22450 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.904007 21756 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.913894 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:52.912581 21754 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:52.916335 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:52.917577 21854 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:52.918104 22054 tablet_bootstrap.cc:492] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:52.954234 22416 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:04:52.956219 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:52.958673 22317 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.959070 22317 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.959435 22317 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.964751 22413 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:04:52.965469 22413 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:52.979382 22411 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.029997 22440 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
W20260709 13:04:53.037277 22474 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:53.073736 22317 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.078293 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:04:53.080432 21972 leader_election.cc:343] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.076764 22317 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:53.108809 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:53.109671 21930 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:53.111254 22413 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:53.112627 22440 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.113376 22440 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.117216 22440 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.125254 22058 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:53.125710 22058 ts_tablet_manager.cc:1403] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.371s	user 0.022s	sys 0.022s
I20260709 13:04:53.128700 22058 raft_consensus.cc:359] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.129535 22058 raft_consensus.cc:385] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.129844 22058 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.130661 22058 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.132244 22058 ts_tablet_manager.cc:1434] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:53.138250 22058 tablet_bootstrap.cc:492] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:53.141477 22054 tablet_bootstrap.cc:654] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.145028 22058 tablet_bootstrap.cc:654] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:53.161092 21823 leader_election.cc:343] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.161675 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:53.162581 22463 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.110992 22052 tablet_bootstrap.cc:654] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.165580 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.179989 22463 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.183367 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:53.184157 21859 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.196738 22058 tablet_bootstrap.cc:492] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:53.197257 22058 ts_tablet_manager.cc:1403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.059s	user 0.026s	sys 0.016s
I20260709 13:04:53.197743 21932 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.202172 22058 raft_consensus.cc:359] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.210788 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.211896 22121 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.202857 22058 raft_consensus.cc:385] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.214486 22058 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.215894 22052 tablet_bootstrap.cc:492] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:53.215375 22058 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.216451 22052 ts_tablet_manager.cc:1403] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.323s	user 0.003s	sys 0.033s
I20260709 13:04:53.217970 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.218814 21858 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:04:53.222687 22052 raft_consensus.cc:359] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.223707 22052 raft_consensus.cc:385] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:53.223837 21972 consensus_peers.cc:597] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:53.224738 22052 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.225059 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:53.230120 22473 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:53.230530 22473 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:04:53.230898 22473 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:53.235617 22052 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.236754 22002 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.241869 21859 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:53.242988 22121 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.243949 22121 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.245718 22121 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.242389 22473 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.246493 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:53.248060 21898 leader_election.cc:343] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.249060 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:53.249589 22052 ts_tablet_manager.cc:1434] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.033s	user 0.001s	sys 0.006s
I20260709 13:04:53.249858 21930 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:53.251695 22473 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:53.243150 22058 ts_tablet_manager.cc:1434] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:04:53.253201 22058 tablet_bootstrap.cc:492] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:53.255074 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:53.256362 22411 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.257063 22411 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:53.257859 22411 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.260069 22054 tablet_bootstrap.cc:492] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:53.260581 22054 ts_tablet_manager.cc:1403] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.343s	user 0.009s	sys 0.003s
I20260709 13:04:53.261026 22058 tablet_bootstrap.cc:654] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.263680 22052 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:53.270327 22054 raft_consensus.cc:359] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.271435 22054 raft_consensus.cc:385] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.271960 22054 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.272914 22054 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.271899 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:53.274295 22058 tablet_bootstrap.cc:492] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:53.274760 22058 ts_tablet_manager.cc:1403] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 13:04:53.275198 22054 ts_tablet_manager.cc:1434] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:04:53.276036 22054 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:53.277179 22058 raft_consensus.cc:359] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.277803 22058 raft_consensus.cc:385] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.278079 22058 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.278651 22058 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.280463 22058 ts_tablet_manager.cc:1434] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:04:53.281340 22058 tablet_bootstrap.cc:492] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:53.285745 22052 tablet_bootstrap.cc:654] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.285797 21858 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:04:53.286651 22058 tablet_bootstrap.cc:654] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.290210 22054 tablet_bootstrap.cc:654] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.292480 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:53.297021 22413 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:53.297505 22413 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:53.298280 22058 tablet_bootstrap.cc:492] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:53.298267 22413 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.299778 22058 ts_tablet_manager.cc:1403] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:04:53.308223 21758 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:53.311127 22054 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:53.311584 22054 ts_tablet_manager.cc:1403] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.001s
I20260709 13:04:53.311717 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.317653 22300 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.318089 22058 raft_consensus.cc:359] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.318800 22058 raft_consensus.cc:385] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.319041 22058 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.319669 22058 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.321260 22058 ts_tablet_manager.cc:1434] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:04:53.322192 22058 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:53.325557 22054 raft_consensus.cc:359] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.326196 22054 raft_consensus.cc:385] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.326460 22054 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.328044 22058 tablet_bootstrap.cc:654] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.332878 21899 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:53.335356 22054 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.337167 22445 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:53.345934 22058 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:53.346392 22058 ts_tablet_manager.cc:1403] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.013s
I20260709 13:04:53.348990 22058 raft_consensus.cc:359] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.349684 22058 raft_consensus.cc:385] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.349938 22058 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.350611 22058 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.351539 22439 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.351924 22054 ts_tablet_manager.cc:1434] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:04:53.352769 22058 ts_tablet_manager.cc:1434] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:53.352936 22054 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:53.353716 22058 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Bootstrap starting.
I20260709 13:04:53.354177 22443 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:53.356573 21899 leader_election.cc:343] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.357033 21899 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:53.358511 22054 tablet_bootstrap.cc:654] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.359632 22058 tablet_bootstrap.cc:654] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.362725 22413 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.363386 22413 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.367472 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.368355 22413 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:53.374099 22443 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.376338 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:53.378769 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:53.380386 22411 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:53.383898 22052 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:53.384555 22052 ts_tablet_manager.cc:1403] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.134s	user 0.012s	sys 0.011s
I20260709 13:04:53.392735 22052 raft_consensus.cc:359] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.393673 22052 raft_consensus.cc:385] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.394001 22052 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.401657 22052 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.412879 22052 ts_tablet_manager.cc:1434] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.001s
I20260709 13:04:53.415127 22413 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.418483 22054 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:53.418969 22054 ts_tablet_manager.cc:1403] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.066s	user 0.023s	sys 0.039s
I20260709 13:04:53.424549 22058 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: No bootstrap required, opened a new log
I20260709 13:04:53.425112 22058 ts_tablet_manager.cc:1403] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Time spent bootstrapping tablet: real 0.072s	user 0.030s	sys 0.038s
I20260709 13:04:53.427925 22058 raft_consensus.cc:359] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.428565 22058 raft_consensus.cc:385] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.428856 22058 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.429509 22058 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.430217 22054 raft_consensus.cc:359] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.430866 22054 raft_consensus.cc:385] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.431106 22054 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.431658 22058 ts_tablet_manager.cc:1434] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:53.431759 22054 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.438459 22052 tablet_bootstrap.cc:492] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Bootstrap starting.
I20260709 13:04:53.444183 22054 ts_tablet_manager.cc:1434] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:04:53.445119 22054 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Bootstrap starting.
I20260709 13:04:53.452409 22054 tablet_bootstrap.cc:654] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.454417 21930 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.463654 22473 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.470837 22054 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: No bootstrap required, opened a new log
I20260709 13:04:53.471583 22054 ts_tablet_manager.cc:1403] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.006s
I20260709 13:04:53.474231 22054 raft_consensus.cc:359] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.474998 22054 raft_consensus.cc:385] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.475394 22054 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.476608 22054 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.477741 22453 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.478308 22453 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.482144 22054 ts_tablet_manager.cc:1434] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:04:53.484052 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.484700 22002 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:53.491305 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.492285 22230 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.492641 22230 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.492931 22230 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.510268 22453 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.513211 22230 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.532357 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:53.532980 22002 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.537844 22230 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.540800 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:53.541734 21928 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:53.545096 22413 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.545629 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:53.546211 21928 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.545635 22413 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.552837 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.553476 21859 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:53.554463 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.555148 22411 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.555500 22411 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.555836 22411 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.558079 22052 tablet_bootstrap.cc:654] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.563293 21758 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:53.564813 22230 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.565310 22230 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.566658 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:53.567142 22230 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.567301 21934 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 0.
I20260709 13:04:53.567023 22411 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.568538 22413 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:53.570734 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:53.570982 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:53.571319 21859 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.571476 21934 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.578083 21934 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:53.579244 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:53.580176 22411 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:53.580941 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.544785 22002 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:53.583597 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.585297 22453 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.585772 22453 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:53.586649 22453 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.597380 21855 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.599747 22413 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.600306 22413 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:53.601146 22413 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.603739 22443 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.604283 22443 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.608363 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.608963 21858 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 0.
I20260709 13:04:53.612663 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.615047 22439 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.615430 22439 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.615754 22439 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.624578 22439 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.627149 22459 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.627658 22459 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.629657 22459 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.630426 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:53.630978 21858 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.634250 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.634850 22008 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 0.
I20260709 13:04:53.635768 21824 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.636518 22459 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.636869 22459 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.637208 22459 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.641563 22443 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.647466 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.653071 21859 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:04:53.644240 22459 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.661784 22439 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.668401 22443 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.668922 22443 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.674427 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:53.675069 22002 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.678149 21858 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:53.679461 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.680748 22445 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.681421 22445 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:04:53.682240 22445 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.685833 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:53.688542 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:53.689157 21933 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.694007 21758 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:53.696875 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:53.699178 22052 tablet_bootstrap.cc:492] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: No bootstrap required, opened a new log
I20260709 13:04:53.699620 22052 ts_tablet_manager.cc:1403] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent bootstrapping tablet: real 0.261s	user 0.009s	sys 0.000s
I20260709 13:04:53.710532 21859 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:53.711979 22473 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:53.713872 21933 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:53.715049 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.715639 21855 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
W20260709 13:04:53.689080 21899 leader_election.cc:343] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.716566 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.717980 22443 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.716915 22459 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.720631 21757 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.722924 21928 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:53.724355 22440 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:53.724566 22002 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:04:53.724767 22440 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.725140 22440 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:53.727123 21824 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:53.728972 22300 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.729131 22052 raft_consensus.cc:359] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.729477 22300 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:53.730365 22300 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.733948 22052 raft_consensus.cc:385] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.734377 22052 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.738878 21934 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.765833 22052 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.788030 22052 ts_tablet_manager.cc:1434] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent starting tablet: real 0.088s	user 0.005s	sys 0.004s
W20260709 13:04:53.792025 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 75406.4us
I20260709 13:04:53.773388 22440 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.799266 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:53.799988 22008 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.809758 21823 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:53.811738 22300 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.825325 22008 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:53.822228 21928 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:53.838824 21806 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:53.845295 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.828262 22443 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.828368 22413 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:04:53.876279 22440 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.879710 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:53.858145 22443 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.894932 22443 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.895444 21754 catalog_manager.cc:5697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.905953 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:53.907552 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:53.914211 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:53.914655 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:53.914923 21859 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:04:53.915477 22439 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:53.916633 21898 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:04:53.917539 22510 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:53.917984 22510 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.918387 22510 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:53.947607 22514 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:53.950333 22473 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:53.950693 22473 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:53.957031 21758 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.960047 22464 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.960317 22473 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.964926 22473 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:53.966043 22508 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.091s	user 0.001s	sys 0.003s
I20260709 13:04:53.961148 22510 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:53.975142 21930 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:04:53.978780 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:53.981631 22510 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:53.985213 21757 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:53.986078 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:53.987846 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:53.988560 22510 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:53.995069 22413 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.995965 22413 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:53.996346 22413 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:53.998140 22413 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:54.022601 22413 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:54.091526 22512 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:54.095391 22473 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:54.103647 22512 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:54.118088 22515 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:54.137883 22510 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:54.158347 22511 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:54.190537 22512 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:54.214911 22414 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:55.218542 22513 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:04:55.219092 22513 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.221424 22513 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:55.222479 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:55.222769 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:55.223265 21933 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:55.223803 22005 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:04:55.225147 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:04:55.226076 22513 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:55.226416 22513 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:04:55.226737 22513 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.233809 22513 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.236301 22513 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:55.237932 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:55.238919 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:04:55.246634 22005 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:04:55.238897 21933 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:04:55.247759 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:04:55.248733 22513 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.249308 22513 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:04:55.250946 22513 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.263679 21757 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.301309 22515 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.302013 22515 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.310037 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:55.310909 21933 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:04:55.314329 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:55.315375 22473 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:55.315771 22473 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.316121 22473 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.328222 22515 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:55.333554 22473 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.352347 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:04:55.361994 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 83755us
I20260709 13:04:55.417220 22473 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:04:55.419803 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:04:55.420758 21930 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.439769 21930 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:04:55.451392 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:04:55.452498 22515 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.453128 22515 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:04:55.453691 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:04:55.454228 22515 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:04:55.472502 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 58466.8us
I20260709 13:04:55.507578 22512 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:04:55.508205 22512 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.511415 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:04:55.511662 22512 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:55.512336 21933 raft_consensus.cc:2368] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3.
I20260709 13:04:55.513733 21823 leader_election.cc:400] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3.
I20260709 13:04:55.514050 21823 leader_election.cc:403] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:55.514166 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:55.518062 22414 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.523736 21758 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.528147 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 13:04:55.535207 22414 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3.
W20260709 13:04:55.582042 22539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:55.596678 22540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:55.604794 22525 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.608841 22525 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:04:55.613273 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:04:55.661554 22525 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:55.666553 21930 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:55.667255 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:04:55.669729 22003 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:55.671578 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:04:55.672837 22414 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:55.673341 22541 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:55.678520 22543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:55.702183 20746 server_base.cc:1061] running on GCE node
I20260709 13:04:55.705818 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:55.706223 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:55.706489 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602295706457 us; error 0 us; skew 500 ppm
I20260709 13:04:55.707310 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:55.712055 20746 webserver.cc:533] Webserver started at http://127.20.66.132:38365/ using document root <none> and password file <none>
I20260709 13:04:55.712769 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:55.713088 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:55.713553 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:55.715061 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-3-root/instance:
uuid: "34f3b83da84448bf8aac62b37a1486eb"
format_stamp: "Formatted at 2026-07-09 13:04:55 on dist-test-slave-rgrd"
I20260709 13:04:55.729941 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:04:55.747592 22005 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:55.749346 22414 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:55.763536 22553 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:55.767196 20746 fs_manager.cc:730] Time spent opening block manager: real 0.034s	user 0.001s	sys 0.012s
I20260709 13:04:55.767699 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-3-root
uuid: "34f3b83da84448bf8aac62b37a1486eb"
format_stamp: "Formatted at 2026-07-09 13:04:55 on dist-test-slave-rgrd"
I20260709 13:04:55.768149 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:55.806387 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:55.808157 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:55.810149 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:55.813074 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:55.813349 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:55.813642 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:55.813956 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:55.892877 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.132:46099
I20260709 13:04:55.893045 22617 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.132:46099 every 8 connection(s)
I20260709 13:04:55.900851 20746 auto_leader_rebalancer-test.cc:384] add a tserver: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:04:55.925726 22618 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46167
I20260709 13:04:55.926298 22618 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:55.927554 22618 heartbeater.cc:507] Master 127.20.66.190:46167 requested a full tablet report, sending...
I20260709 13:04:55.930097 21758 ts_manager.cc:194] Registered new tserver with Master: 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:04:55.932267 21758 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53918
I20260709 13:04:55.933844 21857 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:55.936379 22473 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.956116 21930 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:55.958501 22531 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:56.949554 22618 heartbeater.cc:499] Master 127.20.66.190:46167 was elected leader, sending a full tablet report...
I20260709 13:04:57.111654 22003 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:57.131173 21857 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:57.131634 21930 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:57.132774 22536 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:57.133775 22473 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:57.163264 22473 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.164801 21930 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.216053 21857 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:04:57.233090 21974 consensus_peers.cc:597] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d059242e679f4cbaab1cc83ba8353d60. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.244387 21757 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.291154 21857 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:57.305751 22001 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:57.305723 21930 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:57.307803 22512 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:57.308871 22513 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:57.325273 21825 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 430edce57310489986e4f739148461a6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.332654 22512 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.335318 21931 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.336540 22001 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.372291 21757 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.384902 22005 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:04:57.395453 21974 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d41480ad8e164cf2aa88df180060345c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.395550 21930 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:57.396840 21859 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:57.398150 22634 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:57.400416 22631 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:57.415707 21859 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.414659 22634 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.424714 21930 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.440920 21859 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:57.445576 21754 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.474501 22005 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:57.475714 21825 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f588420255042dda396ba6419abf6bc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.476090 22513 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:57.488803 21930 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:57.491413 22532 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.493513 22001 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.502739 22513 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:57.522672 21754 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.524452 21931 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:04:57.755945 22654 ts_tablet_manager.cc:933] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:57.766829 22654 tablet_copy_client.cc:323] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:04:57.788511 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet d059242e679f4cbaab1cc83ba8353d60 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:04:57.789325 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet d059242e679f4cbaab1cc83ba8353d60 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-d059242e679f4cbaab1cc83ba8353d60
I20260709 13:04:57.799233 22019 tablet_copy_source_session.cc:215] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:57.808940 22654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d059242e679f4cbaab1cc83ba8353d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:57.829360 22654 tablet_copy_client.cc:806] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:57.830194 22654 tablet_copy_client.cc:670] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:57.836023 22654 tablet_copy_client.cc:538] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:57.847782 22654 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:04:57.853893 22658 ts_tablet_manager.cc:933] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:04:57.866317 22660 ts_tablet_manager.cc:933] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:04:57.876489 22660 tablet_copy_client.cc:323] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:04:57.876991 22658 tablet_copy_client.cc:323] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:04:57.890208 22654 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:57.891438 22654 tablet_bootstrap.cc:492] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:04:57.892220 22654 ts_tablet_manager.cc:1403] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.004s
I20260709 13:04:57.896579 22654 raft_consensus.cc:359] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:57.897563 22654 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:04:57.898288 22654 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:57.904735 21870 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 430edce57310489986e4f739148461a6 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:04:57.905774 21870 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 430edce57310489986e4f739148461a6 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-430edce57310489986e4f739148461a6
I20260709 13:04:57.912441 22654 ts_tablet_manager.cc:1434] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 13:04:57.917407 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-d059242e679f4cbaab1cc83ba8353d60 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:04:57.918031 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-d059242e679f4cbaab1cc83ba8353d60 on tablet d059242e679f4cbaab1cc83ba8353d60 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:04:57.918316 21870 tablet_copy_source_session.cc:215] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:57.919534 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 5f588420255042dda396ba6419abf6bc from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:04:57.920176 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 5f588420255042dda396ba6419abf6bc from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-5f588420255042dda396ba6419abf6bc
I20260709 13:04:57.922534 22658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430edce57310489986e4f739148461a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:57.938135 21869 tablet_copy_source_session.cc:215] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:57.949968 22660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f588420255042dda396ba6419abf6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:57.969220 22658 tablet_copy_client.cc:806] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:57.969898 22658 tablet_copy_client.cc:670] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:57.972208 22660 tablet_copy_client.cc:806] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:57.972875 22660 tablet_copy_client.cc:670] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:57.976963 22658 tablet_copy_client.cc:538] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:57.982759 22660 tablet_copy_client.cc:538] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:57.992421 22654 ts_tablet_manager.cc:933] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:04:57.995816 22658 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:04:58.012753 22660 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:04:58.020536 22654 tablet_copy_client.cc:323] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:04:58.023172 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet d41480ad8e164cf2aa88df180060345c from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:04:58.024037 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet d41480ad8e164cf2aa88df180060345c from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-d41480ad8e164cf2aa88df180060345c
I20260709 13:04:58.040337 22658 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:58.041469 22658 tablet_bootstrap.cc:492] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:04:58.051457 22658 ts_tablet_manager.cc:1403] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.056s	user 0.021s	sys 0.011s
I20260709 13:04:58.056344 22658 raft_consensus.cc:359] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:58.057303 22658 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:04:58.058003 22658 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:58.064356 22658 ts_tablet_manager.cc:1434] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:04:58.067214 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-430edce57310489986e4f739148461a6 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:04:58.067752 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-430edce57310489986e4f739148461a6 on tablet 430edce57310489986e4f739148461a6 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:04:58.077670 22019 tablet_copy_source_session.cc:215] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:58.081756 22654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41480ad8e164cf2aa88df180060345c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:58.118839 22654 tablet_copy_client.cc:806] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:58.119529 22654 tablet_copy_client.cc:670] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:58.125560 22654 tablet_copy_client.cc:538] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:58.132531 22660 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:58.133674 22660 tablet_bootstrap.cc:492] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:04:58.135155 22660 ts_tablet_manager.cc:1403] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.123s	user 0.027s	sys 0.003s
I20260709 13:04:58.137318 22654 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:04:58.139174 22660 raft_consensus.cc:359] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:58.141460 22660 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:04:58.142294 22660 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:58.150053 22660 ts_tablet_manager.cc:1434] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:04:58.153525 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-5f588420255042dda396ba6419abf6bc received from {username='slave'} at 127.0.0.1:42922
I20260709 13:04:58.154021 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-5f588420255042dda396ba6419abf6bc on tablet 5f588420255042dda396ba6419abf6bc with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:04:58.166504 22654 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:58.167639 22654 tablet_bootstrap.cc:492] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:04:58.168409 22654 ts_tablet_manager.cc:1403] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.005s
I20260709 13:04:58.171675 22654 raft_consensus.cc:359] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:58.172588 22654 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:04:58.173300 22654 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:04:58.175657 22654 ts_tablet_manager.cc:1434] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:04:58.178402 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-d41480ad8e164cf2aa88df180060345c received from {username='slave'} at 127.0.0.1:32904
I20260709 13:04:58.178874 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-d41480ad8e164cf2aa88df180060345c on tablet d41480ad8e164cf2aa88df180060345c with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:04:58.238946 22593 raft_consensus.cc:1217] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:04:58.308106 22593 raft_consensus.cc:1217] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:58.485862 22593 raft_consensus.cc:1217] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:58.606169 22593 raft_consensus.cc:1217] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:58.618184 22536 raft_consensus.cc:1064] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:04:58.621531 22536 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:58.633287 21931 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:58.634043 22593 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:58.636225 22633 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:58.641121 21857 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:58.642932 22634 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:58.645145 22633 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:58.661717 22633 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.666729 21931 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.693670 21858 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.694902 22593 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.707923 21754 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:58.808151 22001 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:58.813930 22593 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:58.814128 21931 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:58.815994 22631 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:58.817251 22633 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:58.826131 22676 raft_consensus.cc:1064] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:04:58.827113 22536 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.829743 22676 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:58.829282 22593 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.833956 21933 raft_consensus.cc:2955] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.850041 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d059242e679f4cbaab1cc83ba8353d60 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:58.868286 21756 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.883868 22592 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:58.885141 22001 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:58.887109 21933 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:58.887496 22665 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:58.890800 22665 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:58.893944 22673 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:58.928053 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet d059242e679f4cbaab1cc83ba8353d60 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41260
I20260709 13:04:58.937629 22675 raft_consensus.cc:1064] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:04:58.941663 22675 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:58.944967 22690 tablet_replica.cc:333] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:04:58.951764 22690 raft_consensus.cc:2243] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:58.954844 22665 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:58.965719 22690 raft_consensus.cc:2272] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.012229 22690 ts_tablet_manager.cc:1916] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:59.028837 22592 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.028350 22593 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.039256 21931 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.049628 22005 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.051853 21928 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.051360 22512 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.063565 22674 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.070930 22675 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.149484 21933 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.147118 22674 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.161628 22634 raft_consensus.cc:1064] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:04:59.168278 22634 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:59.167174 21756 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.226199 22593 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.238842 22631 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.247898 22001 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.263852 21931 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.263541 22005 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.269608 21857 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.274354 22634 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.277212 22672 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:59.340600 21858 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:59.340833 21756 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.384730 22592 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:59.387264 22665 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:59.413818 22006 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:59.416530 22512 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:59.420220 22631 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.422928 22513 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.430011 22690 ts_tablet_manager.cc:1929] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:04:59.430728 22690 log.cc:1199] T d059242e679f4cbaab1cc83ba8353d60 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/d059242e679f4cbaab1cc83ba8353d60
I20260709 13:04:59.431772 22592 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.440047 21857 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.444789 22593 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.472162 21933 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.495414 22593 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.442330 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet d059242e679f4cbaab1cc83ba8353d60 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
W20260709 13:04:59.500721 21742 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 59748.7us
I20260709 13:04:59.503641 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5f588420255042dda396ba6419abf6bc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:59.516182 22006 raft_consensus.cc:2955] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:04:59.534386 21742 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 5f588420255042dda396ba6419abf6bc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:04:59.540735 21754 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:59.557322 21758 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.582787 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f588420255042dda396ba6419abf6bc with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:04:59.583798 21857 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:59.594812 22006 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:04:59.603348 22593 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:59.603961 21859 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:59.606132 22631 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:59.612317 22631 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:59.616019 22592 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:59.617719 22513 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:59.624050 21933 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:59.628332 22513 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:59.630512 22673 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.633958 22592 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.644564 22593 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.658334 22701 tablet_replica.cc:333] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:04:59.659919 22701 raft_consensus.cc:2243] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.628914 22631 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.660826 22701 raft_consensus.cc:2272] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.680123 21933 raft_consensus.cc:2955] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.682845 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 430edce57310489986e4f739148461a6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:59.681823 21857 raft_consensus.cc:2955] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.695325 21756 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.709372 22701 ts_tablet_manager.cc:1916] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:59.725157 21754 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:04:59.733911 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet d41480ad8e164cf2aa88df180060345c with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:04:59.752291 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d41480ad8e164cf2aa88df180060345c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:59.753291 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 430edce57310489986e4f739148461a6 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:04:59.782676 22701 ts_tablet_manager.cc:1929] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:59.783646 22701 log.cc:1199] T 5f588420255042dda396ba6419abf6bc P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/5f588420255042dda396ba6419abf6bc
I20260709 13:04:59.785741 22701 tablet_replica.cc:333] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:04:59.788527 22702 tablet_replica.cc:333] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:04:59.795084 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 5f588420255042dda396ba6419abf6bc (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:04:59.805590 22702 raft_consensus.cc:2243] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.806433 22702 raft_consensus.cc:2272] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.807861 22701 raft_consensus.cc:2243] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.808971 22701 raft_consensus.cc:2272] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.813730 22702 ts_tablet_manager.cc:1916] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:59.814083 22701 ts_tablet_manager.cc:1916] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:59.833925 22702 ts_tablet_manager.cc:1929] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:59.833925 22701 ts_tablet_manager.cc:1929] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:59.835162 22701 log.cc:1199] T d41480ad8e164cf2aa88df180060345c P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/d41480ad8e164cf2aa88df180060345c
I20260709 13:04:59.834645 22702 log.cc:1199] T 430edce57310489986e4f739148461a6 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/430edce57310489986e4f739148461a6
I20260709 13:04:59.837662 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet d41480ad8e164cf2aa88df180060345c (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:04:59.837904 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 430edce57310489986e4f739148461a6 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:00.882483 21931 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:00.900204 21857 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:00.900584 22001 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:00.902845 22696 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:00.904065 22670 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:00.918785 22670 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:00.920976 22006 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:05:00.924759 21900 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c96543776baa4cdda2e6862202de4e52. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.935643 21857 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:00.942260 21931 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:00.978998 21754 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:00.996600 22006 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:00.997470 21900 consensus_peers.cc:597] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 951fdd0c64f542cbab511fb32cfa91cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.998152 22713 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:01.000056 21857 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.001535 22713 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:01.024367 22710 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.028898 22006 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.042196 21857 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.046069 21931 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:01.057873 21900 consensus_peers.cc:597] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f466d510c1b46d481ab83686b150d61. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:01.059916 21859 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.059962 22001 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.061661 22713 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:01.065840 22710 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:01.072722 22710 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.078187 21857 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.087936 22006 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.099435 21859 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.116060 22006 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.127756 21928 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.124071 22673 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:01.137669 21756 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.142303 22512 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:01.146863 21825 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a91310e68464d64aa8324092182518f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:01.166253 21757 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.257655 21928 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.268620 22673 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:05:01.325980 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47100.4us
I20260709 13:05:01.327297 22733 ts_tablet_manager.cc:933] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:01.337518 22733 tablet_copy_client.cc:323] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:01.353981 21757 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.378939 22006 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:01.389263 21945 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet c96543776baa4cdda2e6862202de4e52 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:01.390002 21945 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet c96543776baa4cdda2e6862202de4e52 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-c96543776baa4cdda2e6862202de4e52
I20260709 13:05:01.405088 21945 tablet_copy_source_session.cc:215] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:01.414116 22733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96543776baa4cdda2e6862202de4e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:01.430109 22733 tablet_copy_client.cc:806] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:01.430680 22733 tablet_copy_client.cc:670] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:01.434449 22733 tablet_copy_client.cc:538] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:01.440799 22733 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:01.458494 22733 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.459416 22733 tablet_bootstrap.cc:492] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:01.460080 22733 ts_tablet_manager.cc:1403] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.009s
I20260709 13:05:01.463166 22733 raft_consensus.cc:359] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.463969 22733 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:01.464677 22733 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.467633 22733 ts_tablet_manager.cc:1434] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.005s
I20260709 13:05:01.469581 21945 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-c96543776baa4cdda2e6862202de4e52 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:01.470049 21945 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-c96543776baa4cdda2e6862202de4e52 on tablet c96543776baa4cdda2e6862202de4e52 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:01.562288 22733 ts_tablet_manager.cc:933] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:01.563980 22733 tablet_copy_client.cc:323] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:01.565735 21945 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 6f466d510c1b46d481ab83686b150d61 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:01.566210 21945 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 6f466d510c1b46d481ab83686b150d61 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-6f466d510c1b46d481ab83686b150d61
I20260709 13:05:01.572286 21945 tablet_copy_source_session.cc:215] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:01.575138 22733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f466d510c1b46d481ab83686b150d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:01.594754 22740 ts_tablet_manager.cc:933] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:01.602988 22733 tablet_copy_client.cc:806] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:01.604090 22733 tablet_copy_client.cc:670] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:01.612180 22740 tablet_copy_client.cc:323] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:01.614511 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 951fdd0c64f542cbab511fb32cfa91cb from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:01.615105 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 951fdd0c64f542cbab511fb32cfa91cb from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-951fdd0c64f542cbab511fb32cfa91cb
I20260709 13:05:01.622468 22733 tablet_copy_client.cc:538] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:01.632128 21944 tablet_copy_source_session.cc:215] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:01.633117 22733 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:01.650049 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 951fdd0c64f542cbab511fb32cfa91cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:01.654721 22742 ts_tablet_manager.cc:933] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:01.668453 22742 tablet_copy_client.cc:323] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:01.670758 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 2a91310e68464d64aa8324092182518f from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:01.671339 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 2a91310e68464d64aa8324092182518f from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-2a91310e68464d64aa8324092182518f
I20260709 13:05:01.679639 21869 tablet_copy_source_session.cc:215] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:01.683324 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91310e68464d64aa8324092182518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:01.691375 22733 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.692516 22733 tablet_bootstrap.cc:492] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:01.693249 22733 ts_tablet_manager.cc:1403] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.030s
I20260709 13:05:01.697763 22733 raft_consensus.cc:359] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.699374 22733 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:01.700271 22733 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.700819 22742 tablet_copy_client.cc:806] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:01.701562 22742 tablet_copy_client.cc:670] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:01.707315 22733 ts_tablet_manager.cc:1434] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:05:01.710740 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-6f466d510c1b46d481ab83686b150d61 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:01.711213 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-6f466d510c1b46d481ab83686b150d61 on tablet 6f466d510c1b46d481ab83686b150d61 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:01.718771 22742 tablet_copy_client.cc:538] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:01.717335 22740 tablet_copy_client.cc:806] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:01.719743 22740 tablet_copy_client.cc:670] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:01.724598 22740 tablet_copy_client.cc:538] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:01.732021 22740 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:01.733896 22742 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:01.762789 22740 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.763693 22740 tablet_bootstrap.cc:492] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:01.764346 22740 ts_tablet_manager.cc:1403] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.014s
I20260709 13:05:01.773731 22740 raft_consensus.cc:359] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.774757 22740 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:01.775422 22740 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.777822 22740 ts_tablet_manager.cc:1434] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.002s
I20260709 13:05:01.780030 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-951fdd0c64f542cbab511fb32cfa91cb received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:01.780449 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-951fdd0c64f542cbab511fb32cfa91cb on tablet 951fdd0c64f542cbab511fb32cfa91cb with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:01.780668 22742 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.781620 22742 tablet_bootstrap.cc:492] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:01.782284 22742 ts_tablet_manager.cc:1403] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.016s
I20260709 13:05:01.785414 22742 raft_consensus.cc:359] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.786383 22742 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:01.787062 22742 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:01.789364 22742 ts_tablet_manager.cc:1434] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:05:01.790979 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-2a91310e68464d64aa8324092182518f received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:01.791399 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-2a91310e68464d64aa8324092182518f on tablet 2a91310e68464d64aa8324092182518f with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:01.882328 22592 raft_consensus.cc:1217] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:01.956447 22592 raft_consensus.cc:1217] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:02.011747 22592 raft_consensus.cc:1217] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:02.091944 22592 raft_consensus.cc:1217] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:02.391363 22747 raft_consensus.cc:1064] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:02.397063 22747 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.401368 22713 raft_consensus.cc:1064] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:02.409767 22592 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.410553 22005 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.412725 21860 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.406203 22713 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.419025 22670 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.423537 22710 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:05:02.445780 22750 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.490545 22747 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.492784 22593 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.498706 22592 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.500975 22749 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.505704 22005 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:05:02.506830 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 90420.5us
I20260709 13:05:02.507462 22749 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.511415 21860 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.521311 22749 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:05:02.530068 22005 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.536234 21757 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.562783 21855 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.563042 22750 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.576747 22703 raft_consensus.cc:1064] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:02.575652 21860 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.580186 22703 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.580668 22592 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.590514 22006 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.636520 22593 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.639765 22006 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.640128 21931 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.641520 22703 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.641664 21928 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.641777 21754 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.642649 22673 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:02.648794 22713 raft_consensus.cc:1064] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:02.655478 22001 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:02.661180 22710 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:02.663069 22593 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:02.665910 22764 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.666087 22747 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:02.669719 22713 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.680352 22750 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.695978 22593 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.690771 22005 raft_consensus.cc:2955] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.705293 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c96543776baa4cdda2e6862202de4e52 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:02.716415 21754 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.725293 22703 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.726835 21928 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.731346 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet c96543776baa4cdda2e6862202de4e52 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:02.734164 22006 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.741683 22591 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.758213 22749 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:02.760412 22006 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.762266 22591 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:02.770727 22001 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:02.769152 22750 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:05:02.771862 22747 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:02.776048 21860 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:02.779356 22749 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:02.781661 22670 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:02.784979 22710 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.784085 21754 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.825948 22005 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.821228 21932 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.818744 22770 tablet_replica.cc:333] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:02.804502 22591 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.849858 21757 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.848349 22593 raft_consensus.cc:2955] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.863081 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f466d510c1b46d481ab83686b150d61 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:02.872241 22770 raft_consensus.cc:2243] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.873135 22770 raft_consensus.cc:2272] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.882341 22770 ts_tablet_manager.cc:1916] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:02.882524 22670 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.886431 22590 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.902010 22001 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.921888 21860 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:02.928807 22591 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:02.930121 21928 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:02.931020 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6f466d510c1b46d481ab83686b150d61 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:02.936074 22703 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:02.938035 22703 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:02.965844 22765 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.958391 21860 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.989389 21754 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:03.009817 21931 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:03.013506 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2a91310e68464d64aa8324092182518f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:03.015797 21932 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:02.969383 22591 raft_consensus.cc:2955] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:03.019213 22770 ts_tablet_manager.cc:1929] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:03.027805 22770 log.cc:1199] T c96543776baa4cdda2e6862202de4e52 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/c96543776baa4cdda2e6862202de4e52
I20260709 13:05:03.021534 21860 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:03.028070 21757 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.029450 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet c96543776baa4cdda2e6862202de4e52 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:03.029692 22670 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:03.030236 22770 tablet_replica.cc:333] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:03.031700 22770 raft_consensus.cc:2243] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.032392 22770 raft_consensus.cc:2272] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.038084 22750 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:03.039921 22770 ts_tablet_manager.cc:1916] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:03.044371 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a91310e68464d64aa8324092182518f with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:03.049948 21860 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:03.058385 22770 ts_tablet_manager.cc:1929] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:03.058961 22770 log.cc:1199] T 6f466d510c1b46d481ab83686b150d61 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/6f466d510c1b46d481ab83686b150d61
I20260709 13:05:03.061694 22593 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:03.065187 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet 6f466d510c1b46d481ab83686b150d61 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:03.067118 22710 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:03.070483 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 951fdd0c64f542cbab511fb32cfa91cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:03.083633 22775 tablet_replica.cc:333] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:03.085677 22590 raft_consensus.cc:2955] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:03.087623 22775 raft_consensus.cc:2243] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.092211 22775 raft_consensus.cc:2272] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.094323 21754 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:03.097447 22775 ts_tablet_manager.cc:1916] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:03.114132 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 951fdd0c64f542cbab511fb32cfa91cb with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:03.120042 22775 ts_tablet_manager.cc:1929] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:03.120499 22775 log.cc:1199] T 2a91310e68464d64aa8324092182518f P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/2a91310e68464d64aa8324092182518f
I20260709 13:05:03.123064 22775 tablet_replica.cc:333] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:03.123064 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 2a91310e68464d64aa8324092182518f (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:03.125263 22775 raft_consensus.cc:2243] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.126224 22775 raft_consensus.cc:2272] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.130124 22775 ts_tablet_manager.cc:1916] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:03.147276 22775 ts_tablet_manager.cc:1929] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:03.147715 22775 log.cc:1199] T 951fdd0c64f542cbab511fb32cfa91cb P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/951fdd0c64f542cbab511fb32cfa91cb
I20260709 13:05:03.149240 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 951fdd0c64f542cbab511fb32cfa91cb (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:04.188038 21860 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.193872 22009 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:04.194257 21825 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec2c60ea913b4b8a9b1cbfd733880203. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:04.195084 21931 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:04.195775 22725 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.196935 22765 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.203965 22765 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.205946 22009 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.220053 22001 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.214458 21931 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.223450 21757 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:05:04.249058 21974 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97cf25498fcd4e4594d809b889d236d3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:04.255005 21932 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:04.258885 22704 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.265849 21860 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:04.267328 22704 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.272544 22730 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.278179 21932 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.282908 21859 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.302727 22003 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.315864 21757 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:04.321494 21932 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:04.321708 21974 consensus_peers.cc:597] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55c6a16f96334a6a8bd95d99ea2ffc7e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:04.323242 22791 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.325726 21860 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:04.328972 22791 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.344798 22790 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.346844 21932 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.346503 21860 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.361706 21754 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.362270 21931 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.384310 22003 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:04.384325 21900 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 958f1a2d78484a3bab2de506da912756. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:04.384904 21860 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:04.386159 22783 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.386971 22710 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.419070 22710 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.421926 21860 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.425828 22003 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:04.438753 21757 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:04.636144 22799 ts_tablet_manager.cc:933] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:04.638809 22799 tablet_copy_client.cc:323] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:04.640264 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet ec2c60ea913b4b8a9b1cbfd733880203 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:04.640712 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet ec2c60ea913b4b8a9b1cbfd733880203 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-ec2c60ea913b4b8a9b1cbfd733880203
I20260709 13:05:04.648584 21869 tablet_copy_source_session.cc:215] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:04.654346 22799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec2c60ea913b4b8a9b1cbfd733880203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.667565 22799 tablet_copy_client.cc:806] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:04.668123 22799 tablet_copy_client.cc:670] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:04.672825 22799 tablet_copy_client.cc:538] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:04.683172 22799 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:04.712275 22799 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.713115 22799 tablet_bootstrap.cc:492] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:04.713848 22799 ts_tablet_manager.cc:1403] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.005s
I20260709 13:05:04.716660 22799 raft_consensus.cc:359] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.717514 22799 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:04.718158 22799 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.720875 22799 ts_tablet_manager.cc:1434] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:04.722711 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-ec2c60ea913b4b8a9b1cbfd733880203 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:04.723130 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-ec2c60ea913b4b8a9b1cbfd733880203 on tablet ec2c60ea913b4b8a9b1cbfd733880203 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:04.770222 22799 ts_tablet_manager.cc:933] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:04.772388 22799 tablet_copy_client.cc:323] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:04.774775 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 97cf25498fcd4e4594d809b889d236d3 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:04.775446 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 97cf25498fcd4e4594d809b889d236d3 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-97cf25498fcd4e4594d809b889d236d3
I20260709 13:05:04.785120 22019 tablet_copy_source_session.cc:215] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:04.789206 22799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cf25498fcd4e4594d809b889d236d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.807864 22799 tablet_copy_client.cc:806] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:04.808561 22799 tablet_copy_client.cc:670] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:04.812705 22799 tablet_copy_client.cc:538] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:04.822257 22799 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:04.844698 22799 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.845695 22799 tablet_bootstrap.cc:492] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:04.846655 22799 ts_tablet_manager.cc:1403] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:05:04.849676 22799 raft_consensus.cc:359] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.850538 22799 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:04.851271 22799 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.855011 22799 ts_tablet_manager.cc:1434] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:04.856642 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-97cf25498fcd4e4594d809b889d236d3 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:04.857077 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-97cf25498fcd4e4594d809b889d236d3 on tablet 97cf25498fcd4e4594d809b889d236d3 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:04.907541 22799 ts_tablet_manager.cc:933] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:04.910806 22799 tablet_copy_client.cc:323] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:04.912577 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:04.913194 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 55c6a16f96334a6a8bd95d99ea2ffc7e from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-55c6a16f96334a6a8bd95d99ea2ffc7e
I20260709 13:05:04.918830 22019 tablet_copy_source_session.cc:215] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:04.922881 22799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.938686 22799 tablet_copy_client.cc:806] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:04.939226 22799 tablet_copy_client.cc:670] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:04.944644 22799 tablet_copy_client.cc:538] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:04.951083 22799 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:04.970011 22806 ts_tablet_manager.cc:933] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:04.977276 22799 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.978257 22806 tablet_copy_client.cc:323] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:04.978598 22799 tablet_bootstrap.cc:492] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:04.979306 22799 ts_tablet_manager.cc:1403] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.000s
I20260709 13:05:04.980327 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 958f1a2d78484a3bab2de506da912756 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:04.980893 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 958f1a2d78484a3bab2de506da912756 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-958f1a2d78484a3bab2de506da912756
I20260709 13:05:04.982443 22799 raft_consensus.cc:359] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.983244 22799 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:04.983826 22799 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:04.985893 22799 ts_tablet_manager.cc:1434] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:05:04.988677 21944 tablet_copy_source_session.cc:215] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:04.989176 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-55c6a16f96334a6a8bd95d99ea2ffc7e received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:04.989642 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-55c6a16f96334a6a8bd95d99ea2ffc7e on tablet 55c6a16f96334a6a8bd95d99ea2ffc7e with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:05.005249 22806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958f1a2d78484a3bab2de506da912756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:05.024452 22806 tablet_copy_client.cc:806] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:05.025188 22806 tablet_copy_client.cc:670] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:05.031427 22806 tablet_copy_client.cc:538] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:05.039319 22806 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:05.060703 22806 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.061640 22806 tablet_bootstrap.cc:492] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:05.062281 22806 ts_tablet_manager.cc:1403] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.007s
I20260709 13:05:05.065239 22806 raft_consensus.cc:359] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:05.066082 22806 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:05.066725 22806 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:05.069550 22806 ts_tablet_manager.cc:1434] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:05:05.072763 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-958f1a2d78484a3bab2de506da912756 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:05.073239 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-958f1a2d78484a3bab2de506da912756 on tablet 958f1a2d78484a3bab2de506da912756 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:05.222177 22590 raft_consensus.cc:1217] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:05.356595 22590 raft_consensus.cc:1217] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:05.471084 22590 raft_consensus.cc:1217] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:05.520076 22590 raft_consensus.cc:1217] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:05.778702 22725 raft_consensus.cc:1064] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:05.780593 22725 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:05.787330 22590 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.789417 22798 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:05.790916 21928 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.797410 22810 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:05.804200 22001 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.807454 22798 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:05.831544 22590 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.831534 22798 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.840628 22003 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.859050 21754 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.863893 21928 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.884645 22812 raft_consensus.cc:1064] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:05.892671 22812 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:05.911947 22590 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.912433 21860 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:05.914268 22812 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:05.917513 21859 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.918004 21928 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.934978 22791 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:05.936214 22591 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:05.936440 22823 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:05.938136 22798 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:05.941421 22001 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:05.944396 22725 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:05.944676 22811 raft_consensus.cc:1064] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:05.948310 22811 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:05.961833 22590 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.963405 22818 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:05.972391 21860 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.975359 22811 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:05.977806 22003 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:05.980093 22791 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.982118 21928 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:05.992846 21855 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.007195 22814 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.010608 22818 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:06.011636 22730 raft_consensus.cc:1064] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:06.013931 22730 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:06.021292 22591 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.028541 21756 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.041779 22593 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.066690 22001 raft_consensus.cc:2955] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.076309 21855 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:06.077024 22593 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:06.077934 21928 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:06.078421 22812 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:06.080132 22823 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:06.085246 22812 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:06.090523 22818 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.094487 22593 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.102367 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec2c60ea913b4b8a9b1cbfd733880203 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:06.103271 21754 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.105655 22003 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:06.110757 22001 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.131835 21860 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.136912 21757 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.152988 22791 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.154268 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet ec2c60ea913b4b8a9b1cbfd733880203 with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:06.155925 21928 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.162024 22593 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:06.166162 21859 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.170105 21860 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:06.171926 22812 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:06.173813 22730 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:06.175320 22837 tablet_replica.cc:333] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:06.176803 22837 raft_consensus.cc:2243] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:06.177573 22837 raft_consensus.cc:2272] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:06.178812 22590 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.181177 22837 ts_tablet_manager.cc:1916] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:06.196538 22837 ts_tablet_manager.cc:1929] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:06.196911 22837 log.cc:1199] T ec2c60ea913b4b8a9b1cbfd733880203 P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/ec2c60ea913b4b8a9b1cbfd733880203
I20260709 13:05:06.201392 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet ec2c60ea913b4b8a9b1cbfd733880203 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:06.210832 22823 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.217200 21860 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.224725 21757 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.228981 22593 raft_consensus.cc:2955] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.248764 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 97cf25498fcd4e4594d809b889d236d3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:06.256198 22005 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:06.260135 21928 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:06.262642 21932 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:06.264065 22730 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:06.267252 22590 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:06.267527 22593 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:06.268177 21860 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:06.268812 22749 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:06.270521 22730 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:06.270576 22821 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:06.276957 22710 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.278411 22591 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.283896 22791 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.286268 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 958f1a2d78484a3bab2de506da912756 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:06.291082 22590 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.294145 21928 raft_consensus.cc:2955] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.306517 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:06.317500 21860 raft_consensus.cc:2955] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.319621 21757 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.318012 21754 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.337546 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 958f1a2d78484a3bab2de506da912756 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:06.351706 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet 97cf25498fcd4e4594d809b889d236d3 with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:06.352605 22837 tablet_replica.cc:333] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:06.353659 22837 raft_consensus.cc:2243] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:06.354409 22837 raft_consensus.cc:2272] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:06.358124 22837 ts_tablet_manager.cc:1916] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:06.362097 21754 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:06.371789 22839 tablet_replica.cc:333] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:06.378958 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet 55c6a16f96334a6a8bd95d99ea2ffc7e with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:06.381038 22840 tablet_replica.cc:333] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:06.382160 22840 raft_consensus.cc:2243] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:06.382889 22840 raft_consensus.cc:2272] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:06.386204 22840 ts_tablet_manager.cc:1916] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:06.389806 22837 ts_tablet_manager.cc:1929] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:06.390314 22837 log.cc:1199] T 97cf25498fcd4e4594d809b889d236d3 P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/97cf25498fcd4e4594d809b889d236d3
I20260709 13:05:06.391932 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 97cf25498fcd4e4594d809b889d236d3 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:06.394135 22839 raft_consensus.cc:2243] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:06.394747 22839 raft_consensus.cc:2272] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:06.398301 22839 ts_tablet_manager.cc:1916] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:06.402923 22840 ts_tablet_manager.cc:1929] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:06.403474 22840 log.cc:1199] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/55c6a16f96334a6a8bd95d99ea2ffc7e
I20260709 13:05:06.409310 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet 55c6a16f96334a6a8bd95d99ea2ffc7e (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:06.418088 22839 ts_tablet_manager.cc:1929] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:06.418730 22839 log.cc:1199] T 958f1a2d78484a3bab2de506da912756 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/958f1a2d78484a3bab2de506da912756
I20260709 13:05:06.420637 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 958f1a2d78484a3bab2de506da912756 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:07.453258 21860 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:07.470273 21825 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ecc321cb1e14e99a3838b646f82e71a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:07.470394 22006 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.472433 21933 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.474210 22810 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:07.475348 22725 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.483443 22842 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.485375 22006 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.485317 21928 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.503383 21860 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:07.504151 21756 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:05:07.513276 21825 consensus_peers.cc:597] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71a9145a148c4edebd5d8df1c9956298. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:07.513422 22006 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.513935 21928 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.515383 22842 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.517980 22725 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.534585 22842 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.536150 21928 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.545687 22006 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.548774 21932 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:07.558982 21900 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f24c37d4e05d4b7eb71e234ac493bff5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:07.561206 21755 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:07.562637 21860 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.567827 22821 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.584251 22006 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.584563 22749 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.585749 22821 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.586335 21860 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.602772 22006 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.605744 21757 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.606125 21860 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:07.633569 21825 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be9ae799c8f043d7b4fc384aa1edf5ce. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:07.634459 22005 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.636289 22843 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:07.635022 21932 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:07.638653 22842 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.646394 22810 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.648181 22005 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.654486 21932 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.667865 21755 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:07.941898 22862 ts_tablet_manager.cc:933] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:07.943954 22862 tablet_copy_client.cc:323] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:07.945909 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 0ecc321cb1e14e99a3838b646f82e71a from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:07.946455 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 0ecc321cb1e14e99a3838b646f82e71a from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-0ecc321cb1e14e99a3838b646f82e71a
I20260709 13:05:07.952903 21869 tablet_copy_source_session.cc:215] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:07.961228 22862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecc321cb1e14e99a3838b646f82e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:07.976272 22862 tablet_copy_client.cc:806] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:07.976910 22862 tablet_copy_client.cc:670] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:07.982832 22862 tablet_copy_client.cc:538] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:07.989631 22862 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:07.997359 22864 ts_tablet_manager.cc:933] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:08.021416 22864 tablet_copy_client.cc:323] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:08.023097 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 71a9145a148c4edebd5d8df1c9956298 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:08.023661 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 71a9145a148c4edebd5d8df1c9956298 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-71a9145a148c4edebd5d8df1c9956298
I20260709 13:05:08.028767 22862 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:08.030072 22862 tablet_bootstrap.cc:492] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:08.030791 22862 ts_tablet_manager.cc:1403] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.017s
I20260709 13:05:08.032613 21869 tablet_copy_source_session.cc:215] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:08.035789 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71a9145a148c4edebd5d8df1c9956298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:08.033776 22862 raft_consensus.cc:359] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.039335 22862 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:08.039883 22862 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.049086 22862 ts_tablet_manager.cc:1434] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.004s
I20260709 13:05:08.056524 22864 tablet_copy_client.cc:806] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:08.057489 22864 tablet_copy_client.cc:670] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:08.057667 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-0ecc321cb1e14e99a3838b646f82e71a received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:08.058145 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-0ecc321cb1e14e99a3838b646f82e71a on tablet 0ecc321cb1e14e99a3838b646f82e71a with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:08.067334 22864 tablet_copy_client.cc:538] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:08.079728 22864 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:08.101116 22864 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:08.102085 22864 tablet_bootstrap.cc:492] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:08.102837 22864 ts_tablet_manager.cc:1403] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.004s
I20260709 13:05:08.106096 22864 raft_consensus.cc:359] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.106971 22864 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:08.108537 22864 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.112874 22864 ts_tablet_manager.cc:1434] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:08.114466 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-71a9145a148c4edebd5d8df1c9956298 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:08.114933 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-71a9145a148c4edebd5d8df1c9956298 on tablet 71a9145a148c4edebd5d8df1c9956298 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:08.154904 22864 ts_tablet_manager.cc:933] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:08.158195 22864 tablet_copy_client.cc:323] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:08.162328 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet f24c37d4e05d4b7eb71e234ac493bff5 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:08.162885 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet f24c37d4e05d4b7eb71e234ac493bff5 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-f24c37d4e05d4b7eb71e234ac493bff5
I20260709 13:05:08.173306 21944 tablet_copy_source_session.cc:215] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:08.176420 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c37d4e05d4b7eb71e234ac493bff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:08.195436 22864 tablet_copy_client.cc:806] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:08.196094 22864 tablet_copy_client.cc:670] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:08.200353 22864 tablet_copy_client.cc:538] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:08.209260 22864 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:08.218513 22862 ts_tablet_manager.cc:933] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:08.225507 22862 tablet_copy_client.cc:323] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:08.227183 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet be9ae799c8f043d7b4fc384aa1edf5ce from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:08.227734 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet be9ae799c8f043d7b4fc384aa1edf5ce from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-be9ae799c8f043d7b4fc384aa1edf5ce
I20260709 13:05:08.235865 21869 tablet_copy_source_session.cc:215] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:08.239349 22862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9ae799c8f043d7b4fc384aa1edf5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:08.245368 22864 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:08.246410 22864 tablet_bootstrap.cc:492] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:08.247189 22864 ts_tablet_manager.cc:1403] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.004s
I20260709 13:05:08.250280 22864 raft_consensus.cc:359] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.251219 22864 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:08.251925 22864 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.254387 22864 ts_tablet_manager.cc:1434] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:08.266479 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-f24c37d4e05d4b7eb71e234ac493bff5 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:08.266945 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-f24c37d4e05d4b7eb71e234ac493bff5 on tablet f24c37d4e05d4b7eb71e234ac493bff5 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:08.279636 22862 tablet_copy_client.cc:806] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:08.280354 22862 tablet_copy_client.cc:670] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:08.284693 22862 tablet_copy_client.cc:538] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:08.292869 22862 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:08.313453 22862 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:08.314168 22862 tablet_bootstrap.cc:492] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:08.314738 22862 ts_tablet_manager.cc:1403] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:05:08.316810 22862 raft_consensus.cc:359] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.317489 22862 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:08.317997 22862 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:08.320073 22862 ts_tablet_manager.cc:1434] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:08.322326 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-be9ae799c8f043d7b4fc384aa1edf5ce received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:08.322733 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-be9ae799c8f043d7b4fc384aa1edf5ce on tablet be9ae799c8f043d7b4fc384aa1edf5ce with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:08.343459 22591 raft_consensus.cc:1217] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:08.563297 22591 raft_consensus.cc:1217] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:08.689457 22591 raft_consensus.cc:1217] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:08.741806 22591 raft_consensus.cc:1217] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:08.772011 22810 raft_consensus.cc:1064] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:08.774091 22810 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:08.780635 22591 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:08.782021 22810 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:08.783285 22005 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:08.784121 21932 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:08.787051 22873 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:08.794369 22873 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:08.823408 22843 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.825264 21928 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.834124 22591 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.846553 22005 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.853025 21756 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.899578 21859 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:08.905206 22590 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:08.907513 22873 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:08.907827 22003 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:08.909160 22843 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:08.918453 22878 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.929086 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0ecc321cb1e14e99a3838b646f82e71a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:08.929844 22005 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.926600 22590 raft_consensus.cc:2955] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.968257 21755 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:08.985670 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ecc321cb1e14e99a3838b646f82e71a with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:08.987691 22883 tablet_replica.cc:333] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:08.988648 22883 raft_consensus.cc:2243] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:08.989419 22883 raft_consensus.cc:2272] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:08.992089 22883 ts_tablet_manager.cc:1916] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:09.005420 22883 ts_tablet_manager.cc:1929] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:09.006081 22883 log.cc:1199] T 0ecc321cb1e14e99a3838b646f82e71a P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/0ecc321cb1e14e99a3838b646f82e71a
I20260709 13:05:09.008383 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 0ecc321cb1e14e99a3838b646f82e71a (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:09.108712 22810 raft_consensus.cc:1064] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:09.111022 22810 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:09.133242 22590 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.135352 22873 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:09.173813 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46813.6us
I20260709 13:05:09.202476 22003 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.237732 22888 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.253539 22821 raft_consensus.cc:1064] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:09.266237 22821 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:09.309441 22590 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.313024 22895 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.316869 21931 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.329042 22888 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.341895 22003 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.347254 22810 raft_consensus.cc:1064] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:09.350005 22810 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:09.365250 22003 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.371767 22893 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.375362 21860 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.379204 21931 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.392776 22590 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.397785 22005 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:09.403369 22893 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.413859 22590 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.417452 22895 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:09.423064 21755 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.432765 22889 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.437741 22892 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.458580 22810 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:09.486143 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50169.2us
I20260709 13:05:09.525400 22873 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.531019 22821 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.531278 21928 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.532311 21859 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.545329 22593 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.554142 22590 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.563652 21757 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.570966 21755 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.581537 22006 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.622771 21859 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:09.626784 21930 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.631258 22590 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:09.631337 22006 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:09.633481 22873 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:09.639055 22005 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.635445 22873 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:05:09.646544 22889 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.654268 22593 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.663894 22003 raft_consensus.cc:2955] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.668977 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be9ae799c8f043d7b4fc384aa1edf5ce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:09.669782 21755 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.685714 21931 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:09.687561 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet be9ae799c8f043d7b4fc384aa1edf5ce with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:09.690683 22005 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:09.691427 22908 tablet_replica.cc:333] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:09.692253 22895 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:09.692308 22593 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:09.694651 22892 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:09.697470 22908 raft_consensus.cc:2243] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:09.698100 22908 raft_consensus.cc:2272] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:09.702729 22908 ts_tablet_manager.cc:1916] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:09.704103 22892 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.711481 22590 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.714715 22003 raft_consensus.cc:2955] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:09.733651 21755 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:09.737888 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f24c37d4e05d4b7eb71e234ac493bff5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:09.746809 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet f24c37d4e05d4b7eb71e234ac493bff5 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:09.752230 22909 tablet_replica.cc:333] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:09.755213 22908 ts_tablet_manager.cc:1929] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:09.756115 22908 log.cc:1199] T be9ae799c8f043d7b4fc384aa1edf5ce P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/be9ae799c8f043d7b4fc384aa1edf5ce
I20260709 13:05:09.758605 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet be9ae799c8f043d7b4fc384aa1edf5ce (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:09.764297 22909 raft_consensus.cc:2243] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:09.765219 22909 raft_consensus.cc:2272] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:09.769232 22909 ts_tablet_manager.cc:1916] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:09.782965 22909 ts_tablet_manager.cc:1929] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:09.783454 22909 log.cc:1199] T f24c37d4e05d4b7eb71e234ac493bff5 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/f24c37d4e05d4b7eb71e234ac493bff5
I20260709 13:05:09.785964 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet f24c37d4e05d4b7eb71e234ac493bff5 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:10.104081 21858 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:10.111629 22593 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.114974 22889 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:10.121308 21928 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.123059 22843 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:10.129089 22893 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:10.129179 22593 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:10.131364 21928 raft_consensus.cc:2955] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:10.148010 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 71a9145a148c4edebd5d8df1c9956298 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:10.150216 21757 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:10.166360 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 71a9145a148c4edebd5d8df1c9956298 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:10.168237 22910 tablet_replica.cc:333] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:10.169401 22910 raft_consensus.cc:2243] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.170161 22910 raft_consensus.cc:2272] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.173614 22910 ts_tablet_manager.cc:1916] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:10.188283 22910 ts_tablet_manager.cc:1929] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:10.188635 22910 log.cc:1199] T 71a9145a148c4edebd5d8df1c9956298 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/71a9145a148c4edebd5d8df1c9956298
I20260709 13:05:10.190435 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 71a9145a148c4edebd5d8df1c9956298 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:11.232506 21858 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.241506 21931 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:11.241638 21825 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: caafc8397a4f4963abeff030a28f5c47. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.241892 22003 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.242918 22810 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.243777 22843 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:11.253713 22843 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.255600 21930 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.259349 22003 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.271057 21928 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.272389 21755 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:11.288904 22003 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:11.289968 21900 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f6eae3eb9a4941fc88763a5da4ee7638. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.290625 22821 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.290783 21858 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.293056 22821 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.301930 22892 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.315784 22005 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.352342 21858 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.353624 22001 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.361675 21757 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:11.363457 21974 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1cf1fd218a894ed39170b8db4ecc6ee8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.366683 21928 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.368135 22925 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.372020 21859 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.376950 22925 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.393042 22894 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.400892 21930 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.402805 21860 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.409735 21931 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.421921 22001 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:11.422570 21900 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c70dbf50c904384a2ab23b1d7a9dbfd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.424226 22821 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.425422 21860 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.427304 21756 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.429268 22821 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.442312 22892 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.443769 21860 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.456916 22001 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.477233 21756 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:11.730067 22935 ts_tablet_manager.cc:933] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:11.732685 22935 tablet_copy_client.cc:323] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:11.734810 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet f6eae3eb9a4941fc88763a5da4ee7638 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:11.735373 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet f6eae3eb9a4941fc88763a5da4ee7638 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-f6eae3eb9a4941fc88763a5da4ee7638
I20260709 13:05:11.745769 21944 tablet_copy_source_session.cc:215] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:11.749783 22935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6eae3eb9a4941fc88763a5da4ee7638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.766889 22935 tablet_copy_client.cc:806] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:11.766886 22937 ts_tablet_manager.cc:933] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:11.767508 22935 tablet_copy_client.cc:670] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:11.770100 22937 tablet_copy_client.cc:323] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:11.772665 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet caafc8397a4f4963abeff030a28f5c47 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:11.773269 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet caafc8397a4f4963abeff030a28f5c47 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-caafc8397a4f4963abeff030a28f5c47
I20260709 13:05:11.774686 22935 tablet_copy_client.cc:538] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:11.783886 22935 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:11.787628 21869 tablet_copy_source_session.cc:215] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:11.804397 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caafc8397a4f4963abeff030a28f5c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.829660 22937 tablet_copy_client.cc:806] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:11.830361 22937 tablet_copy_client.cc:670] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:11.836282 22935 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:11.837395 22935 tablet_bootstrap.cc:492] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:11.837548 22937 tablet_copy_client.cc:538] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:11.838192 22935 ts_tablet_manager.cc:1403] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.055s	user 0.024s	sys 0.013s
I20260709 13:05:11.841838 22935 raft_consensus.cc:359] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.842600 22935 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:11.843298 22935 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.858721 22937 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:11.862783 22935 ts_tablet_manager.cc:1434] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.012s
I20260709 13:05:11.864946 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-f6eae3eb9a4941fc88763a5da4ee7638 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:11.866132 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-f6eae3eb9a4941fc88763a5da4ee7638 on tablet f6eae3eb9a4941fc88763a5da4ee7638 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:11.889811 22937 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:11.890723 22937 tablet_bootstrap.cc:492] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:11.891414 22937 ts_tablet_manager.cc:1403] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.017s
I20260709 13:05:11.894439 22937 raft_consensus.cc:359] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.895344 22937 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:11.895990 22937 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:11.898175 22935 ts_tablet_manager.cc:933] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:11.898452 22937 ts_tablet_manager.cc:1434] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:11.900883 22935 tablet_copy_client.cc:323] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:11.902664 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-caafc8397a4f4963abeff030a28f5c47 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:11.903122 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-caafc8397a4f4963abeff030a28f5c47 on tablet caafc8397a4f4963abeff030a28f5c47 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:11.905934 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 1cf1fd218a894ed39170b8db4ecc6ee8 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:11.906526 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 1cf1fd218a894ed39170b8db4ecc6ee8 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-1cf1fd218a894ed39170b8db4ecc6ee8
I20260709 13:05:11.923203 22019 tablet_copy_source_session.cc:215] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:11.929481 22935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cf1fd218a894ed39170b8db4ecc6ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.932502 22937 ts_tablet_manager.cc:933] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:11.943145 22937 tablet_copy_client.cc:323] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:11.944746 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:11.945277 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 0c70dbf50c904384a2ab23b1d7a9dbfd from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-0c70dbf50c904384a2ab23b1d7a9dbfd
I20260709 13:05:11.952306 21944 tablet_copy_source_session.cc:215] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:11.954466 22935 tablet_copy_client.cc:806] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:11.955118 22935 tablet_copy_client.cc:670] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:11.957162 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.960278 22935 tablet_copy_client.cc:538] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:11.969806 22935 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:11.976836 22937 tablet_copy_client.cc:806] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:11.977721 22937 tablet_copy_client.cc:670] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:11.984030 22937 tablet_copy_client.cc:538] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:11.996235 22935 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:11.997412 22935 tablet_bootstrap.cc:492] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:11.998145 22935 ts_tablet_manager.cc:1403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.007s
I20260709 13:05:12.001670 22935 raft_consensus.cc:359] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:12.002947 22935 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:12.003670 22935 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:12.006574 22935 ts_tablet_manager.cc:1434] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:12.007993 22937 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:12.008508 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-1cf1fd218a894ed39170b8db4ecc6ee8 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:12.008986 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-1cf1fd218a894ed39170b8db4ecc6ee8 on tablet 1cf1fd218a894ed39170b8db4ecc6ee8 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:12.040062 22937 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:12.040964 22937 tablet_bootstrap.cc:492] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:12.041688 22937 ts_tablet_manager.cc:1403] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.002s
I20260709 13:05:12.044550 22937 raft_consensus.cc:359] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:12.045409 22937 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:12.046025 22937 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:12.048683 22937 ts_tablet_manager.cc:1434] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:12.050362 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-0c70dbf50c904384a2ab23b1d7a9dbfd received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:12.050777 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-0c70dbf50c904384a2ab23b1d7a9dbfd on tablet 0c70dbf50c904384a2ab23b1d7a9dbfd with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:12.344658 22591 raft_consensus.cc:1217] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:12.362172 22591 raft_consensus.cc:1217] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:12.385293 22591 raft_consensus.cc:1217] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:12.401113 22591 raft_consensus.cc:1217] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:12.764439 22810 raft_consensus.cc:1064] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:12.766611 22810 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:12.781081 22946 raft_consensus.cc:1064] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:12.783533 22946 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:12.784162 22591 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.797170 22810 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.800547 22001 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.800823 22006 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.802855 22928 raft_consensus.cc:1064] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:12.802973 22591 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.804190 22947 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.805840 21930 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.806861 22821 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.807642 22928 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:12.808395 22810 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.808516 22821 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.817310 21859 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.820152 22946 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.823266 22590 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.824978 22928 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.831037 21859 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.832612 22925 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:12.869568 21930 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:12.883157 22925 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.882589 22843 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:12.888170 22001 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:12.901722 22821 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:12.958344 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53551.3us
I20260709 13:05:12.967875 22005 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:12.979676 21756 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:12.995386 21757 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:13.012086 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52156.5us
I20260709 13:05:13.014539 22925 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.019384 21860 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.006052 22593 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.027925 22590 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.035636 21928 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.089224 22946 raft_consensus.cc:1064] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:13.090785 22591 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.091738 22946 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:13.094911 21860 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.093333 21928 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.120625 22005 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.133468 22588 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.135020 22966 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:05:13.151830 21860 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.155941 22966 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:13.178249 22960 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:13.187531 21754 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.225287 21928 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:13.232343 21859 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.234006 22966 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:13.242774 22821 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.243662 22009 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:13.249830 22591 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.250878 21933 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.251286 22791 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:13.252920 22928 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:13.253460 22009 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.232343 22590 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.257203 21859 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.264724 22925 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.264433 22946 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:13.270659 21858 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:13.284389 22590 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.284626 21757 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.287880 21933 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.286965 22926 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.288609 22591 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.289083 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1cf1fd218a894ed39170b8db4ecc6ee8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:13.289394 22965 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:13.292584 22593 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.293406 22843 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:13.301478 22965 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.304049 22588 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.303265 21928 raft_consensus.cc:2955] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.314992 21933 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.309702 21860 raft_consensus.cc:2955] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.327999 21755 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.351361 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet 1cf1fd218a894ed39170b8db4ecc6ee8 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:13.351545 22590 raft_consensus.cc:2955] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.354969 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet caafc8397a4f4963abeff030a28f5c47 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:13.365314 21755 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.380069 21933 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:13.381672 22979 tablet_replica.cc:333] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:13.382977 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:13.385723 22979 raft_consensus.cc:2243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:13.387459 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c70dbf50c904384a2ab23b1d7a9dbfd with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:13.387888 22588 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.389840 22009 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.390203 22926 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:13.392829 21756 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.393708 22966 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:13.394986 22979 raft_consensus.cc:2272] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:13.400480 22979 ts_tablet_manager.cc:1916] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:13.408789 22966 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.423348 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f6eae3eb9a4941fc88763a5da4ee7638 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:13.424100 22980 tablet_replica.cc:333] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:13.424216 22588 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.424892 22009 raft_consensus.cc:2955] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:13.426175 22980 raft_consensus.cc:2243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:13.427064 22980 raft_consensus.cc:2272] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:13.433195 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet caafc8397a4f4963abeff030a28f5c47 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:13.437145 22980 ts_tablet_manager.cc:1916] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:13.456766 22979 ts_tablet_manager.cc:1929] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:13.457269 22979 log.cc:1199] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/1cf1fd218a894ed39170b8db4ecc6ee8
I20260709 13:05:13.459022 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet 1cf1fd218a894ed39170b8db4ecc6ee8 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:13.452373 21755 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.467435 22980 ts_tablet_manager.cc:1929] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:13.467935 22980 log.cc:1199] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/0c70dbf50c904384a2ab23b1d7a9dbfd
I20260709 13:05:13.469655 22980 tablet_replica.cc:333] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:13.469837 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 0c70dbf50c904384a2ab23b1d7a9dbfd (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:13.470695 22980 raft_consensus.cc:2243] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:13.471441 22980 raft_consensus.cc:2272] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:13.475126 22980 ts_tablet_manager.cc:1916] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:13.478641 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet f6eae3eb9a4941fc88763a5da4ee7638 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:13.479622 22979 tablet_replica.cc:333] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:13.480573 22979 raft_consensus.cc:2243] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:13.481252 22979 raft_consensus.cc:2272] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:13.484519 22979 ts_tablet_manager.cc:1916] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:13.488644 22980 ts_tablet_manager.cc:1929] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:13.491281 22980 log.cc:1199] T caafc8397a4f4963abeff030a28f5c47 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/caafc8397a4f4963abeff030a28f5c47
I20260709 13:05:13.493088 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet caafc8397a4f4963abeff030a28f5c47 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:13.499203 22979 ts_tablet_manager.cc:1929] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:13.499656 22979 log.cc:1199] T f6eae3eb9a4941fc88763a5da4ee7638 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/f6eae3eb9a4941fc88763a5da4ee7638
I20260709 13:05:13.501395 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet f6eae3eb9a4941fc88763a5da4ee7638 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:14.542172 22009 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:14.549578 21928 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:14.549886 21974 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b5d11192c5d4d72b0bb9f3ebe9b715a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:14.551270 22984 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:14.551748 21860 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:14.553128 22925 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:14.559975 22964 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.561731 21933 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.575711 21860 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.579463 22009 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:14.587466 21974 consensus_peers.cc:597] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2bcf30627084357a4745445324a2430. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:14.594074 21933 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:14.595376 21755 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.595853 22964 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:14.596269 21860 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:14.603786 22964 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:14.607779 22925 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.611720 21928 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.619891 21858 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.640005 21928 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:14.694247 21756 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.703589 22009 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:14.705152 22994 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:14.705797 21858 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:14.707031 22966 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:14.705155 21900 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a76564a0ee84b74a3af01c79300d4b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:14.718384 22994 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.720671 22009 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.725643 21858 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.733916 22005 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:14.740983 21757 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.799764 21928 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:14.801447 22998 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.058s	user 0.004s	sys 0.004s
W20260709 13:05:14.803680 21974 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4021839a030d44ab886833307930ad33. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:14.806383 22984 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:14.811378 21860 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:14.827986 22984 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:14.845079 22964 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.847959 21860 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.848178 21928 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:14.869158 21757 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.074635 23004 ts_tablet_manager.cc:933] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:15.077075 23004 tablet_copy_client.cc:323] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:15.079089 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:15.079619 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-8b5d11192c5d4d72b0bb9f3ebe9b715a
I20260709 13:05:15.087483 22019 tablet_copy_source_session.cc:215] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:15.093242 23004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:15.097251 23006 ts_tablet_manager.cc:933] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:15.099177 23006 tablet_copy_client.cc:323] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:15.101329 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 8a76564a0ee84b74a3af01c79300d4b7 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:15.101861 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 8a76564a0ee84b74a3af01c79300d4b7 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-8a76564a0ee84b74a3af01c79300d4b7
I20260709 13:05:15.107992 21944 tablet_copy_source_session.cc:215] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:15.117902 23006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a76564a0ee84b74a3af01c79300d4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:15.128857 23004 tablet_copy_client.cc:806] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:15.129581 23004 tablet_copy_client.cc:670] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:15.133581 23004 tablet_copy_client.cc:538] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:15.143226 23004 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:15.157404 23006 tablet_copy_client.cc:806] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:15.158064 23006 tablet_copy_client.cc:670] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:15.161573 23006 tablet_copy_client.cc:538] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:15.167482 23004 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:15.169863 23006 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:15.168478 23004 tablet_bootstrap.cc:492] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:15.171953 23004 ts_tablet_manager.cc:1403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.005s
I20260709 13:05:15.183933 23004 raft_consensus.cc:359] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.184859 23004 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:15.185590 23004 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.200765 23004 ts_tablet_manager.cc:1434] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.004s
I20260709 13:05:15.205231 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-8b5d11192c5d4d72b0bb9f3ebe9b715a received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:15.205672 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-8b5d11192c5d4d72b0bb9f3ebe9b715a on tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:15.208213 23006 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:15.213760 23006 tablet_bootstrap.cc:492] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:15.215592 23006 ts_tablet_manager.cc:1403] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.012s
I20260709 13:05:15.219764 23006 raft_consensus.cc:359] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.220628 23006 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:15.221319 23006 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.223654 23006 ts_tablet_manager.cc:1434] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:15.225675 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-8a76564a0ee84b74a3af01c79300d4b7 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:15.226063 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-8a76564a0ee84b74a3af01c79300d4b7 on tablet 8a76564a0ee84b74a3af01c79300d4b7 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:15.238500 23006 ts_tablet_manager.cc:933] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:15.241981 23006 tablet_copy_client.cc:323] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:15.246968 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet d2bcf30627084357a4745445324a2430 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:15.247499 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet d2bcf30627084357a4745445324a2430 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-d2bcf30627084357a4745445324a2430
I20260709 13:05:15.253756 22019 tablet_copy_source_session.cc:215] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:15.256314 23006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bcf30627084357a4745445324a2430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:15.269642 23006 tablet_copy_client.cc:806] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:15.270342 23006 tablet_copy_client.cc:670] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:15.273785 23006 tablet_copy_client.cc:538] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:15.279997 23006 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:15.298507 23006 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:15.299408 23006 tablet_bootstrap.cc:492] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:15.300005 23006 ts_tablet_manager.cc:1403] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:05:15.302572 23006 raft_consensus.cc:359] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.303256 23006 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:15.303869 23006 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.306155 23006 ts_tablet_manager.cc:1434] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:05:15.307999 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-d2bcf30627084357a4745445324a2430 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:15.308368 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-d2bcf30627084357a4745445324a2430 on tablet d2bcf30627084357a4745445324a2430 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:15.359877 23006 ts_tablet_manager.cc:933] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:15.362128 23006 tablet_copy_client.cc:323] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:15.363765 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 4021839a030d44ab886833307930ad33 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:15.364362 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 4021839a030d44ab886833307930ad33 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-4021839a030d44ab886833307930ad33
I20260709 13:05:15.372588 22019 tablet_copy_source_session.cc:215] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:15.376150 23006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4021839a030d44ab886833307930ad33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:15.392901 23006 tablet_copy_client.cc:806] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:15.393550 23006 tablet_copy_client.cc:670] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:15.397156 23006 tablet_copy_client.cc:538] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:15.405114 23006 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:15.434284 23006 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:15.435249 23006 tablet_bootstrap.cc:492] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:15.435983 23006 ts_tablet_manager.cc:1403] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.031s	user 0.025s	sys 0.004s
I20260709 13:05:15.438941 23006 raft_consensus.cc:359] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.439875 23006 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:15.440455 23006 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:15.442775 23006 ts_tablet_manager.cc:1434] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:15.446214 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-4021839a030d44ab886833307930ad33 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:15.446668 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-4021839a030d44ab886833307930ad33 on tablet 4021839a030d44ab886833307930ad33 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:15.586249 22590 raft_consensus.cc:1217] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:15.776808 22590 raft_consensus.cc:1217] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:15.919692 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:16.251251 22589 raft_consensus.cc:1217] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:16.304437 23011 raft_consensus.cc:1064] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:16.314967 23011 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.417583 22588 raft_consensus.cc:1217] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:16.433311 23023 raft_consensus.cc:1064] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:16.511165 23023 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:16.453595 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56203.5us
W20260709 13:05:16.515630 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.530915 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 406979us
W20260709 13:05:16.550390 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.566901 22006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:16.567548 22006 raft_consensus.cc:606] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:16.570700 22001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:16.571247 22001 raft_consensus.cc:606] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:16.593207 22588 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.597733 23023 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:16.599174 23024 raft_consensus.cc:993] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:16.602953 23024 raft_consensus.cc:1081] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:16.610466 22592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57084
I20260709 13:05:16.611155 22592 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:16.611547 22592 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.620054 22588 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.621789 23011 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.634586 21931 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.634650 21857 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.639183 21857 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.641976 21931 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.645813 23027 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.636483 22592 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.652299 23027 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.657864 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:16.658576 21931 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.666473 21931 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:16.667671 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:16.668555 23010 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:16.669656 23023 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.673413 23023 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.678310 21933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:16.679021 21933 raft_consensus.cc:606] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:16.682271 21933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:16.682821 21933 raft_consensus.cc:606] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:16.688406 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:16.689182 21932 raft_consensus.cc:606] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:16.690461 23010 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:16.691489 23010 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.708129 21933 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:16.711593 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:16.712215 22001 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:16.712520 22001 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:16.713629 22001 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.714653 22001 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.723299 22001 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:16.725366 21757 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:16.726367 23027 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.732295 21857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:16.732975 21857 raft_consensus.cc:606] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:16.734308 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:16.735426 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:16.737092 22592 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:16.744429 21933 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.764343 23011 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.773386 21857 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.773975 21860 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.780738 22592 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.798525 21756 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:16.799088 21755 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.810130 22588 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.830371 23022 raft_consensus.cc:993] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:16.830910 23022 raft_consensus.cc:1081] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:16.832939 22587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57084
I20260709 13:05:16.833554 22587 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:16.833901 22587 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.813017 21931 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:16.838754 23024 raft_consensus.cc:1064] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:16.841398 23030 raft_consensus.cc:993] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:16.841930 23030 raft_consensus.cc:1081] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:16.842939 23024 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.852792 22008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:16.853402 22008 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:16.853758 22008 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.862762 22008 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:16.873085 23016 raft_consensus.cc:1064] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:16.873719 22008 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:16.873669 23016 raft_consensus.cc:1068] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Service unavailable: leader transfer in progress
I20260709 13:05:16.849166 22587 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.874893 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:16.890460 21855 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.891824 22587 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:16.893306 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:16.875550 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:16.893927 21928 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.894313 21931 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:16.894660 21931 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:16.895419 21931 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:16.896399 21931 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.902580 21855 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:16.885207 21933 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.880107 22588 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.904043 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:16.916108 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:16.915904 23011 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:16.916617 22001 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:16.917038 22001 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:16.917120 23024 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:16.917662 23024 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:16.917961 22001 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.918715 22925 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.918542 23024 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:16.919426 22001 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.883976 21860 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:16.925089 21931 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:16.926195 22925 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:16.949731 21928 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:16.958151 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:16.959266 23010 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:16.959815 23010 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:16.960744 23010 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:16.991727 22001 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:17.020267 22925 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.023032 21859 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.036341 21933 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.039155 23016 raft_consensus.cc:993] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:17.039605 23016 raft_consensus.cc:1081] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:17.038794 22588 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.041211 22001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:17.041782 22001 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.042168 22001 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.108140 23032 raft_consensus.cc:993] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:17.108695 23032 raft_consensus.cc:1081] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:17.110677 22592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57102
I20260709 13:05:17.111199 22592 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.111572 22592 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.119047 22592 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.120896 22001 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:17.122500 23018 raft_consensus.cc:993] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:17.123025 23018 raft_consensus.cc:1081] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:17.123363 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:17.123932 21859 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.124068 22001 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:17.125650 22588 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57098
I20260709 13:05:17.126230 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:17.126617 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:17.126744 21860 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.127652 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:17.128186 21931 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.128486 21931 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:17.130064 21931 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:17.130651 21859 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:17.122824 22592 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:17.131362 21931 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.132541 21933 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.132848 21933 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:17.133359 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:17.134179 23010 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.133957 21933 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.134701 21860 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:17.134687 23010 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:17.135710 21972 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:17.135807 23010 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.136409 22925 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.136832 22925 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:17.137681 22925 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:17.139739 21933 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.147336 21933 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:17.148041 21931 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:17.126225 22588 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.151907 22588 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.162137 21754 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.175570 22588 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.157460 22001 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.177821 22588 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:17.178387 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:17.178617 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:17.178992 21859 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.179225 22008 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.179363 21859 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:17.180111 21859 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.181386 21859 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.180860 21757 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.182871 21757 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.187579 22008 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:17.189599 21859 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:17.190035 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:17.190902 23010 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.191381 23010 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:17.192219 23010 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.198251 23040 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:17.203872 22001 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.208633 22006 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.211401 22004 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:17.217499 22588 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.218079 22592 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.219669 22925 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.221354 22925 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.223395 21859 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.223642 21754 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.230995 22592 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.230908 23021 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.236138 21859 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.238243 23011 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:17.239382 21933 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.243311 23011 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.248620 23027 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:17.249051 22588 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.252897 21859 raft_consensus.cc:2955] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.246161 21757 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.255031 23027 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:17.257376 23023 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.273239 22592 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.278836 22925 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.288193 21931 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.290357 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d2bcf30627084357a4745445324a2430 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:17.292881 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4021839a030d44ab886833307930ad33 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:17.292748 23010 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:17.335289 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 43937.9us
I20260709 13:05:17.336911 22008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:17.337456 22008 raft_consensus.cc:606] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:05:17.364256 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08001e4cc0 after lost signal to thread 20749
I20260709 13:05:17.367261 22592 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:17.374105 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08001df540 after lost signal to thread 21792
W20260709 13:05:17.379191 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08001033e0 after lost signal to thread 22617
W20260709 13:05:17.613642 23035 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:05:17.686893 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 186206us
I20260709 13:05:17.705495 22008 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.709290 21933 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:17.713896 23064 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.097s	user 0.001s	sys 0.001s
I20260709 13:05:17.713474 23010 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 13:05:17.723812 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 334515us
I20260709 13:05:17.754001 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:17.767560 21859 raft_consensus.cc:2955] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:17.786110 23071 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:17.789940 22006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:17.790802 22006 raft_consensus.cc:606] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:05:17.790256 21931 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:17.798815 21933 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.802590 21933 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.801266 23024 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 13:05:17.793504 21743 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 8a76564a0ee84b74a3af01c79300d4b7 already in progress [suppressed 3 similar messages]
I20260709 13:05:17.810815 21860 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:17.814955 23078 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:17.830412 23010 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:17.895253 21858 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.945252 22006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:17.945878 22006 raft_consensus.cc:606] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:05:17.946360 23023 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:17.949110 23081 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.156s	user 0.003s	sys 0.000s
I20260709 13:05:17.949059 23080 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:17.967149 23023 raft_consensus.cc:993] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:17.967643 23023 raft_consensus.cc:1081] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Signalling peer 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:17.969842 21857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:41262
I20260709 13:05:17.987067 22000 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:17.994894 21806 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:05:17.996989 21806 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 8a76564a0ee84b74a3af01c79300d4b7 already in progress
W20260709 13:05:17.997357 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 8a76564a0ee84b74a3af01c79300d4b7 already in progress
I20260709 13:05:17.994841 21857 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.003971 21857 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.030112 21857 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:18.038353 21858 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:18.040411 23078 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:18.043473 21858 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:18.052546 23078 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:18.074097 23078 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:18.127440 22592 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Advancing to term 2
I20260709 13:05:18.146137 22592 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:18.163594 22984 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:05:18.185276 21758 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 212) took 1022 ms. Trace:
I20260709 13:05:18.185917 21758 rpcz_store.cc:276] 0709 13:05:17.162717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.163002 (+   285us) service_pool.cc:224] Handling call
0709 13:05:17.167149 (+  4147us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b5400011210. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6ab0b2 00007f3b76624a89 00007f3b765bcda6 00007f3b7658e088 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc
0709 13:05:18.185060 (+1017911us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":819404,"reported_tablets":2,"spinlock_wait_cycles":4718848}
I20260709 13:05:17.813572 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:18.213392 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80864.8us
I20260709 13:05:18.246855 21857 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:18.298333 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 75964.8us
I20260709 13:05:18.298426 23018 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:18.318928 23018 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:18.341995 23066 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:18.342830 23066 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:18.348230 23066 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:18.354192 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:18.354809 21934 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.361999 21934 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:18.365381 23097 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:18.365942 23097 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:18.372218 23097 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:18.372044 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:18.390216 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:18.402340 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:18.425041 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:18.434430 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 198188us
I20260709 13:05:18.429052 23079 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:18.455379 23079 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:18.473387 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:18.435804 23018 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:18.521703 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 179833us
I20260709 13:05:18.535835 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:18.523788 21930 raft_consensus.cc:2955] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:18.529592 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:18.456405 23079 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:18.540220 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 65923.9us
W20260709 13:05:18.600473 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57443.1us
W20260709 13:05:18.605464 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61972.3us
W20260709 13:05:18.702221 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93549.4us
W20260709 13:05:18.773212 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 162493us
W20260709 13:05:18.818531 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44664.7us
I20260709 13:05:18.610033 23016 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:18.851943 23102 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:05:18.772961 23011 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:18.844849 23016 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:18.880620 23074 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:18.856233 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 311849us
W20260709 13:05:18.884645 23110 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
W20260709 13:05:18.890232 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 186306us
I20260709 13:05:18.873054 23022 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:18.856369 23066 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:18.894361 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43873.2us
I20260709 13:05:18.881256 23074 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:18.901721 23070 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:18.902477 23070 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:18.905812 23070 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:18.871865 23011 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:18.927416 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf880 after lost signal to thread 20749
W20260709 13:05:18.954367 23117 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:18.917372 23066 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:18.958429 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8c40 after lost signal to thread 21792
W20260709 13:05:18.986653 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 147307us
W20260709 13:05:18.988432 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 90281.4us
I20260709 13:05:19.001230 23016 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:18.915267 23022 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:19.006548 21900 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:19.009562 23100 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.010994 21825 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:18.993880 23097 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:18.999699 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08000a50e0 after lost signal to thread 21884
I20260709 13:05:19.017503 23066 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:19.018118 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:19.020926 23011 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:19.022814 23022 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:19.025506 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.026525 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.031490 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:19.030505 21881 debug-util.cc:398] Leaking SignalData structure 0x7b080030fb20 after lost signal to thread 21959
I20260709 13:05:19.018164 23097 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:19.036408 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:19.037917 22007 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:19.038218 22007 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:19.038461 23097 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:19.038988 22007 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:19.047217 22007 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:19.055557 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:19.059659 21929 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:19.071522 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:19.072091 23098 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:19.081050 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 70150.3us
I20260709 13:05:19.082283 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.084173 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:19.076368 21881 debug-util.cc:398] Leaking SignalData structure 0x7b0800332780 after lost signal to thread 22033
I20260709 13:05:19.140203 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:19.142074 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52029.8us
W20260709 13:05:19.141439 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 54188.3us
I20260709 13:05:19.056329 23100 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:19.149094 21898 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:19.135818 23106 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.075558 23098 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:19.167604 23111 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:19.168496 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08003515a0 after lost signal to thread 22617
W20260709 13:05:19.193709 23126 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:05:19.210263 23020 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:19.220640 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.177560 23100 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:19.228240 23108 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.380s	user 0.002s	sys 0.002s
I20260709 13:05:19.229691 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 206) took 1053 ms. Trace:
I20260709 13:05:19.230051 22593 rpcz_store.cc:276] 0709 13:05:18.176674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.177141 (+   467us) service_pool.cc:224] Handling call
0709 13:05:18.177660 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.178139 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.178180 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.178795 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.469410 (+290615us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:18.487494 (+ 18084us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.487742 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.487837 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.048166 (+560329us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b180014c560. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:05:19.227044 (+178878us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:05:19.229493 (+  2449us) raft_consensus.cc:1685] finished
0709 13:05:19.229512 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.229620 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19874,"prepare.run_wall_time_us":44,"raft.queue_time_us":42916,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":3972,"spinlock_wait_cycles":40354304,"thread_start_us":207054,"threads_started":1,"wal-append.queue_time_us":389226}
I20260709 13:05:19.228888 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:19.238176 23098 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:19.239562 23131 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:05:19.216221 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:19.244781 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77559.7us
I20260709 13:05:19.226130 23020 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:19.265650 23020 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:19.267469 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:19.271374 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:19.277464 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:19.280848 22007 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:19.286811 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:19.290428 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:19.309249 23021 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.322767 23074 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:19.324381 23070 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:19.324939 23070 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:19.327111 23070 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:19.336413 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.338120 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.339236 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.342943 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:19.344429 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:19.347839 23100 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:19.349121 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.355391 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:19.356657 23025 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.371218 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:19.372840 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:19.373632 23066 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:19.375569 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:19.376695 23122 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.422358 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:19.431520 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:19.437438 21855 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:19.442307 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:19.448231 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:19.448580 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:19.450979 21754 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:19.459167 23070 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.462849 22926 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.454771 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:19.473625 23118 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.472019 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:19.479760 23134 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.509774 21935 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:05:19.521162 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69367us
I20260709 13:05:19.537971 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:19.540565 23020 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:19.564294 23018 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.567703 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:19.585863 23079 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.585093 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:19.588819 23079 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.590426 22000 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:19.481210 21754 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:19.628474 21754 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:19.692298 22589 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Advancing to term 3
I20260709 13:05:19.711723 23070 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:05:19.868633 22589 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:19.885705 23118 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:19.924634 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:19.925479 21934 raft_consensus.cc:606] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:19.929069 21854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:19.930330 21854 raft_consensus.cc:606] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:19.934012 22004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:19.934998 22004 raft_consensus.cc:606] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:19.945520 22003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:19.948274 22003 raft_consensus.cc:606] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:19.953706 23079 raft_consensus.cc:1064] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:19.958350 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:19.958973 21934 raft_consensus.cc:606] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:19.966365 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:19.956853 23079 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:19.968078 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:20.016922 22587 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:20.019567 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet 4021839a030d44ab886833307930ad33 with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:20.019817 22965 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:20.021819 21916 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:20.024030 21757 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.028672 21754 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 213) took 1786 ms. Trace:
I20260709 13:05:20.037083 23153 tablet_replica.cc:333] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:20.038444 23153 raft_consensus.cc:2243] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:20.061088 23153 raft_consensus.cc:2272] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:20.041177 22003 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:20.068843 23153 ts_tablet_manager.cc:1916] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:20.029182 21754 rpcz_store.cc:276] 0709 13:05:18.254878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.261482 (+  6604us) service_pool.cc:224] Handling call
0709 13:05:20.028558 (+1767076us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:19.748203 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:19.748725 (+   522us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:19.748764 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:05:19.748790 (+    26us) tablet.cc:662] Decoding operations
0709 13:05:19.756789 (+  7999us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:19.764742 (+  7953us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:19.766119 (+  1377us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:05:19.822052 (+ 55933us) tablet.cc:701] Row locks acquired
0709 13:05:19.854276 (+ 32224us) write_op.cc:260] PREPARE: finished
0709 13:05:19.866323 (+ 12047us) write_op.cc:270] Start()
0709 13:05:19.866492 (+   169us) write_op.cc:276] Timestamp: P: 1783602319866237 usec, L: 0
0709 13:05:19.866533 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:05:19.867569 (+  1036us) log.cc:844] Serialized 1302 byte log entry
0709 13:05:19.920731 (+ 53162us) op_driver.cc:464] REPLICATION: finished
0709 13:05:19.983466 (+ 62735us) write_op.cc:301] APPLY: starting
0709 13:05:19.985690 (+  2224us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:19.989337 (+  3647us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:19.993032 (+  3695us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:20.002216 (+  9184us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:20.002462 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:05:20.002510 (+    48us) write_op.cc:312] APPLY: finished
0709 13:05:20.002970 (+   460us) log.cc:844] Serialized 35 byte log entry
0709 13:05:20.003277 (+   307us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.003394 (+   117us) write_op.cc:454] Released schema lock
0709 13:05:20.003463 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:20.003909 (+   446us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.003949 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":60444,"apply.run_cpu_time_us":3880,"apply.run_wall_time_us":21819,"num_ops":3,"prepare.queue_time_us":53092,"prepare.run_cpu_time_us":8195,"prepare.run_wall_time_us":133090,"raft.queue_time_us":15438,"raft.run_cpu_time_us":2946,"raft.run_wall_time_us":30659,"replication_time_us":54050,"thread_start_us":62727,"threads_started":4,"wal-append.queue_time_us":12271}]]}
I20260709 13:05:20.013554 21934 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:20.072844 21757 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 224) took 1911 ms. Trace:
I20260709 13:05:20.073943 23120 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:20.082506 23097 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:20.084794 21753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 209) took 1014 ms. Trace:
I20260709 13:05:20.086086 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet d2bcf30627084357a4745445324a2430 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:20.095502 23073 raft_consensus.cc:993] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:20.092427 23079 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.099984 22003 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.110062 23111 raft_consensus.cc:1081] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:20.080881 21757 rpcz_store.cc:276] 0709 13:05:18.161898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.162308 (+   410us) service_pool.cc:224] Handling call
0709 13:05:18.181306 (+ 18998us) spinlock_profiling.cc:243] Waited 8.51 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:20.072782 (+1891476us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:20.050576 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:20.051128 (+   552us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:20.051171 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:05:20.051190 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:20.051538 (+   348us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:20.051591 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:20.051620 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:20.051801 (+   181us) tablet.cc:701] Row locks acquired
0709 13:05:20.051821 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:05:20.051965 (+   144us) write_op.cc:270] Start()
0709 13:05:20.052078 (+   113us) write_op.cc:276] Timestamp: P: 1783602320051922 usec, L: 0
0709 13:05:20.052103 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:05:20.052740 (+   637us) log.cc:844] Serialized 536 byte log entry
0709 13:05:20.056732 (+  3992us) op_driver.cc:464] REPLICATION: finished
0709 13:05:20.057178 (+   446us) write_op.cc:301] APPLY: starting
0709 13:05:20.057251 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:20.057341 (+    90us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:20.057590 (+   249us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:20.057900 (+   310us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:20.058059 (+   159us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:20.058106 (+    47us) write_op.cc:312] APPLY: finished
0709 13:05:20.058300 (+   194us) log.cc:844] Serialized 23 byte log entry
0709 13:05:20.058469 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.058542 (+    73us) write_op.cc:454] Released schema lock
0709 13:05:20.058582 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:20.058970 (+   388us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.059001 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":553907,"reported_tablets":7,"spinlock_wait_cycles":19583360,"child_traces":[["op",{"apply.queue_time_us":143,"apply.run_cpu_time_us":1722,"apply.run_wall_time_us":1721,"num_ops":1,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":2451,"prepare.run_wall_time_us":2451,"replication_time_us":4486}]]}
I20260709 13:05:20.102586 21753 rpcz_store.cc:276] 0709 13:05:19.070051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.070226 (+   175us) service_pool.cc:224] Handling call
0709 13:05:19.446114 (+375888us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b5400011210. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6ab640 00007f3b76582b76 00007f3b7668cc3e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:20.084586 (+638472us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":6846208}
I20260709 13:05:20.136075 22589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57102
I20260709 13:05:20.136790 22589 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.137208 22589 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.140364 21754 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), now has a pending config: VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130), VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.153554 21758 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 118) took 1107 ms. Trace:
I20260709 13:05:20.161063 23072 raft_consensus.cc:993] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:20.162005 21758 rpcz_store.cc:276] 0709 13:05:19.045664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.045784 (+   120us) service_pool.cc:224] Handling call
0709 13:05:20.153364 (+1107580us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4548,"reported_tablets":8}
I20260709 13:05:20.170861 23011 raft_consensus.cc:1081] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:20.190217 22587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57084
I20260709 13:05:20.190954 22587 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.191555 22587 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.219954 22587 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.223179 22587 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:20.226461 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:20.227470 22003 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.227893 22003 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:20.229251 22003 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.237660 22003 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.210906 21934 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.245283 23082 raft_consensus.cc:993] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:20.247749 23082 raft_consensus.cc:1081] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:20.246264 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:20.269243 23154 tablet_replica.cc:333] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:20.273330 22008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41328
I20260709 13:05:20.273960 22008 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.274345 22008 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.275692 22589 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.277189 21854 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.291710 22003 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:20.282954 23154 raft_consensus.cc:2243] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:20.302202 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:20.302911 21935 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.303296 21935 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:20.304430 21935 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.305562 21935 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.309995 23154 raft_consensus.cc:2272] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:20.323190 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:20.324429 23113 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.318684 22008 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:20.330605 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:20.333636 22003 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.338418 22587 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.317292 23153 ts_tablet_manager.cc:1929] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:20.345569 23154 ts_tablet_manager.cc:1916] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:20.329711 22589 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:20.346619 21754 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 5, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130), VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.350637 23072 raft_consensus.cc:993] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:20.371652 23072 raft_consensus.cc:1081] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:20.369364 21854 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:20.373719 21928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33888
I20260709 13:05:20.374282 21928 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.374640 21928 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.375690 22008 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:20.362912 23153 log.cc:1199] T 4021839a030d44ab886833307930ad33 P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/4021839a030d44ab886833307930ad33
I20260709 13:05:20.379732 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:20.381045 21932 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.365051 23100 raft_consensus.cc:993] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:20.378793 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:20.381999 21860 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.382297 21860 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:20.383152 23100 raft_consensus.cc:1081] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:20.383059 21860 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:20.388926 22593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57102
I20260709 13:05:20.389475 22593 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.389727 21860 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.389859 22593 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.358050 23113 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:20.401306 23113 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.412223 21932 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:20.417595 22003 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:20.418110 21935 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:20.422806 21928 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:20.424216 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 4021839a030d44ab886833307930ad33 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:20.431991 21854 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.438796 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:20.416368 23153 tablet_replica.cc:333] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:20.440392 23155 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.440946 23155 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:20.445334 21860 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:20.452526 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:20.454470 23072 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.455044 23072 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:20.456056 23072 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:20.447114 22593 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.466334 23155 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.475729 21932 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:20.480471 21928 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:20.484503 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 116108us
I20260709 13:05:20.487324 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:20.493954 22008 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.486979 22593 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:20.494393 22008 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:20.489886 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:20.496810 22008 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:20.497179 21854 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.498169 22008 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.487504 23128 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:05:20.509287 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:20.510003 21935 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.510314 21935 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:20.508437 21758 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.519131 22008 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:20.520315 21935 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.527037 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:20.528733 23066 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.529990 23066 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:20.515498 22965 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:20.535898 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.537469 21935 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.548372 21758 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.551555 23066 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:20.550511 21928 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.565774 23153 raft_consensus.cc:2243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:20.566706 23153 raft_consensus.cc:2272] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:20.530020 21854 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:20.572729 23154 ts_tablet_manager.cc:1929] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:20.573524 23154 log.cc:1199] T d2bcf30627084357a4745445324a2430 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/d2bcf30627084357a4745445324a2430
I20260709 13:05:20.575685 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet d2bcf30627084357a4745445324a2430 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:20.575354 23153 ts_tablet_manager.cc:1916] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:20.584115 21935 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:20.587003 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:20.588625 23161 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.589902 23161 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:20.596616 23161 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:20.602403 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:20.604089 21758 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.608063 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8a76564a0ee84b74a3af01c79300d4b7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:20.609347 22008 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.624228 23153 ts_tablet_manager.cc:1929] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:20.596807 22587 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:20.625670 22008 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:20.629858 23153 log.cc:1199] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/8b5d11192c5d4d72b0bb9f3ebe9b715a
I20260709 13:05:20.630735 22965 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:20.632282 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 8b5d11192c5d4d72b0bb9f3ebe9b715a (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:20.640969 22587 raft_consensus.cc:2955] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:20.644529 21757 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 5 to 6, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.679324 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a76564a0ee84b74a3af01c79300d4b7 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:20.678259 21758 catalog_manager.cc:5697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.696888 23165 tablet_replica.cc:333] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:20.709322 21754 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.712466 23165 raft_consensus.cc:2243] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:20.713609 23165 raft_consensus.cc:2272] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:20.742728 23165 ts_tablet_manager.cc:1916] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:20.768841 23165 ts_tablet_manager.cc:1929] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:05:20.769311 23165 log.cc:1199] T 8a76564a0ee84b74a3af01c79300d4b7 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/8a76564a0ee84b74a3af01c79300d4b7
I20260709 13:05:20.771042 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 8a76564a0ee84b74a3af01c79300d4b7 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:20.877209 21932 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:20.879312 23161 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:20.914038 22003 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:20.916183 23161 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:20.933123 22001 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:20.934841 23161 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:20.956027 21854 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:20.969347 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:20.969910 23155 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:21.000721 21860 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:21.002839 23066 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.026098 21932 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:21.028030 23011 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:21.032123 22003 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:21.037247 23100 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:21.061089 21854 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:21.063083 23011 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:21.075392 21928 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:21.077420 23113 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:21.164822 23176 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:05:21.177700 22008 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:21.179697 23113 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:21.242659 21932 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:21.243896 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:21.270277 21900 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b42af043129e482498af1e2c3882a042. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.279337 21860 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.279587 22008 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.281924 23111 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.306258 23100 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:21.321975 23111 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.328924 21860 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.341567 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d830a9119594381aa253bf173c99566"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:21.342219 21932 raft_consensus.cc:606] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:21.356654 21854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1db8966fba10424a8df85ad63defc925"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:21.357524 21854 raft_consensus.cc:606] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:21.370620 21854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:21.371299 21854 raft_consensus.cc:606] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:21.372713 21860 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:21.365046 22008 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.379951 21754 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.384060 21855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:21.384671 21855 raft_consensus.cc:606] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:21.392794 22003 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.393116 21860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:21.393788 21860 raft_consensus.cc:606] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:21.394810 21928 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.395077 23082 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.401386 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:21.401952 22965 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.403117 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:05:21.406807 21825 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b8c4da9f5144e2b8007c8ffc563a584. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.471621 23133 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.488235 22003 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.493711 21928 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.533210 22003 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:21.532557 21757 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:21.656029 23133 raft_consensus.cc:993] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:21.656375 23111 raft_consensus.cc:993] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:21.656651 23133 raft_consensus.cc:1081] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:21.656900 23111 raft_consensus.cc:1081] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:21.666194 22593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57098
I20260709 13:05:21.666808 22593 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.667167 22593 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:21.671052 21928 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.675556 23024 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.677026 22001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d830a9119594381aa253bf173c99566"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:21.677806 22001 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.678251 22001 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.682660 22593 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:21.687219 22593 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:21.691056 21974 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a226c66b13a04eb389353b8e2deb175c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.691293 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:21.691640 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:21.691948 21855 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:21.692364 21855 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:21.692224 21934 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:21.693382 21855 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:21.695264 21855 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:21.704165 21855 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:21.707834 23193 ts_tablet_manager.cc:933] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:21.709122 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:21.710032 21934 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:21.712684 23113 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:21.713833 23113 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:21.714655 23193 tablet_copy_client.cc:323] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:21.714833 23113 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:21.717221 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet b42af043129e482498af1e2c3882a042 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:21.717996 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet b42af043129e482498af1e2c3882a042 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-b42af043129e482498af1e2c3882a042
I20260709 13:05:21.731552 22965 raft_consensus.cc:993] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:21.732383 23079 raft_consensus.cc:1081] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:21.735139 21934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:05:21.736861 21934 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.737926 21934 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:21.739343 23190 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.743690 21935 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.757434 23082 raft_consensus.cc:993] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:21.757923 23082 raft_consensus.cc:1081] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:21.756618 21854 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:21.759527 22003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1db8966fba10424a8df85ad63defc925"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41328
I20260709 13:05:21.760033 22003 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.760412 22003 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.763583 21758 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:21.713379 21860 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.766628 23027 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.768946 22001 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.781266 22001 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:21.781704 22008 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.784281 23082 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:21.790498 21825 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6622001ac78d44caa90b4ce7f5283aa1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.818413 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:21.770476 22003 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.833427 22003 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:21.778353 21855 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.835402 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:21.836135 21930 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.838495 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:21.839347 21854 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:21.839682 21854 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:21.823904 21928 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:21.849696 21854 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.854562 21854 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.866539 22965 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:21.878037 21934 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.880756 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:21.881434 21859 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:21.881810 21859 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:21.882553 21859 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.883687 21859 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:21.888494 21930 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:21.890915 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:21.895581 23191 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:21.896062 23191 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:21.896979 23191 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.899636 21860 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.901896 21934 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:21.829541 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:21.909111 21932 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:21.909497 21932 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:21.909711 21854 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:21.773785 21944 tablet_copy_source_session.cc:215] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:21.916039 23193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42af043129e482498af1e2c3882a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.909806 21754 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 4 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:21.912657 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:21.918989 22003 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:21.926481 22965 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.938481 23133 raft_consensus.cc:993] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:21.939354 23133 raft_consensus.cc:1081] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:21.951946 21754 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:21.957726 21928 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.962352 22003 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:21.964236 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:21.910364 21932 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.965070 23066 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:21.965600 23066 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:21.966513 23066 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.969484 21934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:05:21.970050 21934 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.970146 21859 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:21.970402 21934 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.975055 21932 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.978324 21934 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.980245 21860 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:21.984442 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:21.985291 23011 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:21.985715 23011 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:21.984820 22003 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:21.986416 23011 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:21.994002 21934 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:22.011873 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:22.011339 21754 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 3 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.012486 21859 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:22.012864 21859 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:22.013634 21859 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:22.014837 21859 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.018651 21758 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.026414 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:22.028755 22008 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.029856 21859 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:22.050272 21932 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:22.058650 23193 tablet_copy_client.cc:806] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:22.060238 21898 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:22.061203 23111 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:22.061931 23111 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:22.062877 23111 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:22.072778 23193 tablet_copy_client.cc:670] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:22.083472 22008 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:22.084228 23193 tablet_copy_client.cc:538] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:22.093299 23204 ts_tablet_manager.cc:933] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:22.101065 21754 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.088245 21758 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:22.127521 23193 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:22.129390 23204 tablet_copy_client.cc:323] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:22.131336 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 0b8c4da9f5144e2b8007c8ffc563a584 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:22.131901 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 0b8c4da9f5144e2b8007c8ffc563a584 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-0b8c4da9f5144e2b8007c8ffc563a584
I20260709 13:05:22.149044 21869 tablet_copy_source_session.cc:215] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:22.154641 23204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b8c4da9f5144e2b8007c8ffc563a584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.179221 23204 tablet_copy_client.cc:806] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:22.179833 23204 tablet_copy_client.cc:670] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:22.194860 23204 tablet_copy_client.cc:538] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:22.201792 23193 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:22.209668 23193 tablet_bootstrap.cc:492] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:22.210528 23193 ts_tablet_manager.cc:1403] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.083s	user 0.025s	sys 0.002s
I20260709 13:05:22.220626 23204 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:22.220556 23193 raft_consensus.cc:359] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:22.221683 23193 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:22.225533 23193 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:22.231678 23193 ts_tablet_manager.cc:1434] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:05:22.237300 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-b42af043129e482498af1e2c3882a042 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:22.237769 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-b42af043129e482498af1e2c3882a042 on tablet b42af043129e482498af1e2c3882a042 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:22.243505 23193 ts_tablet_manager.cc:933] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:22.245795 21860 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:22.247527 23155 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:22.256560 21932 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:22.258844 23155 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:22.266450 23193 tablet_copy_client.cc:323] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:22.268406 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet a226c66b13a04eb389353b8e2deb175c from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:22.268955 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet a226c66b13a04eb389353b8e2deb175c from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-a226c66b13a04eb389353b8e2deb175c
I20260709 13:05:22.276638 22019 tablet_copy_source_session.cc:215] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:22.295976 23193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226c66b13a04eb389353b8e2deb175c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.313997 22593 raft_consensus.cc:1217] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:22.335908 23193 tablet_copy_client.cc:806] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:22.348809 23193 tablet_copy_client.cc:670] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:22.352145 23208 ts_tablet_manager.cc:933] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:22.369366 23193 tablet_copy_client.cc:538] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:22.378769 23193 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:22.388576 23208 tablet_copy_client.cc:323] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:22.393996 23204 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:22.393872 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 6622001ac78d44caa90b4ce7f5283aa1 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:22.394629 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 6622001ac78d44caa90b4ce7f5283aa1 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-6622001ac78d44caa90b4ce7f5283aa1
I20260709 13:05:22.395174 23204 tablet_bootstrap.cc:492] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:22.398777 21934 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:22.396040 23204 ts_tablet_manager.cc:1403] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.176s	user 0.025s	sys 0.006s
I20260709 13:05:22.400478 23011 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:22.404649 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:22.413415 21869 tablet_copy_source_session.cc:215] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:22.416656 21860 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:22.418529 23208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6622001ac78d44caa90b4ce7f5283aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.423745 23111 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:22.430485 21934 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:22.432590 23204 raft_consensus.cc:359] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:22.433933 23204 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:22.436765 23011 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:22.514351 21860 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:22.515969 23190 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:22.466001 23204 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:22.520282 23204 ts_tablet_manager.cc:1434] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.000s
I20260709 13:05:22.524684 22008 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:22.528486 23073 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:22.537715 22589 raft_consensus.cc:1217] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:22.556740 21859 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:22.577383 23066 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:05:22.636493 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-0b8c4da9f5144e2b8007c8ffc563a584 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:22.636989 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-0b8c4da9f5144e2b8007c8ffc563a584 on tablet 0b8c4da9f5144e2b8007c8ffc563a584 with peer 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:05:22.663419 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78785.8us
I20260709 13:05:22.667160 23193 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:22.668097 23193 tablet_bootstrap.cc:492] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:22.668764 23193 ts_tablet_manager.cc:1403] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.290s	user 0.015s	sys 0.017s
I20260709 13:05:22.673873 21860 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:22.678992 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50677.5us
I20260709 13:05:22.689404 22008 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:22.710098 23073 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:22.691537 23231 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:05:22.718525 23193 raft_consensus.cc:359] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:22.719498 23193 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:22.723968 23208 tablet_copy_client.cc:806] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:22.724558 23208 tablet_copy_client.cc:670] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:22.815850 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:22.831497 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47903.9us
I20260709 13:05:22.720240 23193 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:22.895478 23208 tablet_copy_client.cc:538] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:05:22.906535 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 76570.3us
I20260709 13:05:22.943297 23193 ts_tablet_manager.cc:1434] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.268s	user 0.009s	sys 0.000s
I20260709 13:05:22.952577 23073 raft_consensus.cc:1064] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:22.957872 23073 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.009915 22587 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.010404 22008 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.011516 23246 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:23.012416 23241 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:23.024955 21858 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.026482 23073 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:23.036592 23208 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
W20260709 13:05:23.061174 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 155737us
W20260709 13:05:23.059564 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 129610us
I20260709 13:05:23.122675 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-a226c66b13a04eb389353b8e2deb175c received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:23.123217 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-a226c66b13a04eb389353b8e2deb175c on tablet a226c66b13a04eb389353b8e2deb175c with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:23.148314 23100 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.167012 23237 raft_consensus.cc:1064] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:23.169638 23237 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.203799 23208 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:23.209898 23208 tablet_bootstrap.cc:492] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:23.212790 23208 ts_tablet_manager.cc:1403] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.176s	user 0.022s	sys 0.004s
I20260709 13:05:23.216652 21855 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.230888 22002 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.232254 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:23.232813 21928 raft_consensus.cc:606] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:23.241479 21753 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.235289 23208 raft_consensus.cc:359] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:23.243990 23208 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:23.244648 23208 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:23.247510 23208 ts_tablet_manager.cc:1434] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:05:23.256013 22008 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.250423 22587 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.262985 22589 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.264406 23082 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:23.268290 21928 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.266144 23237 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:05:23.271193 23235 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:23.273880 22589 raft_consensus.cc:1217] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:23.296906 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:23.297660 21928 raft_consensus.cc:606] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:23.308070 22001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:23.308730 22001 raft_consensus.cc:606] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:23.330595 21855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:23.331472 21855 raft_consensus.cc:606] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:23.335290 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:23.337620 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:23.339879 23246 raft_consensus.cc:993] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:23.340366 23246 raft_consensus.cc:1081] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:23.344586 22001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:23.345104 22001 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.345454 22001 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.357609 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-6622001ac78d44caa90b4ce7f5283aa1 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:23.358125 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-6622001ac78d44caa90b4ce7f5283aa1 on tablet 6622001ac78d44caa90b4ce7f5283aa1 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:23.367795 23236 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.370254 22593 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.378191 22001 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.389603 22001 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:23.391345 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:23.392001 21932 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:23.392345 21932 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:23.393431 21932 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.394682 21932 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.396836 21928 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.397967 22001 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.399822 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:23.401835 21855 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.420938 21855 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:23.424221 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:23.429829 21932 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:23.437805 23230 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:23.440757 23230 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:23.444482 23230 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.457791 23248 raft_consensus.cc:993] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:23.458312 23248 raft_consensus.cc:1081] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:23.460482 21932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33888
I20260709 13:05:23.461457 21932 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.462433 21932 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.486800 21932 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.489283 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:23.489909 21858 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.495837 21858 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:23.497653 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:23.498095 21932 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:23.498273 22008 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:23.498134 22587 raft_consensus.cc:1217] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:23.498709 22008 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:23.499503 22008 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.497772 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:23.500912 22008 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.501530 23073 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:23.502224 23073 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:23.503293 23073 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.509263 22008 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:23.552445 23082 raft_consensus.cc:993] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:23.552995 23082 raft_consensus.cc:1081] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:23.557576 23111 raft_consensus.cc:993] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:23.558097 23111 raft_consensus.cc:1081] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:23.560289 21935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:05:23.560812 21935 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.561174 22008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:23.561833 22008 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.561174 21935 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.562273 22008 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.569896 21935 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.543926 21758 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:23.576534 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:23.577191 22587 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.582993 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:23.583690 21858 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:23.588063 21858 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:23.589052 21858 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.592500 21858 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.594434 21935 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:23.591682 22587 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:23.606180 22008 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.610773 22008 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:23.612506 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:23.613180 21854 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.617861 21753 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.622061 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:23.626405 21932 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:23.626855 21932 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:23.629053 21932 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.636649 21854 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:23.632156 21932 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.643020 21757 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:23.645426 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:23.660571 23073 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:23.661099 23073 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:23.662135 23073 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.652948 21855 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.679239 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:23.679826 21935 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.682083 23230 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:23.689498 23230 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:23.690394 23230 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:23.683312 21858 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:23.685937 21932 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:23.687420 22587 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:23.688838 21934 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:23.709988 23201 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:05:23.712827 23237 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:23.716665 22587 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:23.717221 22001 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:23.721443 23111 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:23.727530 23237 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.722882 23246 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:23.737545 21935 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.742868 22587 raft_consensus.cc:2955] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.759684 23229 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.761284 22001 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.768342 21757 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:23.783740 23263 raft_consensus.cc:1064] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:23.786561 23263 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.784734 21753 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.809545 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0b8c4da9f5144e2b8007c8ffc563a584 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:23.852406 21757 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:23.845980 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75014.1us
I20260709 13:05:23.874571 22587 raft_consensus.cc:2955] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.892077 21858 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:23.893172 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b42af043129e482498af1e2c3882a042 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:23.906737 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b8c4da9f5144e2b8007c8ffc563a584 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:23.900210 23247 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:05:23.918313 21757 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:23.932256 23274 tablet_replica.cc:333] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:23.932503 21935 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:23.939944 23272 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:23.942102 22008 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:23.943933 22587 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:23.945670 23229 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:23.945972 23274 raft_consensus.cc:2243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.948406 23272 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:23.949846 23274 raft_consensus.cc:2272] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.953960 23274 ts_tablet_manager.cc:1916] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:23.982398 23201 raft_consensus.cc:1064] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:23.985845 23201 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:23.994810 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet b42af043129e482498af1e2c3882a042 with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:24.008477 23274 ts_tablet_manager.cc:1929] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:24.019263 23274 log.cc:1199] T 0b8c4da9f5144e2b8007c8ffc563a584 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/0b8c4da9f5144e2b8007c8ffc563a584
I20260709 13:05:24.016152 21854 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:24.009634 23277 tablet_replica.cc:333] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:24.026762 23277 raft_consensus.cc:2243] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.027813 23277 raft_consensus.cc:2272] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.031416 23277 ts_tablet_manager.cc:1916] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:24.049218 21854 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:24.050567 23066 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:24.054208 21932 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:24.068482 22589 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:24.071213 23066 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:24.079154 23272 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:24.089270 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 0b8c4da9f5144e2b8007c8ffc563a584 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:24.097582 23277 ts_tablet_manager.cc:1929] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:24.098084 23277 log.cc:1199] T b42af043129e482498af1e2c3882a042 P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/b42af043129e482498af1e2c3882a042
I20260709 13:05:24.099901 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet b42af043129e482498af1e2c3882a042 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:24.128628 23248 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:24.164395 22587 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:24.164543 23263 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.166589 21932 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:24.174041 23082 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:24.179418 22965 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:24.181511 22002 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:24.187929 21859 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.188644 22589 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.211537 21860 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:24.220335 21852 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:24.230878 22965 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:24.237504 21754 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:24.257928 23263 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:05:24.284843 22001 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:24.288879 21928 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:24.244375 23066 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:05:24.306569 21860 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:24.308305 23263 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:24.325735 22965 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.330443 21935 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.339113 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:24.344008 22587 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:24.347096 23263 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:24.348454 23249 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:24.375712 22587 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.391848 23245 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.398978 22593 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.402298 21754 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.471051 21854 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:24.482858 21859 raft_consensus.cc:2955] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.490515 21932 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:24.497395 23082 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:24.505497 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a226c66b13a04eb389353b8e2deb175c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:05:24.520187 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44712.6us
I20260709 13:05:24.533836 22965 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.534991 22593 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:24.526014 21758 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.528519 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:24.564085 21932 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.572086 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet a226c66b13a04eb389353b8e2deb175c with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:24.573235 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6622001ac78d44caa90b4ce7f5283aa1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:24.584311 23082 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:24.586696 21754 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:24.579773 23299 tablet_replica.cc:333] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:24.590505 23299 raft_consensus.cc:2243] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.591042 23299 pending_rounds.cc:70] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Trying to abort 1 pending ops.
I20260709 13:05:24.591370 23299 pending_rounds.cc:77] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305635118223081472 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "a226c66b13a04eb389353b8e2deb175c" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } } }
I20260709 13:05:24.592733 23299 raft_consensus.cc:2917] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:05:24.593225 23299 raft_consensus.cc:2889] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:05:24.593644 23299 raft_consensus.cc:2272] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.598063 23299 ts_tablet_manager.cc:1916] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:24.616335 22593 raft_consensus.cc:2955] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:24.638262 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 6622001ac78d44caa90b4ce7f5283aa1 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:24.660840 23300 tablet_replica.cc:333] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:05:24.662032 23300 raft_consensus.cc:2243] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.662789 23300 pending_rounds.cc:70] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Trying to abort 1 pending ops.
I20260709 13:05:24.663391 23300 pending_rounds.cc:77] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305635119039111168 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } } }
I20260709 13:05:24.664760 23300 raft_consensus.cc:2917] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:05:24.665283 23300 raft_consensus.cc:2889] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:05:24.666057 23300 raft_consensus.cc:2272] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.680613 22008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:24.681209 22008 raft_consensus.cc:606] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:24.682442 23300 ts_tablet_manager.cc:1916] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:24.691995 21859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f588420255042dda396ba6419abf6bc"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:24.692627 21859 raft_consensus.cc:606] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:24.696318 23299 ts_tablet_manager.cc:1929] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:24.699673 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:24.706313 23299 log.cc:1199] T a226c66b13a04eb389353b8e2deb175c P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/a226c66b13a04eb389353b8e2deb175c
I20260709 13:05:24.706316 21932 raft_consensus.cc:606] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:24.710256 23230 raft_consensus.cc:993] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:24.710757 23230 raft_consensus.cc:1081] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Signalling peer 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:24.711347 23300 ts_tablet_manager.cc:1929] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:24.711845 23300 log.cc:1199] T 6622001ac78d44caa90b4ce7f5283aa1 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/6622001ac78d44caa90b4ce7f5283aa1
I20260709 13:05:24.714138 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 6622001ac78d44caa90b4ce7f5283aa1 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:24.715410 21859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:41262
I20260709 13:05:24.715941 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet a226c66b13a04eb389353b8e2deb175c (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:24.716733 21859 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:24.717170 21859 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.717082 22008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bcf30627084357a4745445324a2430"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:24.717751 22008 raft_consensus.cc:606] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:24.719184 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:24.720770 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:24.728596 21859 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:24.733161 21859 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:24.736500 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:24.737111 21932 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.747897 21932 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:24.753568 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:24.754544 22965 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:24.755008 22965 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:24.755383 23229 raft_consensus.cc:993] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:24.755940 23229 raft_consensus.cc:1081] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:24.755738 22965 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:24.771437 21757 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:24.772476 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:24.773097 22008 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:24.773392 22008 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:24.774049 22008 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:24.775218 22008 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.779321 22002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:24.779937 22002 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:24.780306 22002 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.787823 22002 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:24.790155 22002 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:24.796126 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:24.796763 21860 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.801400 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:24.801597 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:24.802016 21935 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:24.802361 21935 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:24.803046 21935 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:24.804104 21935 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.810483 21860 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:24.811985 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:24.813221 23245 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:24.813745 23245 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:24.814631 23245 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:24.818121 21935 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:24.829149 21757 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.056838 23082 raft_consensus.cc:993] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:25.057436 22965 raft_consensus.cc:1081] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:25.058863 22593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f588420255042dda396ba6419abf6bc"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57098
I20260709 13:05:25.059360 22593 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.059674 22593 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.065387 22593 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:25.067759 22593 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:25.068553 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:25.069515 21860 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.069871 21860 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:25.070667 21860 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:25.071786 21860 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.071902 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:25.072681 22002 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.080494 21860 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:25.081698 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:25.082752 23297 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.083249 23297 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:25.084117 23297 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:25.098117 22002 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:25.104403 21757 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.124017 23230 raft_consensus.cc:993] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:25.124511 23230 raft_consensus.cc:1081] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:25.126825 22593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2bcf30627084357a4745445324a2430"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57084
I20260709 13:05:25.127525 22593 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.127948 22593 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.136426 22593 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:25.139384 22593 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:25.139756 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:25.140439 21935 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.141031 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:25.146998 22002 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.147455 22002 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:25.148468 22002 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:25.149560 22008 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:25.149782 22002 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.151031 23201 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:25.157912 21935 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:25.159191 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:25.160175 23296 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.161505 23296 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:25.162525 23296 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:25.185686 22002 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:25.191946 21754 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.234553 21935 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:25.242218 23306 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:25.329167 21934 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.330580 23245 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.333617 21860 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.336179 23230 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:25.582284 21935 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:25.584026 23296 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.601743 22008 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:25.603554 23296 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.663386 21860 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:25.666206 23297 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.678388 22008 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:25.689815 23297 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:25.721978 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:25.773288 21859 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:25.785616 21825 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ef7ed88974a427b9cab09af73283182. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.789036 22003 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:25.789041 21934 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:25.790856 23306 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.792930 23306 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.804620 22965 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:25.812506 22008 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:25.817340 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:25.820336 21934 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:25.842563 22003 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:25.848093 21758 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.889050 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:25.889685 21932 raft_consensus.cc:606] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:25.927841 21859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:25.928520 21859 raft_consensus.cc:606] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:25.930161 21974 consensus_peers.cc:597] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7d337058bd1f4df1bd938c43960820be. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.934967 21932 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:25.938131 21859 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:25.943257 22003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:25.943557 23245 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.943872 22003 raft_consensus.cc:606] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:25.948372 23272 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.951582 22003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:25.952232 22003 raft_consensus.cc:606] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:25.953543 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:25.955331 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:25.963054 23272 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:25.965795 21934 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:25.982782 21859 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:25.984489 21758 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:05:25.987346 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 10076817fc0f46b4943de726ac0b508c: Remote error: Service unavailable: leader transfer in progress
I20260709 13:05:25.993654 21934 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:26.026875 21900 consensus_peers.cc:597] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f938584aad7a4b76be81dca03c9a647f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.028595 22003 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:26.030016 23246 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:26.040769 21859 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:26.041862 23111 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:26.044203 23246 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:26.059516 22003 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:26.069792 21860 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:26.073081 21758 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.165365 23334 raft_consensus.cc:993] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:26.165940 23334 raft_consensus.cc:1081] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:26.168468 22003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:26.169062 22003 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:26.169425 22003 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.186206 22003 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:26.189661 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:26.190274 21860 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.211484 21860 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:26.213425 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:26.214848 22003 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:26.214924 23272 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.216620 23272 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:26.217518 23272 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:26.220109 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:26.236617 21934 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.237119 21934 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:26.238139 21934 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:26.239475 21934 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.260422 21754 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:26.262544 21934 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:26.264500 23337 ts_tablet_manager.cc:933] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:26.268344 23327 raft_consensus.cc:993] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:26.269809 23327 raft_consensus.cc:1081] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:26.278167 21932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33888
I20260709 13:05:26.278793 21932 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:26.279142 21932 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.300657 21932 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:26.306820 21932 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:26.310324 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:26.310906 22002 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:26.311194 22002 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:26.311919 22002 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:26.310269 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:26.313171 22002 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.314010 22587 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.320729 23337 tablet_copy_client.cc:323] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:26.324165 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 5ef7ed88974a427b9cab09af73283182 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:26.324719 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 5ef7ed88974a427b9cab09af73283182 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-5ef7ed88974a427b9cab09af73283182
I20260709 13:05:26.328928 22587 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:26.330816 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:26.332018 23333 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:26.332640 21869 tablet_copy_source_session.cc:215] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:26.335592 23333 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:26.336541 23333 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:26.337642 23337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef7ed88974a427b9cab09af73283182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:26.359184 22002 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:26.386322 21754 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 3 to 4, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.395988 23337 tablet_copy_client.cc:806] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:26.396768 23337 tablet_copy_client.cc:670] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:26.404538 23306 raft_consensus.cc:993] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:26.413118 23306 raft_consensus.cc:1081] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:26.423159 23337 tablet_copy_client.cc:538] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:26.425284 21932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:05:26.426121 21932 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:26.426563 21932 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.443720 21932 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:26.454686 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:26.455330 21859 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:26.455641 21859 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:26.456422 21859 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:26.457738 21859 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.471312 23340 ts_tablet_manager.cc:933] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:26.486397 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:26.487102 22002 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.494534 22002 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:26.495934 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:26.496874 23334 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:26.497375 23334 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:26.498157 23334 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:26.500274 21859 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:26.504793 23337 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:26.510290 23230 raft_consensus.cc:993] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:26.510823 23230 raft_consensus.cc:1081] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:26.517835 21860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:41262
I20260709 13:05:26.518462 21860 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:26.522090 21860 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.559073 21932 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:26.562901 23340 tablet_copy_client.cc:323] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:26.566390 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 7d337058bd1f4df1bd938c43960820be from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:26.566989 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 7d337058bd1f4df1bd938c43960820be from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-7d337058bd1f4df1bd938c43960820be
I20260709 13:05:26.576376 23337 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:26.577452 23337 tablet_bootstrap.cc:492] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:26.578222 23337 ts_tablet_manager.cc:1403] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.074s	user 0.019s	sys 0.016s
I20260709 13:05:26.581758 23337 raft_consensus.cc:359] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:26.582679 23337 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:26.583362 23337 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:26.586138 23344 ts_tablet_manager.cc:933] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:26.590590 22019 tablet_copy_source_session.cc:215] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:26.600425 23337 ts_tablet_manager.cc:1434] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 13:05:26.602365 21860 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:26.628243 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:26.628794 22003 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.629081 22003 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:26.629930 22003 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:26.631150 22003 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.613404 23340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d337058bd1f4df1bd938c43960820be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:26.619407 21758 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 3 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.623528 23344 tablet_copy_client.cc:323] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:26.624938 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-5ef7ed88974a427b9cab09af73283182 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:26.641544 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-5ef7ed88974a427b9cab09af73283182 on tablet 5ef7ed88974a427b9cab09af73283182 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:26.649605 21860 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:26.650949 22003 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:26.651036 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:26.651722 22589 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.653822 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:26.654891 23201 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.656169 23201 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:26.657187 23201 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:26.672798 21860 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.674948 23272 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:26.679719 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet f938584aad7a4b76be81dca03c9a647f from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:26.680263 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet f938584aad7a4b76be81dca03c9a647f from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-f938584aad7a4b76be81dca03c9a647f
I20260709 13:05:26.687253 22587 raft_consensus.cc:1217] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:26.688560 22589 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:26.688750 21944 tablet_copy_source_session.cc:215] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:26.688937 21935 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.692030 23245 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:26.695948 23344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f938584aad7a4b76be81dca03c9a647f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:26.753512 23344 tablet_copy_client.cc:806] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:26.754132 23344 tablet_copy_client.cc:670] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:26.764017 23344 tablet_copy_client.cc:538] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:26.771246 23340 tablet_copy_client.cc:806] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:26.776726 23340 tablet_copy_client.cc:670] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:26.781970 23344 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:26.786382 23340 tablet_copy_client.cc:538] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:26.792680 22008 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:26.807883 23246 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:26.823988 21757 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 2 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.869680 23340 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:26.887666 23344 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:26.893465 23344 tablet_bootstrap.cc:492] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:26.894469 23344 ts_tablet_manager.cc:1403] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.113s	user 0.023s	sys 0.012s
I20260709 13:05:26.898334 23344 raft_consensus.cc:359] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:26.899253 23344 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:26.899972 23344 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:26.902094 23344 ts_tablet_manager.cc:1434] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:05:26.907424 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-f938584aad7a4b76be81dca03c9a647f received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:26.907935 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-f938584aad7a4b76be81dca03c9a647f on tablet f938584aad7a4b76be81dca03c9a647f with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:26.940264 22587 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:26.942667 23246 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:26.956532 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:26.997730 23340 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:27.020792 23340 tablet_bootstrap.cc:492] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:27.022029 23340 ts_tablet_manager.cc:1403] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.165s	user 0.024s	sys 0.000s
I20260709 13:05:27.040292 23340 raft_consensus.cc:359] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:27.045827 23340 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:27.046667 23340 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:27.085402 23340 ts_tablet_manager.cc:1434] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.003s
I20260709 13:05:27.087314 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-7d337058bd1f4df1bd938c43960820be received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:27.087837 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-7d337058bd1f4df1bd938c43960820be on tablet 7d337058bd1f4df1bd938c43960820be with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:27.185523 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:27.219648 22589 raft_consensus.cc:1217] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:05:27.378741 22002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:27.379366 22002 raft_consensus.cc:606] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:27.387513 22008 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:27.407859 23334 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:27.435808 22007 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:27.437808 23306 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:27.470008 21858 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:27.480314 23246 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:27.528009 22593 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:27.542088 23201 raft_consensus.cc:1064] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:27.544859 23201 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.586664 22002 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:27.591058 23306 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:27.593453 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 200644us
I20260709 13:05:27.595191 23361 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:27.610562 21859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:27.611196 21859 raft_consensus.cc:606] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:27.627553 21934 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:05:27.655051 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70573.4us
I20260709 13:05:27.664038 23364 raft_consensus.cc:993] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:27.664845 23364 raft_consensus.cc:1081] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:27.665704 23201 raft_consensus.cc:993] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:27.663609 23361 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:27.667244 23334 raft_consensus.cc:1064] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:27.674918 22589 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:27.674458 23334 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.682613 21858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:41262
I20260709 13:05:27.683374 21858 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:27.687415 21858 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.694229 23368 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:27.697737 22593 raft_consensus.cc:1217] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:27.699775 23082 raft_consensus.cc:1081] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:27.702222 21934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:05:27.702765 21934 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:27.703121 21934 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.715265 21930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:27.715921 21930 raft_consensus.cc:606] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:27.717141 22593 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:27.718151 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:27.718698 23333 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:27.720037 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:27.722812 22008 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:27.728892 21858 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:27.731803 23363 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:27.732836 21859 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:27.735363 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:27.735972 21935 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.732887 21934 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.771481 21858 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:27.776564 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:27.780227 22008 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:27.780558 22008 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:27.781323 22008 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:27.781709 21934 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:27.782496 22008 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.790030 21935 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:27.792930 23379 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:05:27.793777 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:27.794358 22588 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.797623 23246 raft_consensus.cc:993] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:27.800017 23347 raft_consensus.cc:1081] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:27.807693 22007 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:27.808276 22007 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:27.808619 22007 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.840256 23368 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:27.791742 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:27.845888 22002 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:27.846820 22588 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
W20260709 13:05:27.852279 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60621.9us
I20260709 13:05:27.853637 22008 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:27.852049 22007 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.857546 23322 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:27.859040 23322 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:27.859891 23322 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:27.858070 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:27.866294 23246 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:27.866750 23246 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:27.772622 23334 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:05:27.876057 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:27.867399 23246 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.876636 22588 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.876760 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:27.877331 21935 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:27.877662 21935 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:27.878465 21935 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.879484 21935 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.875957 22007 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:27.886437 21935 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:27.894822 22588 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:27.896842 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:27.897575 21859 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:27.897907 21859 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:27.898679 21859 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:27.899927 21859 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.905236 21935 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:27.896983 22588 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:27.890278 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:27.919901 21859 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:27.945214 23366 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:27.968436 21758 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 2 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.974375 21758 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:27.980769 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66011.1us
I20260709 13:05:28.011085 23363 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.012953 22593 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:28.037462 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43932us
I20260709 13:05:28.055562 23366 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:28.057061 23366 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:28.075170 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 98890.3us
W20260709 13:05:28.237198 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 176127us
W20260709 13:05:28.263417 21956 debug-util.cc:398] Leaking SignalData structure 0x7b080004d260 after lost signal to thread 20749
W20260709 13:05:28.264635 21956 debug-util.cc:398] Leaking SignalData structure 0x7b080018c040 after lost signal to thread 21792
W20260709 13:05:28.265976 21956 debug-util.cc:398] Leaking SignalData structure 0x7b0800234ec0 after lost signal to thread 21884
W20260709 13:05:28.267143 21956 debug-util.cc:398] Leaking SignalData structure 0x7b080034f020 after lost signal to thread 21959
W20260709 13:05:28.269335 21956 debug-util.cc:398] Leaking SignalData structure 0x7b0800253aa0 after lost signal to thread 22033
W20260709 13:05:28.273777 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 140105us
I20260709 13:05:28.290452 21753 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.302685 23272 raft_consensus.cc:1064] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:28.311295 23272 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:28.313278 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 242239us
W20260709 13:05:28.309235 21956 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4500 after lost signal to thread 22617
I20260709 13:05:28.332547 22004 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:28.334102 23306 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:28.338083 21852 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.345237 21935 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:28.350564 22004 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.353812 23399 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:28.404973 22588 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:28.404990 21932 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:28.406695 23272 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:28.409703 21859 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:28.411746 23400 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:28.414355 23390 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:28.435658 21854 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.438184 23387 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:28.476826 22587 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.484432 23347 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:28.510906 22593 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.523516 23400 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:28.580324 23413 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:05:28.582665 23400 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.585865 22588 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.592048 21932 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.596431 21934 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.633395 21754 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.635438 21754 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.618367 23366 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:05:28.683663 21858 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:28.690970 21932 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:28.694149 21758 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.707767 21852 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.724552 22004 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:28.726387 23306 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:28.728076 22589 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.729534 23333 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:28.734977 22592 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:28.737607 23306 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:28.739527 23382 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.754520 22004 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.757889 22589 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.759647 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:28.760457 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f938584aad7a4b76be81dca03c9a647f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:28.762158 23082 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.767210 22002 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:28.801822 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ef7ed88974a427b9cab09af73283182 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:28.795826 23333 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:05:28.933828 23375 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:05:28.940196 23383 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 13:05:28.940006 23377 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:28.976449 22002 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:28.964818 23422 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 13:05:29.038261 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 196824us
I20260709 13:05:29.076721 22589 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:05:29.146301 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60102.7us
W20260709 13:05:29.153527 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66136.8us
I20260709 13:05:29.155555 22007 raft_consensus.cc:2955] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:29.107786 23390 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 13:05:29.214246 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 83041.8us
I20260709 13:05:29.291890 21754 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:29.333716 22589 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:29.329849 23245 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:29.353257 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 131552us
I20260709 13:05:29.320137 22592 raft_consensus.cc:2955] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:29.386708 21758 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 4 to 5, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:29.396754 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 117005us
W20260709 13:05:29.431131 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 135200us
I20260709 13:05:29.439565 21928 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:29.445124 21743 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7d337058bd1f4df1bd938c43960820be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:29.469123 23435 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:29.471400 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:29.477119 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ef7ed88974a427b9cab09af73283182 with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33884
I20260709 13:05:29.490870 23453 tablet_replica.cc:333] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:05:29.495071 23453 raft_consensus.cc:2243] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.499915 23453 raft_consensus.cc:2272] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.514204 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet f938584aad7a4b76be81dca03c9a647f with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41260
W20260709 13:05:29.516201 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47074.7us
I20260709 13:05:29.517109 21754 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 1 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.528033 23453 ts_tablet_manager.cc:1916] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:29.552254 21848 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d337058bd1f4df1bd938c43960820be with delete_type TABLET_DATA_TOMBSTONED (TS 8f3b35a35257416db3eaaf364cf6b52b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41260
I20260709 13:05:29.549708 21935 raft_consensus.cc:2955] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:29.557199 23454 tablet_replica.cc:333] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:29.559835 23454 raft_consensus.cc:2243] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.561575 23454 raft_consensus.cc:2272] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.564504 23454 ts_tablet_manager.cc:1916] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:29.569770 23453 ts_tablet_manager.cc:1929] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:29.571970 23453 log.cc:1199] T 5ef7ed88974a427b9cab09af73283182 P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/5ef7ed88974a427b9cab09af73283182
I20260709 13:05:29.584568 23454 ts_tablet_manager.cc:1929] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:05:29.585088 23454 log.cc:1199] T f938584aad7a4b76be81dca03c9a647f P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/f938584aad7a4b76be81dca03c9a647f
I20260709 13:05:29.611552 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 5ef7ed88974a427b9cab09af73283182 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
W20260709 13:05:29.614192 21743 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 7d337058bd1f4df1bd938c43960820be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:05:29.696657 23454 tablet_replica.cc:333] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:05:29.682674 21935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:29.700704 21935 raft_consensus.cc:606] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:29.741968 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet f938584aad7a4b76be81dca03c9a647f (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:29.768852 22003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:29.769716 22003 raft_consensus.cc:606] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:29.803766 21852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:29.804651 21852 raft_consensus.cc:606] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:29.831615 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:29.832433 21932 raft_consensus.cc:606] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:29.836428 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:29.838294 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:29.863282 23454 raft_consensus.cc:2243] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:29.864607 23454 raft_consensus.cc:2272] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:29.867791 23454 ts_tablet_manager.cc:1916] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:29.887936 23435 raft_consensus.cc:993] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:29.888554 23435 raft_consensus.cc:1081] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 34f3b83da84448bf8aac62b37a1486eb to start an election
I20260709 13:05:29.890571 22593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
 from {username='slave'} at 127.0.0.1:57084
I20260709 13:05:29.891232 22593 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.891687 22593 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.901249 22593 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:29.905987 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:29.906898 22004 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.907179 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:29.907290 22004 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:29.918076 22004 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:29.919628 22004 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.919878 21932 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.908056 23363 raft_consensus.cc:993] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:29.921371 23363 raft_consensus.cc:1081] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:29.923524 22001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:29.924074 22001 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.924423 22001 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.907454 22593 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:29.942023 22004 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:29.944348 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:29.946050 23449 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.946681 23449 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:29.948042 23449 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:29.953789 23201 raft_consensus.cc:993] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:29.954252 23201 raft_consensus.cc:1081] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:29.959947 23454 ts_tablet_manager.cc:1929] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:29.960361 21928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:05:29.963239 21928 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.963716 21928 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.962225 21932 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:29.953462 22001 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:29.972784 22001 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:29.973937 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:29.974728 21852 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.984733 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:29.989447 21932 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.989880 21932 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:29.990724 21932 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:29.996326 21932 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.991663 23454 log.cc:1199] T 7d337058bd1f4df1bd938c43960820be P 8f3b35a35257416db3eaaf364cf6b52b: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/wals/7d337058bd1f4df1bd938c43960820be
I20260709 13:05:29.988459 21852 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:30.001083 21742 catalog_manager.cc:5028] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): tablet 7d337058bd1f4df1bd938c43960820be (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:05:30.001430 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:30.003396 21928 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:30.005097 23393 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:30.007704 23393 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:30.008881 23393 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:30.021394 21928 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:30.024988 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:30.025964 22003 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:30.037423 23414 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.016s
I20260709 13:05:30.048815 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:30.050544 21857 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:30.050869 21857 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:30.051689 21857 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:30.054657 21857 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.057595 21932 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:30.059638 22003 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:30.032841 21753 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:30.065117 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:30.073378 23382 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:30.074091 23382 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:30.090881 23382 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:30.103940 21758 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:30.111910 21857 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:30.135068 21754 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.246570 23438 raft_consensus.cc:993] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:30.247161 23438 raft_consensus.cc:1081] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:30.249756 22004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:30.250465 22004 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:30.250939 22004 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.273408 22004 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:30.281909 22004 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:30.283142 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:30.283793 21928 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:30.284093 21928 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:30.284242 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:30.285038 22592 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.285071 21928 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:30.291162 21928 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.309166 21928 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:30.310352 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:30.311468 23400 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:30.311980 23400 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:30.312924 23400 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:30.319788 22592 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:30.351854 21754 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.424407 22004 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:30.426348 23449 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:30.439405 21928 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.441236 23393 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:30.482719 22001 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.485082 23387 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:30.494328 21935 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:30.495620 23449 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:30.527683 21858 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.529542 23393 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:30.564100 21852 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.569921 23363 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:30.839931 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:30.877158 22592 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:30.881487 23393 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:30.901022 21932 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:30.925818 21928 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:30.933178 23435 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:30.964564 21900 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4458afcc40954b359ea857cddbfdd681. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.972420 22001 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:30.977530 21857 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:30.981346 23477 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:30.982458 23438 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:31.014139 23394 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.015631 21857 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.057554 22004 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.059841 22001 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:31.101688 21974 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9fc728770e5c4aa1ac36a662cebb8598. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.103228 21754 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.114503 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:31.133422 21928 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:31.146150 21858 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:31.149178 23393 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:31.154475 23442 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:31.182238 23245 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.197860 21928 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:31.202387 21932 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.218048 21857 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:05:31.218024 21900 consensus_peers.cc:597] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a5e5e01f84ee40afb9b688bf0b446c6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.220253 23394 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:31.237421 22004 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:31.239207 23394 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:31.245189 23387 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.253667 21857 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.271867 21858 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.293275 22004 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.303936 22001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:31.305038 22001 raft_consensus.cc:606] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:31.309469 22001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:31.310029 22001 raft_consensus.cc:606] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:31.312886 21858 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:31.313591 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:31.314599 21932 raft_consensus.cc:606] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:31.326771 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:31.328722 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:31.348062 21758 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.350500 21758 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 3, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.362357 21932 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:31.368374 23432 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:31.405906 22001 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:31.408686 23431 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:31.426103 21825 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2167f46d5e1049268b9ce66dbf46f781. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.447476 23387 raft_consensus.cc:993] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:31.448053 23387 raft_consensus.cc:1081] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:31.452625 22004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:31.459508 23431 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.484535 22004 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:31.485137 22004 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.464110 22001 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.509510 21935 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.514055 23491 ts_tablet_manager.cc:933] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:31.538215 23496 ts_tablet_manager.cc:933] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:31.547248 23496 tablet_copy_client.cc:323] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:05:31.549706 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 9fc728770e5c4aa1ac36a662cebb8598 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:05:31.550320 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 9fc728770e5c4aa1ac36a662cebb8598 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-9fc728770e5c4aa1ac36a662cebb8598
I20260709 13:05:31.593550 23245 raft_consensus.cc:993] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:05:31.597463 21754 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index -1 to 2, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 1 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:05:31.606182 22004 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.599695 22019 tablet_copy_source_session.cc:215] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:31.615870 23393 raft_consensus.cc:1081] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
W20260709 13:05:31.619079 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 87269.6us
I20260709 13:05:31.629235 23491 tablet_copy_client.cc:323] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:31.631011 23496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc728770e5c4aa1ac36a662cebb8598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:31.637230 21932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33888
I20260709 13:05:31.637624 22004 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:31.638046 21932 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:31.639293 21932 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.640228 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 4458afcc40954b359ea857cddbfdd681 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:31.640776 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 4458afcc40954b359ea857cddbfdd681 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-4458afcc40954b359ea857cddbfdd681
I20260709 13:05:31.645849 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:31.646477 21928 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:31.646842 21928 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:31.661729 21928 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:31.665097 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 69601.2us
I20260709 13:05:31.667096 21928 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.676787 21928 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:31.675845 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:31.678110 21852 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.684257 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:31.690009 23442 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:31.690560 23442 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:31.691586 23442 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.694957 23435 raft_consensus.cc:993] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:31.695533 23435 raft_consensus.cc:1081] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Signalling peer 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:05:31.690291 21932 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.700043 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:31.700807 22004 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:31.701192 22004 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:31.702765 21932 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:31.704900 21857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:41262
I20260709 13:05:31.705691 21857 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:31.706245 21857 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.704015 21944 tablet_copy_source_session.cc:215] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:31.723559 21857 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.725659 21857 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:31.727540 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:31.728133 22001 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:31.728457 22001 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:31.713548 22004 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.729319 22001 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.730021 21852 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:31.730465 22004 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.730487 22001 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.743180 22001 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:31.744745 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:31.745380 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:31.745956 21932 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.752795 23494 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:31.752674 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 85731.3us
I20260709 13:05:31.753427 23494 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:31.754383 23494 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.756784 23496 tablet_copy_client.cc:806] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:31.757414 23496 tablet_copy_client.cc:670] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:31.766443 22004 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:31.775087 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:31.776473 23502 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:31.776908 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 71697.9us
I20260709 13:05:31.776877 23496 tablet_copy_client.cc:538] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:31.780606 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:31.785921 21857 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.790119 23507 ts_tablet_manager.cc:933] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:31.804772 21932 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:31.830082 21857 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:31.834022 23510 ts_tablet_manager.cc:933] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
I20260709 13:05:31.809175 23502 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:31.862061 23491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4458afcc40954b359ea857cddbfdd681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:31.865235 23507 tablet_copy_client.cc:323] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:05:31.871243 23496 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:31.874655 23502 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:31.880712 21753 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 2 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.884338 21754 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:31.933269 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 86745.8us
I20260709 13:05:31.944353 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet a5e5e01f84ee40afb9b688bf0b446c6c from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:05:31.944963 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet a5e5e01f84ee40afb9b688bf0b446c6c from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-a5e5e01f84ee40afb9b688bf0b446c6c
I20260709 13:05:31.962638 23491 tablet_copy_client.cc:806] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:31.963358 23491 tablet_copy_client.cc:670] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:31.966998 21944 tablet_copy_source_session.cc:215] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:31.979017 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e5e01f84ee40afb9b688bf0b446c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:31.987701 23491 tablet_copy_client.cc:538] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:31.995002 23510 tablet_copy_client.cc:323] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.129:38927
I20260709 13:05:32.000074 21869 tablet_copy_service.cc:140] P 8f3b35a35257416db3eaaf364cf6b52b: Received BeginTabletCopySession request for tablet 2167f46d5e1049268b9ce66dbf46f781 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:42922)
I20260709 13:05:32.000636 21869 tablet_copy_service.cc:161] P 8f3b35a35257416db3eaaf364cf6b52b: Beginning new tablet copy session on tablet 2167f46d5e1049268b9ce66dbf46f781 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:42922: session id = 34f3b83da84448bf8aac62b37a1486eb-2167f46d5e1049268b9ce66dbf46f781
I20260709 13:05:32.021216 23507 tablet_copy_client.cc:806] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:32.022439 23507 tablet_copy_client.cc:670] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:32.025978 23491 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:32.047577 23507 tablet_copy_client.cc:538] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:32.059255 21869 tablet_copy_source_session.cc:215] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:32.061877 21754 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:32.068785 23507 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:32.072160 23510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2167f46d5e1049268b9ce66dbf46f781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:32.116101 23496 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.117298 23510 tablet_copy_client.cc:806] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:32.123930 23510 tablet_copy_client.cc:670] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:32.132566 23510 tablet_copy_client.cc:538] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:32.133363 23496 tablet_bootstrap.cc:492] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:32.140805 21858 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:32.150243 23245 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:32.150571 23496 ts_tablet_manager.cc:1403] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.280s	user 0.027s	sys 0.020s
I20260709 13:05:32.185734 21928 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:32.189858 23497 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:32.188045 23496 raft_consensus.cc:359] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.192852 23496 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:32.194774 23496 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.208793 23491 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.210297 23491 tablet_bootstrap.cc:492] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:32.211377 23491 ts_tablet_manager.cc:1403] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.186s	user 0.030s	sys 0.025s
I20260709 13:05:32.216414 23491 raft_consensus.cc:359] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.217389 23491 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:32.218048 23491 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.225622 23491 ts_tablet_manager.cc:1434] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:05:32.218833 23496 ts_tablet_manager.cc:1434] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.003s
I20260709 13:05:32.229955 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-4458afcc40954b359ea857cddbfdd681 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:32.231328 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-4458afcc40954b359ea857cddbfdd681 on tablet 4458afcc40954b359ea857cddbfdd681 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:32.234107 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-9fc728770e5c4aa1ac36a662cebb8598 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:05:32.234628 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-9fc728770e5c4aa1ac36a662cebb8598 on tablet 9fc728770e5c4aa1ac36a662cebb8598 with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:32.245816 23510 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:05:32.248086 23507 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.250345 23507 tablet_bootstrap.cc:492] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:32.265925 21928 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:32.266513 23507 ts_tablet_manager.cc:1403] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.198s	user 0.028s	sys 0.003s
I20260709 13:05:32.271210 23245 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:32.287858 23507 raft_consensus.cc:359] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.266520 22587 raft_consensus.cc:1217] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:32.290637 21852 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:32.294571 23514 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:32.288821 23507 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:32.298398 23507 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.300935 23507 ts_tablet_manager.cc:1434] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:05:32.310262 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-a5e5e01f84ee40afb9b688bf0b446c6c received from {username='slave'} at 127.0.0.1:37776
I20260709 13:05:32.310727 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-a5e5e01f84ee40afb9b688bf0b446c6c on tablet a5e5e01f84ee40afb9b688bf0b446c6c with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:32.347015 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:32.397248 22004 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:32.411496 23500 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:32.423475 22001 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:32.442059 23477 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:32.537878 23510 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.539050 23510 tablet_bootstrap.cc:492] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:05:32.540009 23510 ts_tablet_manager.cc:1403] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.294s	user 0.021s	sys 0.013s
W20260709 13:05:32.546598 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 82517.2us
W20260709 13:05:32.555035 23521 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:05:32.624117 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 126870us
I20260709 13:05:32.673048 23510 raft_consensus.cc:359] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:32.673919 23510 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:05:32.674623 23510 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:32.750066 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73822.6us
W20260709 13:05:32.943344 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 80491.1us
I20260709 13:05:32.944131 22587 raft_consensus.cc:1217] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 13:05:32.987705 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 336446us
W20260709 13:05:33.037725 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44602.7us
W20260709 13:05:33.037405 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 77666.2us
I20260709 13:05:33.078779 23510 ts_tablet_manager.cc:1434] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.538s	user 0.009s	sys 0.009s
W20260709 13:05:33.094038 23510 ts_tablet_manager.cc:1467] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Tablet startup took 553ms
I20260709 13:05:33.154613 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:33.158700 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 119664us
W20260709 13:05:33.169147 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 84663.3us
W20260709 13:05:33.188313 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 114596us
W20260709 13:05:33.158408 23510 ts_tablet_manager.cc:1469] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Trace:
0709 13:05:31.771271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.773924 (+  2653us) service_pool.cc:224] Handling call
0709 13:05:31.848185 (+ 74261us) ts_tablet_manager.cc:934] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927)
0709 13:05:32.071142 (+222957us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:32.099376 (+ 28234us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:32.099548 (+   172us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:32.113079 (+ 13531us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:32.113385 (+   306us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:32.117036 (+  3651us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:32.117099 (+    63us) inbound_call.cc:177] Queueing success response
0709 13:05:32.245450 (+128351us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:32.540443 (+294993us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:32.600213 (+ 59770us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:32.610561 (+ 10348us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":18009,"raft.queue_time_us":70761,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"tablet-copy.queue_time_us":59509,"tablet-download-pool.queue_time_us":118,"tablet-download-pool.run_cpu_time_us":1125,"tablet-download-pool.run_wall_time_us":7898,"thread_start_us":135422,"threads_started":2}
I20260709 13:05:33.235157 22592 raft_consensus.cc:1217] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:05:33.256683 22592 raft_consensus.cc:1217] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:33.300166 21869 tablet_copy_service.cc:342] P 8f3b35a35257416db3eaaf364cf6b52b: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-2167f46d5e1049268b9ce66dbf46f781 received from {username='slave'} at 127.0.0.1:42922
I20260709 13:05:33.300689 21869 tablet_copy_service.cc:434] P 8f3b35a35257416db3eaaf364cf6b52b: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-2167f46d5e1049268b9ce66dbf46f781 on tablet 2167f46d5e1049268b9ce66dbf46f781 with peer 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:05:33.368976 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53230.4us
W20260709 13:05:33.522882 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 211107us
I20260709 13:05:34.789675 23576 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.789980 23577 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.791005 23577 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.790390 23576 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.793679 23576 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.794032 23577 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.817440 23578 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.818413 23578 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.820017 23577 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:34.820474 23576 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.821162 23578 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.820701 23577 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.823520 23577 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.821288 23576 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.827915 23576 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.834425 23577 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:34.835112 23577 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.837169 23580 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.837402 23577 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.837975 23577 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.837816 23580 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.838537 23577 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.841178 23581 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:34.841715 23581 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.842803 23502 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:34.843346 23502 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.843851 23581 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.845299 23502 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.848878 23578 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:34.849548 23578 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.852398 23578 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.852814 23576 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:34.853154 23581 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:34.854250 23581 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.853526 23576 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.859505 23584 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.860165 23584 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.861959 23576 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.858084 23581 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.864257 23576 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:34.865165 23581 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:34.858568 23578 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:34.867421 23578 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.870240 23578 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.873421 23578 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:34.855093 23583 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:34.876188 23578 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:34.876194 23583 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:34.878706 23583 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:34.878847 23578 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.853681 23502 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.631486 23502 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.633773 23502 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.840715 23577 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.866371 23581 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:35.635886 23581 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.862787 23584 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:34.865057 23576 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.186748 23579 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:34.840741 23580 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:35.578202 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.000s	sys 0.008s
W20260709 13:05:35.637270 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.001s	sys 0.008s
I20260709 13:05:35.637049 23579 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.638234 23576 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.638304 23585 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.638729 23584 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.639220 23579 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.638979 23585 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.639227 23584 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.641211 23581 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.641562 23585 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.641927 23581 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.643419 23581 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.641214 23584 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.650009 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 254) took 2117 ms. Trace:
I20260709 13:05:35.650450 22004 rpcz_store.cc:276] 0709 13:05:33.532293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.649413 (+2117120us) service_pool.cc:224] Handling call
0709 13:05:35.649630 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.649759 (+   129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.649780 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.649812 (+    32us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:35.649849 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.649890 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.649957 (+    67us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.651459 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.652421 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.653735 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.654400 21851 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:35.656565 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.657189 21851 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:35.659040 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.659701 21851 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:35.662142 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.666356 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.666558 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.667049 21856 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:35.667232 21857 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:35.666616 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:35.672540 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:35.673410 23581 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:35.673866 23581 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.674455 23581 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.676712 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 232) took 2338 ms. Trace:
I20260709 13:05:35.677152 21935 rpcz_store.cc:276] 0709 13:05:33.338742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.675812 (+2337070us) service_pool.cc:224] Handling call
0709 13:05:35.676207 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.676381 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.676410 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.676451 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:35.676479 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.676537 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.676645 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.681767 23588 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.682463 23588 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.685555 23588 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.686180 23588 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.686796 23588 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.690078 23588 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.694134 23588 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.694775 23588 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.697036 23588 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.697670 23588 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.698280 23588 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.700439 23588 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.700990 23588 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.701579 23588 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.703876 23588 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.696758 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1454) took 2694 ms. Trace:
I20260709 13:05:35.704887 21858 rpcz_store.cc:276] 0709 13:05:33.002798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.004825 (+  2027us) service_pool.cc:224] Handling call
0709 13:05:35.696007 (+2691182us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:35.696229 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.696415 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.696443 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.696482 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:35.696517 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.696572 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.696671 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15689600}
I20260709 13:05:35.708148 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:35.713552 23588 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:35.719156 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002e5e60 after lost signal to thread 20749
I20260709 13:05:35.718977 23588 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.721530 23588 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.715584 23590 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.722234 23590 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.724475 23590 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.716313 23591 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.716725 23577 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.725173 23591 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.717288 23576 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.726261 23576 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.727483 23591 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.728551 23576 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.729074 23576 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.729521 23576 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:35.731554 23576 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.732026 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:35.732841 23502 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:35.733314 23502 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.733661 23502 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.733757 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:35.734707 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:35.736727 23570 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:35.737119 23570 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.737205 23580 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:35.737519 23570 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.737668 23580 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.738188 23580 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:35.738417 23588 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.739028 23584 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:35.739246 23588 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.717571 23561 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.745360 23561 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.748528 23574 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.749100 23574 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.751294 23574 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.751773 23574 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.752216 23574 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.753418 23557 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.753772 23545 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.754370 23545 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:35.754813 23580 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.757282 23568 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.757797 23568 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.758370 23570 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.760032 23568 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.760411 23570 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.760572 23568 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.754006 23557 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.761292 23559 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.761868 23559 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.763090 23557 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.763964 23559 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.739499 23584 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:35.765033 23503 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.753324 23504 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.765934 23541 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.725246 23577 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.765220 23584 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.718227 23578 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.716928 23587 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.764545 23557 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.756749 23545 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.766070 23504 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.776891 23589 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.765745 23503 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.784070 23563 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.776666 23591 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.780311 23442 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.766973 23393 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.756767 23580 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.784606 23563 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.788125 23560 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:35.784796 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.795157 23574 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.798148 23393 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.832630 23504 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.833835 23393 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.780124 23587 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.828648 22589 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:35.830639 23503 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.798020 23442 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:35.796334 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 60639.2us
I20260709 13:05:35.796438 23591 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.839515 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:35.840240 21858 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.789340 23559 raft_consensus.cc:1064] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:35.803342 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.777455 23502 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.837072 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.780076 23568 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.852717 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.854485 23568 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.855044 23568 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.855492 23568 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.781677 23557 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.859335 23557 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.798872 23588 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.787029 23578 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.847138 23591 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.848315 23577 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.849606 23574 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.844915 23561 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:35.850761 23595 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.850670 23559 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.885159 23563 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.851776 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.852419 23587 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.869227 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.790308 23589 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.870733 23568 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.871194 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:35.769939 23541 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.876232 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.887250 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.887137 21935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:35.890502 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:35.890818 22586 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:35.881769 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:35.892320 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:35.893379 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:35.894681 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:35.897421 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:35.798520 23581 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:35.886240 23595 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:35.901237 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.019153 22007 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:36.019631 22589 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:36.020361 21935 raft_consensus.cc:606] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:35.908107 21858 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:35.891654 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.021137 22008 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:36.022104 22588 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:35.998054 23578 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:35.916813 23592 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.020509 22009 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:35.920085 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.038932 22005 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:35.836931 23560 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.046638 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:35.993672 23540 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:35.937222 23576 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:35.996516 23442 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.002974 23502 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:35.934788 23598 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:35.882275 23574 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.056044 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.030458 23592 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.068202 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.050369 21853 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:36.061321 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 190213us
I20260709 13:05:35.993604 23597 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.046056 23563 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.082772 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.056195 23561 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.067449 22587 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:36.084194 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.088306 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.089071 23590 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:36.089498 23590 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.051412 23545 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.089887 23590 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.090171 23545 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.093377 23545 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.095917 23393 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.065475 23602 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.096477 23393 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.097047 23602 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.098646 23393 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.099213 23393 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.050485 23568 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.059621 23540 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:36.072204 21935 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:36.054986 23595 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.096771 23592 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.076884 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.098870 23590 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.077282 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:36.085983 23597 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.086089 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.082844 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:36.074293 23591 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.114075 23393 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.117288 23502 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:36.109316 23568 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.122611 23596 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:36.132546 23591 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.101490 23563 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.132741 21857 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:36.109966 23540 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.109278 23504 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.066987 23576 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.133812 23596 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.117730 23588 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:36.132342 23393 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.132489 23595 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:36.151666 23595 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.152078 23595 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.152114 23541 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.147464 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:36.147974 23603 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:36.153369 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.114933 23560 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:36.155457 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 70295.4us
I20260709 13:05:36.158989 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.159936 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:36.102669 23561 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.159646 21930 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:36.149473 23596 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.143517 23502 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:36.082795 23598 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.148978 23588 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.179787 23604 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:36.180603 23604 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.180931 23503 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.181224 23604 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.181730 23503 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.184450 23503 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.186009 23606 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:36.186717 23606 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.189695 23503 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.190430 23503 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.156692 23591 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.142803 23584 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.146528 23504 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.211109 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.228132 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.213498 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.214140 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.218935 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.210980 23597 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.193434 23588 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.197451 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.254982 23589 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.255635 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.256431 23600 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:36.257390 23590 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.277885 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.266204 21931 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:36.268530 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.258567 23599 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.210254 23581 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.281800 22002 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:36.216547 23560 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.211019 23606 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.278107 23600 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.260159 23563 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:36.293260 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 205017us
I20260709 13:05:36.294193 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.277032 23503 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.295622 22557 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.276827 23602 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.288908 22000 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:36.310603 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.311204 22002 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:36.301390 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:36.303184 23590 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:36.313570 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.305545 23574 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.308413 23589 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:36.301887 23599 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.311542 23602 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.315038 23590 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.327566 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.327802 23561 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.291410 23560 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:36.401721 23600 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.405977 23607 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:36.412292 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.172344 23603 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.319706 21851 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:36.411729 23568 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.412978 22002 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:36.423596 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.424407 22002 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:36.309303 23502 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:36.425364 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.426081 22002 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:36.427021 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.427985 23599 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.428263 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.417956 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.397882 23590 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.324276 23589 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.327840 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.430347 21860 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:36.410678 23596 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.428938 22002 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:36.432027 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:36.398592 23602 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.307495 23604 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.440112 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.441649 22002 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:36.406808 23607 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:36.447110 23604 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.448925 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:36.449867 22002 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:36.433540 23589 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.433157 23584 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.417760 23540 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:36.452324 23598 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.455950 23504 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:36.462469 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 327391us
W20260709 13:05:36.463376 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 157066us
I20260709 13:05:36.464454 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.466404 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:36.467581 23600 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:36.468160 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:36.468532 23605 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:36.472285 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:36.475203 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.475471 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.478444 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.479373 23602 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.480739 23560 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:36.469652 23600 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:36.481706 23600 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.482985 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.484020 22588 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:36.481724 23615 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:36.486842 23615 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:36.487347 23615 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.486724 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.491236 23596 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.486714 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:36.469403 23607 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:36.477715 23603 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.553098 23540 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:36.490597 23591 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:36.480082 23605 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:36.591140 23605 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:37.370325 23614 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.903s
I20260709 13:05:37.371898 23614 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:37.372470 23614 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:37.373584 23614 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.383949 23614 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:37.397717 23615 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.910s	user 0.013s	sys 0.000s
I20260709 13:05:37.528905 23615 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:37.368844 23605 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.719s	user 0.009s	sys 0.000s
I20260709 13:05:37.666864 23605 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.673740 23615 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:37.887565 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.408s	user 0.002s	sys 0.018s
W20260709 13:05:37.888060 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.002s	sys 0.018s
W20260709 13:05:37.888499 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.890833 22589 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.891258 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:37.892531 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:37.893090 23620 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:37.895253 23620 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:37.895697 23620 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:37.896762 23614 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:37.897286 23614 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:37.897787 23614 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.898609 23615 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:37.899206 23615 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.901382 23615 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:37.901993 23615 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:37.902530 23615 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.905611 23620 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.908334 23620 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:37.970796 23615 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:37.971899 23614 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.972548 23615 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:37.974952 23615 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:37.975961 23614 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:37.977891 23615 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:38.027359 23595 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.875s	user 0.012s	sys 0.012s
I20260709 13:05:38.027908 23595 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.030992 23595 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.044556 23595 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:38.045405 23595 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.048504 23595 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.057934 23561 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.058954 23561 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:38.059713 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 173) took 1915 ms (client timeout 1985 ms). Trace:
W20260709 13:05:38.060166 21851 rpcz_store.cc:269] 0709 13:05:36.144709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.431724 (+287015us) service_pool.cc:224] Handling call
0709 13:05:38.059626 (+1627902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.061252 23561 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.061719 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:38.062549 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 171) took 1924 ms (client timeout 2152 ms). Trace:
W20260709 13:05:38.062898 21851 rpcz_store.cc:269] 0709 13:05:36.138081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.060954 (+1922873us) service_pool.cc:224] Handling call
0709 13:05:38.062496 (+  1542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.063823 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:38.064594 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 169) took 1952 ms (client timeout 2213 ms). Trace:
W20260709 13:05:38.064958 21851 rpcz_store.cc:269] 0709 13:05:36.112273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.063554 (+1951281us) service_pool.cc:224] Handling call
0709 13:05:38.064544 (+   990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.065915 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:38.066504 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 172) took 1924 ms (client timeout 2184 ms). Trace:
W20260709 13:05:38.066833 21851 rpcz_store.cc:269] 0709 13:05:36.142569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.065641 (+1923072us) service_pool.cc:224] Handling call
0709 13:05:38.066463 (+   822us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.068426 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 174) took 1910 ms. Trace:
W20260709 13:05:38.069097 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.653s (SENT)
I20260709 13:05:38.068858 21851 rpcz_store.cc:276] 0709 13:05:36.157736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.067522 (+1909786us) service_pool.cc:224] Handling call
0709 13:05:38.067869 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.068078 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.068097 (+    19us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.068174 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.068218 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.068294 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.068388 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.070539 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1671) took 1668 ms. Trace:
I20260709 13:05:38.071377 22002 rpcz_store.cc:276] 0709 13:05:36.402033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.465380 (+ 63347us) service_pool.cc:224] Handling call
0709 13:05:38.070070 (+1604690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.070232 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.070257 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.070294 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:38.070327 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.070374 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.070465 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.071066 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 175) took 1897 ms. Trace:
I20260709 13:05:38.072796 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.073145 21851 rpcz_store.cc:276] 0709 13:05:36.173682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.070274 (+1896592us) service_pool.cc:224] Handling call
0709 13:05:38.070564 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.070727 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.070752 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.070795 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.070826 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.070899 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.071015 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.074397 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.075456 22002 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:38.077628 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.078429 22002 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.086210 22002 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:38.088371 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1673) took 1685 ms. Trace:
I20260709 13:05:38.088867 22002 rpcz_store.cc:276] 0709 13:05:36.402708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.087527 (+1684819us) service_pool.cc:224] Handling call
0709 13:05:38.087883 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.088058 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.088084 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.088126 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:38.088158 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.088212 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.088308 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:38.094944 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800064aa0 after lost signal to thread 20749
I20260709 13:05:38.101953 23623 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.102880 23623 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:38.106040 23623 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.108834 23623 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.117089 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:38.121932 21853 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 2.009s	user 0.010s	sys 0.022s
W20260709 13:05:38.121829 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1623) took 2046 ms (client timeout 1875 ms). Trace:
I20260709 13:05:38.122483 23623 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:38.122659 21852 rpcz_store.cc:269] 0709 13:05:36.075125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.078206 (+  3081us) service_pool.cc:224] Handling call
0709 13:05:38.116907 (+2038701us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaeee 0000000000375939 00007f3b7571e1e0 00007f3b757e70c4 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:38.121739 (+  4832us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5024640}
W20260709 13:05:38.119529 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 168) took 2018 ms (client timeout 1965 ms). Trace:
W20260709 13:05:38.123868 21860 rpcz_store.cc:269] 0709 13:05:36.100939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.101973 (+  1034us) service_pool.cc:224] Handling call
0709 13:05:38.119279 (+2017306us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540028c654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:38.119447 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637379328}
I20260709 13:05:38.122452 21853 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:38.124993 23623 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.125495 21858 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:38.127316 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 165) took 2052 ms. Trace:
I20260709 13:05:38.127417 23623 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.127676 21858 rpcz_store.cc:276] 0709 13:05:36.074981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.075200 (+   219us) service_pool.cc:224] Handling call
0709 13:05:38.125321 (+2050121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.125890 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.127098 (+  1208us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54002ba050. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:38.127236 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4714004352}
I20260709 13:05:38.128098 23623 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:38.128993 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1621) took 2290 ms (client timeout 2116 ms). Trace:
W20260709 13:05:38.129491 21853 rpcz_store.cc:269] 0709 13:05:35.838486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.841719 (+  3233us) service_pool.cc:224] Handling call
0709 13:05:38.128895 (+2287176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.130824 23623 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.139251 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.140406 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.140889 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:38.141547 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 394) took 1663 ms. Trace:
I20260709 13:05:38.141983 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.141952 22589 rpcz_store.cc:276] 0709 13:05:36.477843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.477966 (+   123us) service_pool.cc:224] Handling call
0709 13:05:36.478268 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.891370 (+1413102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.140603 (+249233us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54006630d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:38.140809 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3248090368}
I20260709 13:05:38.142680 21934 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:38.141052 21929 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.143538 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.144086 21934 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.144423 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.145694 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.146332 21929 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.147226 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.147881 21929 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:38.144846 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.149183 22589 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:38.148216 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.151685 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.152273 21934 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.153674 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.154345 21929 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.154383 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.155230 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.155373 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.155808 22593 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:38.156180 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.156543 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.156829 21926 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
W20260709 13:05:38.157750 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.158155 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.157786 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.159158 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.159655 21926 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:38.160535 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:38.160876 22593 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.161156 21854 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.161190 23623 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.161764 23623 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.162757 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.163903 23623 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.164947 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.163837 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:38.165611 21853 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.165622 22593 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:38.166977 23623 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.167874 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.165141 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.167699 23623 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:38.169556 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.170257 23623 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.181830 21926 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.183277 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.185663 23624 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.185797 21854 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:38.186218 23624 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:38.187572 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.188246 23624 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.188482 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:38.171983 23585 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:38.209455 23585 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:38.210896 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6540 after lost signal to thread 21792
I20260709 13:05:38.212404 23585 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.219874 23576 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.173321 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 513) took 2391 ms. Trace:
W20260709 13:05:38.227582 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.739s	user 0.026s	sys 0.047s
W20260709 13:05:38.174872 22558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.621s	user 0.014s	sys 0.005s
W20260709 13:05:38.228096 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.740s	user 0.026s	sys 0.047s
W20260709 13:05:38.228415 22558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.677s	user 0.015s	sys 0.006s
I20260709 13:05:38.155793 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.229681 21927 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:38.230669 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.231905 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.176666 21853 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:38.177853 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.234978 21929 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:38.235769 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.236522 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.236869 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.237149 22590 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:38.237588 21929 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:38.237952 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:38.238462 22590 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.188429 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.189857 23534 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.190376 23624 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.240258 23534 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:38.240361 23624 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.241317 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.242394 23534 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.242488 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:36.570683 23591 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.243216 21927 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:38.243382 23591 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.244115 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.247395 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.227744 22592 rpcz_store.cc:276] 0709 13:05:35.782286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.172517 (+2390231us) service_pool.cc:224] Handling call
0709 13:05:38.172816 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.172979 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.173038 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.173078 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:38.173119 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.173169 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.173270 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.248804 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.249354 22592 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:38.249794 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.250135 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:38.250335 22002 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.250649 22592 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:38.195715 21824 connection.cc:256] RPC call timeout handler was delayed by -1.981626303999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:05:38.184525 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:38.253358 21824 connection.cc:256] RPC call timeout handler was delayed by -1.7276174979999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
I20260709 13:05:38.253451 21926 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:38.253708 21824 connection.cc:256] RPC call timeout handler was delayed by -1.7151515059999838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:05:38.254002 21824 connection.cc:256] RPC call timeout handler was delayed by -1.673546981999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 13:05:38.254249 21824 connection.cc:256] RPC call timeout handler was delayed by -1.5570801510000365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 13:05:38.254509 21824 connection.cc:256] RPC call timeout handler was delayed by -1.543349371999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
I20260709 13:05:38.254570 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.178544 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.255395 21926 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 2: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:38.255318 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:38.256385 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.242554 23624 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.257529 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1665) took 1895 ms. Trace:
I20260709 13:05:38.258206 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.259375 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.259826 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.259928 21926 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:38.260172 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:38.260424 21935 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:38.238345 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.256457 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.190657 22003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:38.255241 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 234) took 2606 ms. Trace:
I20260709 13:05:38.261055 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:38.262552 23587 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.266185 23587 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.266544 23587 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.267388 22002 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:38.268395 23561 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.268949 23561 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:38.265661 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.269522 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.269853 22003 raft_consensus.cc:606] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:38.270283 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1655) took 2007 ms (client timeout 2024 ms). Trace:
I20260709 13:05:38.269946 23561 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:38.270664 22001 rpcz_store.cc:269] 0709 13:05:36.263270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.269216 (+2005946us) service_pool.cc:224] Handling call
0709 13:05:38.270230 (+  1014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.270969 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:38.254813 21824 connection.cc:256] RPC call timeout handler was delayed by -1.4839263039999651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:05:38.275847 21824 connection.cc:256] RPC call timeout handler was delayed by -1.3745659679999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 13:05:38.276254 21824 connection.cc:256] RPC call timeout handler was delayed by -1.3567504089999147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 13:05:38.260845 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:38.277791 22614 debug-util.cc:398] Leaking SignalData structure 0x7b080004b7a0 after lost signal to thread 21884
W20260709 13:05:38.278131 22614 debug-util.cc:398] Leaking SignalData structure 0x7b0800333760 after lost signal to thread 21885
I20260709 13:05:38.277550 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:38.286350 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08001ab780 after lost signal to thread 21959
W20260709 13:05:38.286717 22614 debug-util.cc:398] Leaking SignalData structure 0x7b0800225e00 after lost signal to thread 21960
W20260709 13:05:38.260922 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 241) took 2138 ms (client timeout 1653 ms). Trace:
I20260709 13:05:38.288152 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:38.288144 21935 rpcz_store.cc:269] 0709 13:05:36.122817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.123067 (+   250us) service_pool.cc:224] Handling call
0709 13:05:38.259693 (+2136626us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:38.260844 (+  1151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35337088}
W20260709 13:05:38.290195 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 81228.6us
I20260709 13:05:38.291301 23629 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.291517 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:38.170312 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:38.292317 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:38.265259 21928 rpcz_store.cc:276] 0709 13:05:35.648343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.254512 (+2606169us) service_pool.cc:224] Handling call
0709 13:05:38.254787 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.254951 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.254975 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.255016 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.255049 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.255097 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.255190 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.294476 23630 raft_consensus.cc:2764] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:05:38.288581 22614 debug-util.cc:398] Leaking SignalData structure 0x7b080030d740 after lost signal to thread 22033
W20260709 13:05:38.309738 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08003435c0 after lost signal to thread 22034
I20260709 13:05:38.309912 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:38.311291 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.324169 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:38.332954 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.334794 23628 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.335276 23628 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.335672 23628 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.257929 22000 rpcz_store.cc:276] 0709 13:05:36.361703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.453266 (+ 91563us) service_pool.cc:224] Handling call
0709 13:05:36.468208 (+ 14942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.257192 (+1788984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.257231 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.257274 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:38.257309 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.257370 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.257474 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.317606 23615 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.338913 23615 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:38.339900 23615 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.295550 23629 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.344743 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:38.346935 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.347191 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.346971 23629 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.358368 23605 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.349860 23534 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:38.361654 23534 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:38.362612 23534 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.364108 21930 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:38.365049 22592 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:38.353989 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5ca0 after lost signal to thread 22617
I20260709 13:05:38.381769 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.383263 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.383594 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.386883 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:38.381203 22614 debug-util.cc:398] Leaking SignalData structure 0x7b0800314140 after lost signal to thread 22618
I20260709 13:05:38.393308 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.388983 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.390605 22590 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:38.375212 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.397938 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.400442 22000 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.403513 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.406457 23568 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.406991 23568 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.407411 23568 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.410790 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.416965 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.419343 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.421972 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:38.432822 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 244437us
I20260709 13:05:38.435691 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:38.444901 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:38.448629 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.448551 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.451287 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:38.518213 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82615.7us
I20260709 13:05:38.519058 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:38.520392 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:38.521138 21934 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:38.453249 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:38.522339 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 69172.3us
W20260709 13:05:38.395795 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002eaf40 after lost signal to thread 23494
I20260709 13:05:38.530058 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.546373 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.548403 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.548681 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.548895 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.549093 22003 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:38.549489 22590 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:38.550024 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:38.556324 23590 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 2.117s	user 0.016s	sys 0.010s
I20260709 13:05:38.556748 23590 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.557288 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.557973 22009 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:38.559363 23590 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.529469 23585 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:05:38.392436 23591 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.562947 23591 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:38.563834 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.565068 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.565660 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.566066 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.568152 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1417) took 5244 ms. Trace:
I20260709 13:05:38.568522 21931 rpcz_store.cc:276] 0709 13:05:33.323901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.564708 (+5240807us) service_pool.cc:224] Handling call
0709 13:05:38.567672 (+  2964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.567848 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.567889 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.567928 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:38.567957 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.568006 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.568101 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:38.575784 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.576951 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.577952 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.578886 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.580384 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.581554 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.579849 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:38.532065 22614 debug-util.cc:398] Leaking SignalData structure 0x7b0800328fc0 after lost signal to thread 23514
I20260709 13:05:38.572397 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:38.532213 23624 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:38.590518 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 299047us
I20260709 13:05:38.594362 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.597067 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.598480 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.602478 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.610483 22586 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:38.611789 22002 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:38.532465 23581 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:05:38.632367 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.586596 23590 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:38.644954 22590 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:38.591740 23624 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.628455 23629 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:38.654743 23629 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:38.655372 23629 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.655743 23629 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:38.656105 23629 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.661296 22001 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:05:38.673482 23600 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 2.191s	user 0.017s	sys 0.012s
I20260709 13:05:38.673906 23600 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.676105 23600 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.676653 22591 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:38.677481 23626 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.677888 23626 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:38.678238 23626 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:38.678970 22591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 399) took 2196 ms (client timeout 1724 ms). Trace:
I20260709 13:05:38.680548 22589 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:38.679744 22591 rpcz_store.cc:269] 0709 13:05:36.482568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.482844 (+   276us) service_pool.cc:224] Handling call
0709 13:05:38.678726 (+2195882us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54004c22d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:38.678876 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5020222208}
I20260709 13:05:38.650210 23624 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.649178 23630 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.692186 23625 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:05:38.692344 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 171998us
I20260709 13:05:38.692642 23625 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:38.693171 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.693575 23625 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.694271 23633 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:38.694877 23625 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.695333 23633 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.695712 23633 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.694417 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:38.696552 22590 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:38.670324 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 105142us
I20260709 13:05:38.697029 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:38.706182 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.711606 21932 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:38.691468 23630 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:38.712023 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 81763.8us
I20260709 13:05:38.712365 23630 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:38.712914 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44428.6us
I20260709 13:05:38.671633 23587 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.696089 23633 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.721032 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.727859 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:38.728641 23634 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:38.726971 23534 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:38.729094 23634 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:38.737504 23580 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.761400 23580 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.742662 23624 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.761890 23580 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.763780 23585 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:38.765512 23635 raft_consensus.cc:1064] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:38.767973 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1603) took 5421 ms. Trace:
I20260709 13:05:38.768148 23628 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:38.753876 23589 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 2.299s	user 0.023s	sys 0.003s
I20260709 13:05:38.768364 21859 rpcz_store.cc:276] 0709 13:05:33.346507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.347547 (+  1040us) service_pool.cc:224] Handling call
0709 13:05:35.727271 (+2379724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.882731 (+155460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.882770 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.882806 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:35.882840 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.767625 (+2884785us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000fbc54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:38.767677 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.767789 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357105536}
W20260709 13:05:38.739867 23534 raft_consensus.cc:1068] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Service unavailable: leader transfer in progress
W20260709 13:05:38.771128 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 201814us
I20260709 13:05:38.789484 23589 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.731238 23639 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 13:05:38.847119 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 119379us
I20260709 13:05:38.848042 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 256) took 2956 ms. Trace:
I20260709 13:05:38.848490 22004 rpcz_store.cc:276] 0709 13:05:35.879659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.879781 (+   122us) service_pool.cc:224] Handling call
0709 13:05:35.880147 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.049388 (+169241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.049416 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.049457 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:36.049487 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.773071 (+2723584us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400437bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:38.836194 (+ 63123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.836335 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388772608}
I20260709 13:05:38.850054 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.854063 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:38.860492 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.861866 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:38.857082 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:38.861467 23585 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.862911 22585 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.863088 23585 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.862548 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:38.863380 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.863543 23585 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:38.866366 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 160263us
I20260709 13:05:38.858567 23587 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.868258 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:38.869174 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.860081 23628 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:38.879815 23580 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:38.881835 23588 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 2.602s	user 0.021s	sys 0.004s
I20260709 13:05:38.882221 23588 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.872134 23567 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.876595 22002 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:38.887475 22002 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:38.875602 23635 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.877538 23624 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.882750 23629 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.884473 22590 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.888212 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.870909 23643 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.908341 23643 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:38.908468 22000 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:38.908771 23643 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:38.909188 23502 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.417s	user 0.011s	sys 0.017s
I20260709 13:05:38.898900 23641 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.900657 22001 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:38.901219 23581 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:38.902667 23503 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.858980 23568 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.907092 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 242) took 2771 ms. Trace:
I20260709 13:05:38.868502 23634 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.910471 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:38.911054 23502 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.419s	user 0.011s	sys 0.017s
I20260709 13:05:38.897182 23567 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:38.916550 23587 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:38.917102 23641 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.918097 23580 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.919727 23626 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.906829 23504 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.924031 23534 raft_consensus.cc:993] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:38.922819 21932 rpcz_store.cc:276] 0709 13:05:36.135613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.135765 (+   152us) service_pool.cc:224] Handling call
0709 13:05:38.362184 (+2226419us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:38.362468 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.720756 (+358288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.906382 (+185626us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540017e154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:38.906995 (+   613us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806986240}
I20260709 13:05:38.928571 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:38.929492 22593 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:38.938090 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:38.941661 21926 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:38.941882 23626 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.942755 22584 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:38.922789 23503 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.945966 23503 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.932420 23588 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:38.946326 23634 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.932438 23568 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.920060 22002 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:38.937182 23641 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.927697 23393 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.935730 23567 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.942770 23626 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.945182 23570 raft_consensus.cc:1081] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:38.940335 23504 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.949066 22002 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.948223 23393 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:38.950026 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.954027 22587 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:38.945312 23612 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:38.929908 23615 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 13:05:38.956808 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.955701 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46467us
I20260709 13:05:38.952646 23504 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.980201 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:38.981194 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:38.989344 22004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:05:38.990240 22004 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.990701 22004 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.000663 23634 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.127235 23600 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.950526 23626 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:39.172659 22004 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:39.175513 22004 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.002136 23588 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:05:38.949826 23630 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.023065 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:38.982793 23629 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:38.982139 23643 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:38.952135 23621 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:39.079905 23596 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:39.011936 23534 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.127143 23589 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.184383 22003 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.056329 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:39.083184 23633 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.050943 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:38.955070 23612 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.097957 22007 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.091977 23585 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.098680 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.096318 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 522) took 3243 ms. Trace:
I20260709 13:05:39.024031 22589 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:39.056692 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.201934 23503 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.026264 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.032279 21753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 303) took 6022 ms. Trace:
I20260709 13:05:39.203382 23633 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.203478 21753 rpcz_store.cc:276] 0709 13:05:33.009896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.010052 (+   156us) service_pool.cc:224] Handling call
0709 13:05:39.032189 (+6022137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3334965,"reported_tablets":3,"spinlock_wait_cycles":36608}
I20260709 13:05:39.204233 23503 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.049418 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:38.951788 23393 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.001341 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:39.208174 22001 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:39.208233 22590 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:05:39.169291 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 442037us
I20260709 13:05:39.210533 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:39.169531 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 241049us
I20260709 13:05:39.169804 23634 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:39.183738 23596 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:05:39.113338 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 132234us
I20260709 13:05:39.178145 23626 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.182161 21789 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2540 after lost signal to thread 21792
I20260709 13:05:39.213969 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:05:39.215763 22008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1653) took 2981 ms (client timeout 2218 ms). Trace:
I20260709 13:05:39.224519 21932 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:39.235126 21932 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.228080 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:39.231133 23646 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:39.234988 22008 rpcz_store.cc:269] 0709 13:05:36.219115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.219350 (+   235us) service_pool.cc:224] Handling call
0709 13:05:39.050737 (+2831387us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:39.200392 (+149655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3471104}
I20260709 13:05:39.243428 21932 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.246696 23585 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.224594 23634 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.191550 23588 consensus_queue.cc:1243] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:39.250499 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:39.220058 23596 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:39.255771 23596 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.257115 23596 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.255934 23595 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:39.199906 22585 rpcz_store.cc:276] 0709 13:05:35.852469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.892308 (+ 39839us) service_pool.cc:224] Handling call
0709 13:05:36.481221 (+588913us) spinlock_profiling.cc:243] Waited 8.72 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:36.481416 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.863335 (+2381919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.096005 (+232670us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b54004c2f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:39.096213 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5479689344}
I20260709 13:05:39.263478 23630 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:39.269541 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.248238 23503 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.272754 21932 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.233122 23570 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:05:39.269327 23623 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.274014 23612 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.303357 23639 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:39.236786 23582 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:05:39.283548 21929 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.284945 23643 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.285794 21928 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.313664 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.274473 23503 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:39.286664 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.288213 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.319120 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.320210 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.320734 21934 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:39.289054 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.319139 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.290012 23625 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.247850 23633 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:05:39.314064 23584 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:39.296424 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 400) took 1037 ms. Trace:
I20260709 13:05:39.297499 21754 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 187) took 6068 ms. Trace:
I20260709 13:05:39.268863 23504 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.234198 23580 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:05:39.300665 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.282681 23588 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:05:39.307448 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.304164 23639 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.286547 21930 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.315774 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.329457 21754 rpcz_store.cc:276] 0709 13:05:33.229319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.229739 (+   420us) service_pool.cc:224] Handling call
0709 13:05:39.297438 (+6067699us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3400849,"reported_tablets":1}
I20260709 13:05:39.252009 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.331221 21935 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:39.325192 23625 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.321039 21934 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.325307 23503 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.328816 22593 rpcz_store.cc:276] 0709 13:05:38.259255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.323918 (+ 64663us) service_pool.cc:224] Handling call
0709 13:05:39.296203 (+972285us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54006a04d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:39.296357 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386714880}
I20260709 13:05:39.323696 22588 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:39.362922 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.339725 22591 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:39.334513 22592 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:39.344022 23504 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.347425 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.347707 21932 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:39.350708 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:39.352492 23623 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.353794 22009 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:39.360029 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.360675 23580 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.327013 23584 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.372630 22001 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:39.361770 23639 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.363400 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 430) took 1191 ms. Trace:
W20260709 13:05:39.361795 22006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1654) took 3142 ms (client timeout 1824 ms). Trace:
W20260709 13:05:39.339087 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78331.5us
I20260709 13:05:39.365166 22592 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.363415 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 167) took 3279 ms. Trace:
I20260709 13:05:39.366070 22587 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:39.316512 23630 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:05:39.382231 22588 rpcz_store.cc:276] 0709 13:05:38.180956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.098447 (+917491us) service_pool.cc:224] Handling call
0709 13:05:39.363348 (+264901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:39.359529 21934 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:39.358273 21789 debug-util.cc:398] Leaking SignalData structure 0x7b080037e620 after lost signal to thread 21884
I20260709 13:05:39.368444 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:39.395251 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.402743 21934 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.382624 22006 rpcz_store.cc:269] 0709 13:05:36.219390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.361609 (+3142219us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:39.361718 (+   109us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:05:39.407068 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.408288 23637 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
W20260709 13:05:39.466508 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64148.7us
I20260709 13:05:39.469107 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:39.372062 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:39.472270 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 103909us
I20260709 13:05:39.474175 22006 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:39.383605 21856 rpcz_store.cc:276] 0709 13:05:36.084228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.084369 (+   141us) service_pool.cc:224] Handling call
0709 13:05:36.084623 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.156917 (+ 72294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.156943 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.156981 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:36.157031 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.363209 (+3206178us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b540010b654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:39.363250 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.363360 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165863168}
W20260709 13:05:39.476199 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.379281 22585 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.376116 23561 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:05:39.492224 23631 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.492026 23615 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:39.494526 23570 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:39.526409 23570 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.372351 23580 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.466114 23641 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.500159 23534 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:39.532696 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 158336us
W20260709 13:05:39.533975 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 65010.5us
I20260709 13:05:39.535485 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:39.470917 21789 debug-util.cc:398] Leaking SignalData structure 0x7b080037e560 after lost signal to thread 21885
I20260709 13:05:39.477367 23589 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.523764 23561 consensus_queue.cc:1243] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:39.480244 23625 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.476568 23635 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:05:39.540509 23635 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.541791 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.533107 23570 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:39.408948 23502 consensus_queue.cc:1175] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 2 } }
I20260709 13:05:39.545966 21933 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:39.374877 23582 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.156s
W20260709 13:05:39.546895 21789 debug-util.cc:398] Leaking SignalData structure 0x7b080037de20 after lost signal to thread 21959
W20260709 13:05:39.549196 23567 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.005s
I20260709 13:05:39.559124 23641 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.562299 21757 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 280) took 6782 ms. Trace:
I20260709 13:05:39.562734 21757 rpcz_store.cc:276] 0709 13:05:32.779474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.779616 (+   142us) service_pool.cc:224] Handling call
0709 13:05:39.562124 (+6782508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3600696,"reported_tablets":3}
I20260709 13:05:39.565018 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1650) took 3355 ms. Trace:
I20260709 13:05:39.560540 23567 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.565512 22007 rpcz_store.cc:276] 0709 13:05:36.209746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.217802 (+  8056us) service_pool.cc:224] Handling call
0709 13:05:36.257246 (+ 39444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.192355 (+2935109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.564701 (+372346us) spinlock_profiling.cc:243] Waited 2.84 s on lock 0x7b5400168854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:39.564923 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6532633984}
I20260709 13:05:39.566185 22005 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.570776 23580 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.572359 23580 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:39.556598 21789 debug-util.cc:398] Leaking SignalData structure 0x7b080037ddc0 after lost signal to thread 21960
I20260709 13:05:39.578136 23503 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:39.579922 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.581583 23504 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:39.584383 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.595283 22008 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.576730 23567 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.596036 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:39.598299 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.600695 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 242079us
I20260709 13:05:39.601998 23649 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.602099 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.602564 23649 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.603636 23649 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.605391 22007 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.605937 23645 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.606170 22592 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.607856 22592 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.610049 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.613893 22006 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.622139 23615 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:39.618541 23645 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.587198 23626 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:39.619268 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388140 after lost signal to thread 22033
I20260709 13:05:39.634143 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.594290 23393 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.588191 23584 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.638247 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:39.619570 23567 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.610951 23621 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:05:39.611220 23589 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:05:39.644596 21823 connection.cc:256] RPC call timeout handler was delayed by -1.1251050650000707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:05:39.645515 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 1.36789e+06us
W20260709 13:05:39.671593 21823 connection.cc:256] RPC call timeout handler was delayed by -1.0189050650000127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:05:39.626586 23649 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:39.672627 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.672714 22003 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:39.637655 23568 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:39.633352 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388160 after lost signal to thread 22034
W20260709 13:05:39.674686 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388a20 after lost signal to thread 22617
W20260709 13:05:39.675019 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388a40 after lost signal to thread 22618
W20260709 13:05:39.675714 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388f00 after lost signal to thread 23593
I20260709 13:05:39.681336 23612 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.681841 23612 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.682248 23612 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.685303 23601 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:39.685817 23601 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.687119 23604 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:39.687668 23604 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.689728 23604 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.690222 23604 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.690971 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.692025 23602 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:39.692579 23602 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.693302 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.694008 21856 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:39.694743 23602 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.695294 23602 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.695616 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.695768 23602 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:39.696867 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.690653 23604 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.699396 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.700321 23592 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.700766 23592 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:39.701175 23592 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.701290 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.700466 23604 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.703689 21857 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:39.705591 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71524.3us
I20260709 13:05:39.635891 23503 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.704605 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.709645 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.712915 23657 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.713433 23657 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.713529 23646 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:39.714015 23646 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:39.707281 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.716382 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.719123 21934 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:39.669368 23576 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:39.720328 23576 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.722473 23576 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.724102 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:39.725592 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 190163us
I20260709 13:05:39.727312 21857 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:39.723652 23600 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 13:05:39.730738 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.732499 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.735484 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.737020 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:39.650921 23634 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.738864 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.725801 23579 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.725488 23561 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.741803 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.732210 22592 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:05:39.759766 23656 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:05:39.730156 23578 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.720484 23646 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.761344 23610 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.725248 23633 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:05:39.742894 21857 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:39.764715 23631 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:39.733292 23540 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:39.811225 23540 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.811721 23540 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.769405 21929 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:39.814518 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.815277 21857 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:39.815369 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.816012 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.816315 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.816665 23633 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.817119 23633 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:39.797751 23503 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:39.818149 23503 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:39.771950 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.779743 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.769909 23393 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.739066 23600 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.788519 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.771054 23601 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.793931 23621 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:39.806210 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1697) took 6474 ms. Trace:
I20260709 13:05:39.826634 21933 rpcz_store.cc:276] 0709 13:05:33.322784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.097266 (+5774482us) service_pool.cc:224] Handling call
0709 13:05:39.097572 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.548432 (+450860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.797135 (+248703us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b5400354dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:39.797360 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028852992}
W20260709 13:05:39.810592 23594 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:05:39.810410 23610 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.818041 23633 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.728261 23604 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.786155 23561 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.824462 21856 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:39.787254 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.819401 23587 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:05:39.854526 23646 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.862830 23587 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.863281 23587 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.865978 23634 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.866110 23570 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.866278 23592 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.866596 23570 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.868404 23592 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.868631 23570 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.869082 22586 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.869374 23633 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.870544 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.871698 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.871905 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.872917 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.873386 23626 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:39.873544 21851 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:39.840111 23610 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.874466 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:39.875641 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.876214 21851 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 1.
I20260709 13:05:39.830453 23577 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:39.877152 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.877822 21851 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:39.881078 23602 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:39.881733 23659 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:05:39.831858 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.797884 23578 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.882493 23578 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.843030 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:39.844903 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.884948 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.877245 23577 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:39.822930 22585 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
W20260709 13:05:39.830986 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.886130 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 98945.3us
I20260709 13:05:39.890493 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:39.891537 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.864176 23587 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.889197 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.898021 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:39.893383 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:39.886632 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:39.785753 23579 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:39.894133 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 359988us
I20260709 13:05:39.841439 23604 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:39.907866 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:39.868983 23592 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:39.889988 23577 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.891577 23626 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.899924 23631 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.900611 23634 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:39.930594 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 291472us
I20260709 13:05:39.907945 23595 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.904749 23647 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:39.908267 23579 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.908018 23580 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:39.911723 23503 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:39.927752 23587 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.954646 23584 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.957502 23652 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.957945 23652 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.935216 23631 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:39.935755 23592 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:39.928704 23612 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.932152 23636 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:05:39.960929 23604 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.933826 23645 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:05:39.966995 23540 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.932826 23561 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.958856 23580 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.974129 23580 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.900357 23657 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.966848 23610 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.974404 23649 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.983383 23649 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:39.984249 23649 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.984726 23610 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:39.985417 23649 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.975353 23630 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.975124 23592 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.976459 23625 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.977159 23561 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.977847 23601 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:39.961526 23631 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:39.944510 23647 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:39.961373 23652 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:39.995805 23652 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.982549 23612 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.002148 23647 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:39.989322 23630 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:40.017818 23630 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.019116 23630 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.018174 23657 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.053073 21758 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 285) took 1901 ms. Trace:
I20260709 13:05:40.053828 21758 rpcz_store.cc:276] 0709 13:05:38.151146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.281729 (+130583us) service_pool.cc:224] Handling call
0709 13:05:40.052588 (+1770859us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1276851,"reported_tablets":5}
W20260709 13:05:40.057078 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 321017us
I20260709 13:05:40.057526 21853 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:39.888651 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.061795 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 232096us
W20260709 13:05:40.062335 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 331500us
I20260709 13:05:40.064139 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:39.908524 21860 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
W20260709 13:05:40.069221 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 176637us
W20260709 13:05:40.069496 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 144618us
I20260709 13:05:40.070314 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:40.071487 21856 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:40.066104 21857 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:40.076434 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 166198us
I20260709 13:05:40.078099 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 300) took 1052 ms. Trace:
I20260709 13:05:40.079694 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:40.081768 22592 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 2: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:40.082093 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:40.084461 21860 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:40.000553 23540 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.090526 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.066053 21854 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:40.093168 23540 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:40.078855 22004 rpcz_store.cc:276] 0709 13:05:39.025281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.182191 (+156910us) service_pool.cc:224] Handling call
0709 13:05:39.185880 (+  3689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.186317 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.186363 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.299281 (+112918us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:05:39.299659 (+   378us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.332012 (+ 32353us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4800130158. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:05:39.443938 (+111926us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:39.443981 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.444052 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.944206 (+500154us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b24002947b8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d720fb0 00007f3b70397488 00007f3b70397be8 00007f3b7036f398 00007f3b7036a76a 00007f3b70369bb6
0709 13:05:40.077368 (+133162us) raft_consensus.cc:1685] finished
0709 13:05:40.077430 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.077750 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":2834,"raft.run_wall_time_us":145200,"spinlock_wait_cycles":34070784,"thread_start_us":140563,"threads_started":1,"wal-append.queue_time_us":105428}
I20260709 13:05:40.090505 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.091209 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.090803 23578 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.095530 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.096515 21857 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.099205 23578 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.100591 23578 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:40.103515 23576 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.104092 23576 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:40.104305 23604 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:40.105206 23502 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:40.105638 23586 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:40.104728 23576 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.106331 23502 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.105247 23581 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:40.109681 21855 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:40.109336 23581 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.112082 23581 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.113755 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:40.106540 23586 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.117504 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.114123 23601 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.112124 23502 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.129330 23586 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.131170 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.131775 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 186793us
I20260709 13:05:40.133819 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:40.135242 21929 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:40.135839 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:40.136668 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.139389 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.146070 23610 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:40.146672 23610 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:40.146112 21930 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:40.155766 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.171470 23647 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.104552 23580 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:05:40.174523 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.181218 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 43482.4us
I20260709 13:05:40.182708 23652 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.183104 23577 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.183384 23652 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.184733 21852 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:40.185247 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.186654 21854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:05:40.187572 21854 raft_consensus.cc:606] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:40.192970 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49233.9us
W20260709 13:05:40.195456 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.205127 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.205651 23654 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:40.204849 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.219564 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.229352 23585 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:40.216643 23580 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.229950 23576 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.230195 23585 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.231794 23580 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.232194 23585 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.232474 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:40.235059 21852 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.236815 23636 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:40.237701 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.239096 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.239584 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.240908 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.206542 23654 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:40.246484 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.221817 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:40.245172 21931 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.250609 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:40.250805 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:40.251442 22587 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:40.251775 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:40.252885 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.216424 23577 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.255965 21857 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
W20260709 13:05:40.256501 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.253052 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.257187 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:40.258553 23652 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.259505 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.260633 21855 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:40.261430 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.263904 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:40.264076 23654 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.264530 23636 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.268921 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:40.269836 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 88797.6us
I20260709 13:05:40.270459 22587 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:40.273461 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.317821 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.318549 21858 raft_consensus.cc:2376] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2. Re-sending same reply.
I20260709 13:05:40.318858 21854 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.320317 21854 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.321858 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.322628 21860 raft_consensus.cc:2376] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3. Re-sending same reply.
I20260709 13:05:40.322687 22588 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:40.322795 21859 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.323580 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.323773 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.323995 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.325037 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.324803 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.237699 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:40.279659 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.280045 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.282872 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.284968 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.286300 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.284330 22001 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:40.294327 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.309962 22592 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:40.309795 21856 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:40.314739 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61070.6us
I20260709 13:05:40.275790 23604 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.326902 22003 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:05:40.329089 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 65271.1us
I20260709 13:05:40.324820 21859 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:40.328482 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:40.342562 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:40.343386 23576 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.336295 22006 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:40.336345 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.346568 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53396.3us
I20260709 13:05:40.343977 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.349016 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.349309 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.349265 22002 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:40.354377 22585 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:40.355827 22008 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:40.355595 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.356149 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.349957 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.357076 23654 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:40.357643 21859 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
W20260709 13:05:40.359021 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.351035 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.363919 22005 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:40.328410 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.365314 22003 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:40.360615 21931 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:40.362102 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.344704 23585 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:40.374704 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43874.3us
I20260709 13:05:40.361129 22000 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:40.374205 22009 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:40.248946 23610 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.382073 21859 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:40.383397 23610 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:40.385462 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.333827 23604 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:40.387005 23604 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.371527 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:40.364904 23675 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.167s	user 0.005s	sys 0.000s
I20260709 13:05:40.363560 23502 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:40.389163 21855 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:40.358353 23636 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:40.410813 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.388957 23604 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:40.415355 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.377868 23585 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:40.415897 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:40.423128 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.424480 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.429844 23636 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.432982 23615 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:40.433296 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 65323.8us
I20260709 13:05:40.433521 23615 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.433982 23615 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.435465 23577 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:40.435921 23577 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:40.436070 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.703s (SENT)
I20260709 13:05:40.436375 23577 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.396950 23502 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.437379 23502 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.436633 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:40.443894 21857 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.439031 23585 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.473080 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.498258 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.502135 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.490168 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.517390 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.521486 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.524514 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.522652 22586 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:40.537072 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73727.5us
I20260709 13:05:40.512365 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.542682 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 54947.1us
W20260709 13:05:40.538898 21898 leader_election.cc:336] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.557s (SENT)
W20260709 13:05:40.548974 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 112956us
I20260709 13:05:40.557930 21857 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:40.561208 23649 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.563766 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:40.564610 21933 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:40.545186 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.569402 21852 raft_consensus.cc:2376] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2. Re-sending same reply.
I20260709 13:05:40.557862 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:40.564105 23679 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:40.583976 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.593372 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:40.615782 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.615782 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.629967 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.633278 22592 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:40.633767 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 98162.5us
I20260709 13:05:40.634024 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:40.636345 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.622550 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.965s (SENT)
I20260709 13:05:40.639119 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:40.639904 21860 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:40.650446 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55382.1us
I20260709 13:05:40.654986 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:40.656844 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.713119 21854 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:40.714417 22593 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:40.681197 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.698565 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43709.2us
I20260709 13:05:40.678562 23540 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:05:40.662652 23630 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.722091 23604 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.722615 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:40.722553 23630 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.722579 23604 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:05:40.659176 23681 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:05:40.671226 23502 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.725193 23576 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:40.724344 23604 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.712476 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.726694 23604 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.726869 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:40.668792 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76486.9us
I20260709 13:05:40.727928 22593 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:40.686252 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.698622 23578 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:40.668007 23649 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.730995 21974 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:40.731980 23633 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:40.707762 23561 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.006s
I20260709 13:05:40.732451 23633 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.732918 23633 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.658557 23584 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.668278 23577 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:40.736258 23577 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.664176 23592 raft_consensus.cc:993] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
W20260709 13:05:40.658310 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 258289us
I20260709 13:05:40.738097 23592 raft_consensus.cc:1081] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:05:40.696381 21856 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:40.686772 23615 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.712195 23601 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:05:40.742301 23601 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.685418 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.742703 23601 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:40.744819 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.745240 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.746480 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:40.747124 22593 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.747655 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.731777 23578 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:40.749385 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.750020 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.689777 23647 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
W20260709 13:05:40.663538 23579 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.711s	user 0.012s	sys 0.000s
W20260709 13:05:40.746721 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.751752 23579 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.752910 21858 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:40.755820 22593 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:40.755911 23579 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.726498 23502 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.758479 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.761723 23615 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:40.761111 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.761956 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.764406 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.681562 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.769018 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.770908 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:40.773087 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.833279 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.836212 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.837266 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.814047 21859 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:40.815176 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.815541 23537 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:40.816512 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.816928 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.817137 23630 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.817770 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:40.817905 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 51963.8us
I20260709 13:05:40.818070 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.818729 21860 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 2: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:40.728705 22556 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.985s (SENT)
I20260709 13:05:40.707053 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:40.855261 22593 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:40.855419 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.855618 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.856436 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.856581 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.856798 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.857561 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.859283 23633 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.861443 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.862644 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.864380 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:40.864565 21852 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:40.865348 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:40.865824 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 137181us
I20260709 13:05:40.866679 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.867743 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.826129 21758 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.868821 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.868938 21758 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.828877 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:40.878048 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:40.811745 23502 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:40.879706 23586 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:40.880119 23586 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:40.881311 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:40.881119 23586 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:40.882162 23586 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.882411 23584 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:40.882361 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.882983 23584 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:40.883383 23584 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.830399 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:40.816553 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:40.834692 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:40.892055 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61757.6us
I20260709 13:05:40.711957 23585 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.837870 23610 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:40.839196 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.839608 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 98733.4us
I20260709 13:05:40.743804 23601 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.847043 21856 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:40.817771 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.869799 23537 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:40.870785 23577 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:40.871589 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.872682 23633 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.909750 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:40.893736 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.890650 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 183700us
I20260709 13:05:40.901455 21758 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 5, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 5 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:40.903107 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.902230 22009 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.901031 23580 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:40.915565 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:40.916193 22586 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.918426 21758 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 3, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:40.910641 23586 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:40.903553 23537 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:40.929044 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.937196 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.931324 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:40.921916 23610 consensus_queue.cc:1243] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:40.928568 23601 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.926479 23654 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:40.941679 22002 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:40.906674 23584 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.945900 22004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41328
I20260709 13:05:40.950666 22004 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:40.946331 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:40.949098 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.934909 23586 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:40.953351 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 136878us
I20260709 13:05:40.954711 22004 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.958890 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:40.956396 23654 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.965193 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:40.978576 21858 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.978993 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:40.957741 23540 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.979176 21858 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:05:40.988790 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.962634 23654 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.993393 23585 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:40.993162 21858 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:40.994510 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:40.994745 21858 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.003299 23592 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.004756 22009 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
W20260709 13:05:41.004998 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.009784 23585 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.011574 23654 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:41.012665 23654 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:41.014076 23582 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:41.014513 23582 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:41.015407 23582 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.016680 23582 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:41.004877 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 92670.5us
I20260709 13:05:41.021209 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.021592 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:41.022203 22593 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:41.022383 22591 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.987815 23610 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.023854 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:41.023403 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:41.028695 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 162267us
I20260709 13:05:41.035449 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.027628 23585 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.061736 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:41.063398 22007 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.065761 22009 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.053260 23561 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:41.034482 23580 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:41.053655 23585 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:41.070123 22008 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:41.070214 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.077706 22591 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:41.081640 22002 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:41.084789 23694 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:41.085016 23690 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:05:41.085312 23694 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.085752 23694 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.087944 23537 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.035432 23586 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.089015 23537 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.081700 22004 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.115134 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.115943 22001 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:41.067261 23592 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:05:41.123384 21858 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:41.083407 23561 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:41.135579 22004 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:41.130035 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72441.3us
I20260709 13:05:41.098021 23579 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.142112 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:41.126658 23584 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.123513 23615 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.143271 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:41.132349 23700 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:41.152815 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.155084 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.157068 21854 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:41.148535 23586 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.158341 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 88200.3us
I20260709 13:05:41.152145 23700 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:41.151476 23592 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.172556 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:41.172494 23702 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.173237 21853 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:41.174268 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:41.174738 23604 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:05:41.175753 22586 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:41.176366 23592 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.185266 23667 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.152485 23615 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.185763 23667 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.186685 23667 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.187247 22593 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.187418 23615 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.153852 23578 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:05:41.187974 23667 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.147585 23585 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.188588 23578 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.188822 23585 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.189076 23578 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.189990 23578 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.190284 23604 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:41.191392 23576 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.185381 23592 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.191745 23576 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.180791 23561 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.193109 23696 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.188462 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.178421 23580 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.193746 23696 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.192780 23576 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.195731 23576 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.182453 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.193179 21974 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.197878 22591 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:41.152040 23540 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:05:41.154520 23610 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:05:41.165822 23584 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.211834 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.211021 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.211670 23633 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.214915 23633 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.217550 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.221949 23695 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.222651 23695 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:41.223747 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:41.224143 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.223834 23695 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:41.224673 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.224591 22591 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:41.197326 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:41.228785 22556 leader_election.cc:336] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.845s (SENT)
I20260709 13:05:41.228022 21931 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:41.229460 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.231887 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.201140 23630 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.233160 22007 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 3. Ops: []
I20260709 13:05:41.198958 23586 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.234277 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.234474 22009 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.177927 23700 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.216898 23633 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:41.238848 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66735.4us
I20260709 13:05:41.245514 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.247112 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.247790 21926 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:41.247956 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:41.249137 23692 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:05:41.250001 23592 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:41.250808 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.251154 23592 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.251595 23592 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.249135 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.255647 22003 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.211351 23584 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.257797 21930 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:41.258957 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.263048 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.234864 23696 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:41.263510 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 53102.6us
I20260709 13:05:41.265374 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.265944 22591 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:41.266054 23610 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.267210 23610 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.268253 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.268579 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:41.269599 22001 raft_consensus.cc:2368] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:41.270942 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.271968 22008 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:41.272733 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.252663 23615 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.248872 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.277313 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:41.277652 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.252064 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.279124 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.282522 22002 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: [1.3-1.3]
W20260709 13:05:41.283579 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62474.7us
I20260709 13:05:41.282353 22007 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:41.260808 21927 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
W20260709 13:05:41.279582 23704 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:05:41.277869 23610 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.301851 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.303704 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.305197 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:41.302590 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45874.8us
I20260709 13:05:41.306466 23714 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.307134 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:41.308732 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 138400us
I20260709 13:05:41.309484 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.307518 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.279291 23592 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.307813 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:41.312311 22556 leader_election.cc:336] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.876s (SENT)
I20260709 13:05:41.314098 23706 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.313975 23703 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:05:41.315060 23709 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.315510 23709 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.315869 23709 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.315958 23663 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.302138 23610 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.312016 21932 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:41.333968 23584 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.334113 23694 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.336342 23577 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.338928 23706 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:41.333112 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.345081 23592 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.339694 23701 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:41.370769 21899 leader_election.cc:400] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:41.340776 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.385187 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.385440 21899 leader_election.cc:403] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:05:41.341374 23712 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:05:41.341748 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:41.346485 23707 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.312646 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:41.350356 22556 leader_election.cc:336] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.152s (SENT)
I20260709 13:05:41.360306 21935 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:41.296980 23580 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.339293 23645 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.371549 23589 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.372488 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:41.370729 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61299.7us
I20260709 13:05:41.393604 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.393705 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:41.386098 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45413us
I20260709 13:05:41.311100 23714 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:05:41.387936 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 112309us
I20260709 13:05:41.312259 23654 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.393635 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.397256 21930 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.398272 23706 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.398653 21927 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.398895 23717 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.401166 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.385497 23615 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.391655 23696 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.418754 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.422348 23580 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.471923 21928 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:05:41.478303 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.478989 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 97113.2us
W20260709 13:05:41.479102 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 84084us
W20260709 13:05:41.481576 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58562.1us
W20260709 13:05:41.467756 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117438us
I20260709 13:05:41.399467 23589 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.417459 21929 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 2: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:41.510008 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.373536 23610 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.476835 23700 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.508697 23615 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.400018 23645 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.534981 23615 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.536110 23645 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.536677 23540 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.536851 22008 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:41.537233 23540 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.537461 23610 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.538417 23540 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.539968 23694 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.547160 23708 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:41.548933 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:41.550139 23695 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.550613 23695 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.551019 23695 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.513273 23715 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.553449 23715 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.554942 23709 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.525072 23701 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.556881 23715 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.558888 23709 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.527295 23604 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.529148 23719 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.585450 23604 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.586644 23719 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.587177 23645 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.587345 23604 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.587862 23645 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.588689 23604 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.555491 23586 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.564437 23577 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:05:41.556599 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.589808 23577 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.590308 23645 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.565910 23713 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.514920 23589 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.569330 23699 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.569713 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:41.570415 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.571566 21926 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:41.571801 23578 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.599903 23667 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.602295 23701 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:41.602530 23695 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.605244 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:41.606034 21933 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 2: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:05:41.608546 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.573685 22007 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:41.609373 23540 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.473130 23714 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.576135 23580 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.576697 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.579200 23706 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
W20260709 13:05:41.514037 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 108401us
I20260709 13:05:41.421474 23630 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:41.618278 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.618935 22593 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:41.619369 22592 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.619928 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:41.599673 23699 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.612447 23713 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.623909 23713 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.617148 23630 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.625234 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.625834 22593 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:41.628402 22590 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:41.471732 23702 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.399636 23707 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:41.590780 23577 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.410531 23654 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:41.598769 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80008.3us
I20260709 13:05:41.600118 22008 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
W20260709 13:05:41.600728 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 90831.3us
I20260709 13:05:41.615490 22002 raft_consensus.cc:2368] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:41.615651 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.615906 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.516209 23720 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 13:05:41.612423 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:41.630789 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.643105 22009 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:41.633723 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:41.633773 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.635133 23719 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.644938 23694 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:41.645397 23694 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.645772 23694 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.646533 22592 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:41.582472 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.184s (SENT)
I20260709 13:05:41.420710 23717 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:41.651374 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 68924.3us
I20260709 13:05:41.651705 22006 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:05:41.651984 22556 leader_election.cc:336] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.213s (SENT)
I20260709 13:05:41.652711 23717 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.654330 23717 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.654868 23717 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.656797 23717 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:41.640425 22587 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:41.660943 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.638824 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.660744 23615 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.664166 21930 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.664760 22584 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:41.664558 23615 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.667306 23615 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.667831 23586 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.636953 23700 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.668883 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.669426 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.669919 23713 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.670380 23702 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.670606 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.670876 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.669687 23700 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.671591 23713 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.671765 23702 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.686311 21935 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:41.672233 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.675437 23717 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:41.665715 23694 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.676118 22584 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:41.692236 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:41.688697 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:41.690770 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51997us
I20260709 13:05:41.693388 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.693679 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.697741 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.698915 21899 leader_election.cc:400] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
W20260709 13:05:41.705196 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.703012 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.706941 22592 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
W20260709 13:05:41.710621 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.688896 23712 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.713546 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:41.697225 23700 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.717692 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:41.689245 23586 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.719631 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:41.719646 22590 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.700379 23714 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.716137 23537 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:05:41.713626 23716 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.729465 23577 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.729743 23604 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.753113 22592 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 1 LEARNER]: Advancing to term 2
I20260709 13:05:41.754272 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.730175 23502 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.730752 23636 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:41.763294 23694 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.731345 23585 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.764444 23585 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:41.765125 22590 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:41.765462 23585 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.723723 23582 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:41.768257 23582 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.768694 23582 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:41.775321 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.744139 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.776962 22587 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:41.777908 23580 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.778337 23580 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:41.779289 23580 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.718788 23703 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.745641 22002 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:05:41.761416 23706 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:41.780441 23706 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.780836 23706 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.782119 23604 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:41.763425 23636 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:41.783761 23636 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.704196 23713 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.785965 23636 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.786310 23707 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:41.735519 23586 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.793159 23706 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.795924 23586 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.797945 23706 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:41.801277 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 107954us
I20260709 13:05:41.803150 23582 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.803781 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.805028 23720 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.806675 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:41.815074 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.815778 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.819080 23709 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.819276 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.819998 22587 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
W20260709 13:05:41.821823 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 142845us
I20260709 13:05:41.825991 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:41.829264 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 92411.9us
I20260709 13:05:41.810591 23654 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.836566 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.817299 23720 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.834065 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.841792 23578 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.842387 23578 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:41.843372 23578 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:41.842396 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.845873 23717 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.847630 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.848364 23717 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.849730 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.827557 23540 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:41.862862 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.867169 22004 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:41.871549 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.872498 22001 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:05:41.873271 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47403.4us
I20260709 13:05:41.874189 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.874899 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.840612 23715 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:41.875145 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 150940us
I20260709 13:05:41.872066 23589 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:41.884325 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:41.841151 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.863837 23716 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:41.887358 22009 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:41.888229 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.863653 23582 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:41.868657 23615 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.890260 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.893497 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.867892 23720 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:41.863978 23717 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.896250 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.901412 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:41.895519 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 72667.5us
I20260709 13:05:41.887202 22590 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:41.905228 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.905205 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:41.907011 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.907032 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:41.959587 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:41.996484 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.016842 22008 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:42.021903 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.015895 22007 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:42.024802 22005 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:42.017413 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:41.960498 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 110857us
I20260709 13:05:42.026403 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:41.899748 23647 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.027452 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:41.892359 23716 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:42.019218 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 112292us
I20260709 13:05:42.029850 22003 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.021762 22006 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:41.957541 23719 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.031769 22001 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:41.899513 22585 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:41.898947 23540 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:42.033285 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:41.959846 23732 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.957381 23604 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.903499 23615 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.960111 23723 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:42.035128 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:42.071157 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169812us
I20260709 13:05:42.039357 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:41.892730 23601 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.141s
I20260709 13:05:42.028388 23709 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.067960 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.068570 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.085101 22005 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:42.085206 23717 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.028872 23647 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.028247 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:41.901407 23579 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 13:05:42.086030 23647 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.086431 23579 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:42.086843 23579 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:42.087690 23579 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.088056 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:42.088981 23579 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.089169 22586 raft_consensus.cc:2368] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
W20260709 13:05:42.093520 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.071933 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:42.032670 23716 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:42.015875 23728 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 13:05:42.078189 23730 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:05:42.079639 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:42.096117 22589 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.037015 23719 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.097553 23601 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.097790 23719 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.082609 23586 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:42.102504 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.083907 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44648.8us
W20260709 13:05:42.105118 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.080649 23702 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.105511 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.105995 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:42.106930 23604 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:42.107698 23604 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:42.108249 23604 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:42.069622 23645 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.113337 23645 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.113694 23645 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.117079 23561 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.117561 23561 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.117887 23561 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.120483 23713 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.088416 23732 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:42.120862 23716 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.124544 23732 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.125960 23716 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:42.128016 23696 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.616s	user 0.001s	sys 0.008s
I20260709 13:05:42.128564 23696 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.068364 23695 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.036130 23715 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.085393 23706 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:42.085562 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47295.4us
W20260709 13:05:42.130139 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 112793us
I20260709 13:05:42.131431 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.095619 23647 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.133687 22009 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:42.036641 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.134900 23647 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.137843 21854 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:42.138160 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:42.138811 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:42.139190 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:42.139688 21858 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:42.140326 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:42.144713 22003 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: [1.3-1.3]
W20260709 13:05:42.145630 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.080911 23654 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:42.149683 21857 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:42.025106 23720 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:42.165938 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:42.166280 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:42.166548 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:42.132887 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:42.122218 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:42.123687 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:42.159312 23695 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:42.122954 23601 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.160516 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.161948 22589 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:42.174391 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:42.179133 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55510.7us
I20260709 13:05:42.180253 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:42.075744 23615 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.076476 23718 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.182425 21855 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:42.122082 23589 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.122275 23723 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:42.164490 23715 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.121179 23733 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.193890 21856 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:42.195710 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62919.1us
I20260709 13:05:42.196851 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.193292 23706 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.191378 23695 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.200718 23706 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:42.201210 23706 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.201766 23719 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.202190 21857 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: [2.4-2.4]
W20260709 13:05:42.202670 23731 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.129s	user 0.001s	sys 0.002s
I20260709 13:05:42.203963 22587 raft_consensus.cc:2368] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.204041 23719 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.204799 23716 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.205384 23716 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:42.205729 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:42.206286 23716 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:42.206902 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.208271 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.176154 23702 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:05:42.195393 23717 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.210995 23717 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:42.211366 23717 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.170246 23502 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.213864 23502 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.214658 23582 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:42.215086 23582 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:42.216089 23502 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.216027 23582 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:42.217428 23649 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.169059 23654 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.220167 23654 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.219823 23706 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:42.195881 23733 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.221697 22589 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:42.222033 23733 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.224381 23706 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.226401 23720 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.227556 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:42.227393 23723 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.228279 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:42.228672 23713 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.229027 22588 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:42.229279 23713 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.229771 23713 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.163005 21853 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
W20260709 13:05:42.230834 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50710.2us
I20260709 13:05:42.191164 23579 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.232570 23579 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:42.233590 23579 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:42.236591 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:42.238356 21932 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:42.239061 22592 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:05:42.239971 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117842us
I20260709 13:05:42.240414 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.244947 22004 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:42.246995 21935 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:42.158298 23585 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:05:42.206705 23695 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.253602 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:42.254890 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56486.4us
I20260709 13:05:42.212488 23604 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.193045 23732 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:42.261554 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:42.258386 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:42.260401 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.262212 22589 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.264487 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:42.256855 23540 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:42.267082 23540 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.267586 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:42.268486 23604 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.268527 23502 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:42.270129 23502 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.270484 23502 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:42.270874 23502 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.248168 23601 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.272351 23601 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:42.273170 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.273910 22006 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:42.275568 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:42.276168 21857 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:42.276934 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:42.277431 21857 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.264345 23732 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.279196 23732 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:42.281085 23700 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 13:05:42.281498 23700 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.264463 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.256603 23577 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.280396 23733 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:42.283527 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:42.278671 21858 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:42.287521 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.288352 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.194769 23714 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:05:42.292243 21856 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:42.315580 21857 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:42.296331 22589 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:42.317170 22007 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:42.274998 23601 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.299059 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.300365 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:42.300535 22592 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:42.300688 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:42.304066 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:42.330448 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.331086 22001 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.330448 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:42.332010 22592 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:42.318140 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.318151 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:42.321532 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:42.323109 21900 leader_election.cc:400] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.263700 23696 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.324584 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:42.326200 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.336161 22008 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:42.273180 23702 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:42.332257 22588 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:42.332556 22007 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.332774 22589 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3.
I20260709 13:05:42.291684 23582 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:42.338838 23733 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.342352 23709 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.344069 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1765) took 1077 ms. Trace:
I20260709 13:05:42.345023 22007 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:42.345096 23604 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:42.346751 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:42.353823 23735 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
W20260709 13:05:42.355223 23734 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.277s	user 0.000s	sys 0.004s
I20260709 13:05:42.316851 23654 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.441563 23654 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.442014 23654 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.360019 23645 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.447000 23645 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.362954 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:42.305702 23580 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:42.355902 23615 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:42.423399 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 124410us
I20260709 13:05:42.360920 23702 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:42.425918 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 142626us
I20260709 13:05:42.361158 23601 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:42.435413 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.352479 23561 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.438506 22584 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:42.438577 23604 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:42.361333 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 43601.5us
I20260709 13:05:42.319840 23715 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 13:05:42.502192 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:42.436663 21932 rpcz_store.cc:276] 0709 13:05:41.266817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.527081 (+260264us) service_pool.cc:224] Handling call
0709 13:05:41.527469 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.238825 (+711356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.343809 (+104984us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400228654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:42.343996 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628578176}
I20260709 13:05:42.361223 23696 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.332928 21900 leader_election.cc:403] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:05:42.577785 23745 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:05:42.455497 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:42.581161 23615 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:42.582549 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 254243us
I20260709 13:05:42.456005 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.323817 23647 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 13:05:42.420418 23540 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.443621 23709 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.506326 22591 raft_consensus.cc:2368] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.453073 23701 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.452415 23706 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:42.511546 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 219887us
I20260709 13:05:42.595319 22008 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:42.514869 22587 raft_consensus.cc:2368] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.515422 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:42.476027 23601 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.384235 23732 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.510154 23696 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:42.523787 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 207889us
I20260709 13:05:42.604822 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.605796 22006 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.525930 23717 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.514261 23577 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:42.508205 23738 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:42.531244 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.521485 23579 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.544692 23578 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:42.551786 23561 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.566632 23604 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.572285 23695 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:42.587637 22007 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:42.587175 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 264237us
I20260709 13:05:42.638482 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.639603 21900 leader_election.cc:400] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.639823 21900 leader_election.cc:403] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:42.560701 23741 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:05:42.592206 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 136299us
I20260709 13:05:42.015295 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:42.602706 23537 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.592770 23582 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:42.644932 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52656.1us
I20260709 13:05:42.633450 23732 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.635794 23579 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.601658 23601 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:42.636759 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 121414us
I20260709 13:05:42.630556 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:42.623427 22001 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:42.637763 23695 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:42.637388 23578 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:42.728744 22006 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:42.612227 23706 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:42.705055 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.731527 22007 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:42.695116 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:42.645960 23584 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.624748 23540 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.646399 23502 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:42.697887 23739 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.192s	user 0.003s	sys 0.000s
W20260709 13:05:42.699618 23742 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:05:42.939435 23713 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.011s
I20260709 13:05:42.940106 23578 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.725414 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:42.939868 23713 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.730623 21758 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 188) took 1991 ms. Trace:
I20260709 13:05:42.645309 23733 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.625873 23723 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:05:42.942255 23582 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.531121 23702 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.944142 23702 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:42.946797 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 215723us
I20260709 13:05:42.742197 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:42.948400 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:42.645576 23577 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:05:42.949960 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.949822 23577 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:42.950515 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.951592 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:42.952358 23584 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:42.726140 23712 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:42.755645 23645 raft_consensus.cc:2764] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:05:42.753345 23649 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:42.956259 23649 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:42.957320 23649 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.753265 23695 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.965449 23695 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:42.757745 23537 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:42.625349 23701 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.971253 23701 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.730471 21757 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 2, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.973106 21757 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:05:42.773567 23696 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.978636 23696 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.979491 21935 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:42.753211 23717 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:42.840597 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 145532us
I20260709 13:05:42.763921 23720 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:42.985420 23720 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.986361 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.987725 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:42.987902 23720 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:42.933691 23709 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.744256 23706 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:42.988684 23636 consensus_queue.cc:1175] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" responder_term: 2 status { last_received { term: 2 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 2 index: 4 } }
I20260709 13:05:42.990198 23709 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:42.993130 23585 consensus_queue.cc:1175] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 2 status { last_received { term: 2 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 2 index: 4 } }
I20260709 13:05:42.690547 23699 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:42.941452 23502 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:42.941787 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:42.726545 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 96114.7us
I20260709 13:05:42.935891 23601 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.029363 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:42.941903 21758 rpcz_store.cc:276] 0709 13:05:40.739662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.740035 (+   373us) service_pool.cc:224] Handling call
0709 13:05:42.730562 (+1990527us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:41.145274 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:41.182261 (+ 36987us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:41.182314 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:05:41.182342 (+    28us) tablet.cc:662] Decoding operations
0709 13:05:41.183041 (+   699us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:41.183121 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:41.183154 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:05:41.183607 (+   453us) tablet.cc:701] Row locks acquired
0709 13:05:41.183659 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:05:41.183842 (+   183us) write_op.cc:270] Start()
0709 13:05:41.183994 (+   152us) write_op.cc:276] Timestamp: P: 1783602341183786 usec, L: 0
0709 13:05:41.184024 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:05:41.184664 (+   640us) log.cc:844] Serialized 1682 byte log entry
0709 13:05:41.334753 (+150089us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b18000be120. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:05:41.525934 (+191181us) op_driver.cc:464] REPLICATION: finished
0709 13:05:41.672471 (+146537us) write_op.cc:301] APPLY: starting
0709 13:05:41.672577 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:41.672765 (+   188us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:41.672814 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:41.675815 (+  3001us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:41.676191 (+   376us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:05:41.676415 (+   224us) write_op.cc:312] APPLY: finished
0709 13:05:41.807395 (+130980us) log.cc:844] Serialized 41 byte log entry
0709 13:05:41.807621 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:41.807726 (+   105us) write_op.cc:454] Released schema lock
0709 13:05:41.807785 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:41.809191 (+  1406us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:41.809219 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":171,"reported_tablets":7,"spinlock_wait_cycles":295680,"child_traces":[["op",{"apply.queue_time_us":144112,"apply.run_cpu_time_us":3726,"apply.run_wall_time_us":136651,"num_ops":4,"prepare.queue_time_us":145797,"prepare.run_cpu_time_us":11030,"prepare.run_wall_time_us":126997,"raft.queue_time_us":178186,"raft.run_cpu_time_us":5731,"raft.run_wall_time_us":119167,"replication_time_us":341721,"spinlock_wait_cycles":4755328,"thread_start_us":300153,"threads_started":4,"wal-append.queue_time_us":90055,"wal-append.run_cpu_time_us":20404,"wal-append.run_wall_time_us":1599609}]]}
I20260709 13:05:43.032294 23578 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.035557 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:43.035773 23540 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:43.036633 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.037262 21930 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:43.038208 23715 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:43.038800 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.035611 23578 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.040529 23720 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.040988 23720 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.041278 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:43.035604 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 310283us
I20260709 13:05:43.042062 21935 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:43.042237 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:43.045706 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.045951 22590 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.028527 23723 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
W20260709 13:05:43.046801 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.044670 23601 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.048437 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:43.032840 23645 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.063839 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:43.060890 23715 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:43.061684 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.064615 22004 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:43.046278 23654 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.604s	user 0.008s	sys 0.010s
I20260709 13:05:43.065286 23715 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.066778 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.067514 21930 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:43.052851 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:43.044881 21825 leader_election.cc:400] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3.
I20260709 13:05:43.068104 22007 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.068217 21825 leader_election.cc:403] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:05:43.046813 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:43.070624 22009 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:43.073678 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:43.074240 22589 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:43.074488 22001 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:43.074316 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.075351 21927 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:43.076187 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.076726 23540 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.060618 23592 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.077548 23592 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.073678 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.079267 22005 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:43.079478 23592 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.081424 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.081935 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:43.082588 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 131078us
I20260709 13:05:43.071123 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:43.086052 22590 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:43.093017 22004 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:43.094303 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.082027 22002 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:43.042049 23589 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.098039 21935 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:43.101148 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.101883 21935 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.042460 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.105024 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.111409 21935 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:43.116734 22007 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:43.065966 23654 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:43.130573 21932 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:43.132767 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:43.123133 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 406909us
W20260709 13:05:43.134631 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51192.7us
I20260709 13:05:43.138425 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.043002 23720 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.146597 21928 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:43.147099 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.147228 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.153580 21935 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:43.042887 23578 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.099987 23601 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.099625 23712 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:43.033607 23717 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.095911 23540 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:43.157229 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.159075 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.099898 23723 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.133558 23592 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.162060 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.173372 21931 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:43.159539 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.165485 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.174073 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.174490 21935 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:43.166111 21932 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:43.166481 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.175863 21928 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.166989 21933 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:43.136204 23701 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.168287 23579 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.168833 23604 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.175325 21900 leader_election.cc:400] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:43.177343 21900 leader_election.cc:403] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:43.178655 21934 raft_consensus.cc:2410] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:05:43.183224 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.183840 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:43.182699 21900 leader_election.cc:400] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:43.184324 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.184680 21900 leader_election.cc:403] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:43.098702 23577 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:05:43.157814 23647 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:43.153105 23589 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:43.184857 21928 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:43.162774 23537 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:43.163008 23712 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.187911 23502 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:43.189692 22590 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:43.190856 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:43.193298 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:43.193665 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.201822 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.202301 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.155495 21926 raft_consensus.cc:2376] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4. Re-sending same reply.
I20260709 13:05:43.187162 22007 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:43.209514 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:43.214776 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 168035us
W20260709 13:05:43.216842 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.176836 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.178601 23636 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:43.156270 23586 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:05:43.238754 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.240432 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.240983 21931 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.233675 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:43.181814 23582 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.264060 22587 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.195431 23701 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:43.186842 23733 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:43.234954 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50700.8us
I20260709 13:05:43.217839 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.216074 23695 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4.
I20260709 13:05:43.200717 23723 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:43.229465 23712 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.266595 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:43.282358 22590 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:43.241319 23728 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.272277 21932 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:43.231578 23592 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:43.269457 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.287218 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.218036 23719 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:05:43.255847 23759 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:43.290736 21931 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:43.261631 23763 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.229219 23604 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.297875 23589 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.266251 23764 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.236358 23725 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:05:43.305105 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.268677 23718 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:43.322642 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89647.9us
W20260709 13:05:43.322811 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49368.8us
I20260709 13:05:43.328586 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.324648 22001 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.329319 21928 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:43.282604 23636 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.277686 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:43.328559 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:43.286834 23582 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:43.331259 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:43.290613 23579 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:43.290253 23749 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:43.287096 21929 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:43.334149 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99777.1us
I20260709 13:05:43.297338 23766 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:43.326840 23580 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.869s	user 0.005s	sys 0.013s
I20260709 13:05:43.276782 23561 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:05:43.339418 23713 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.331380 23718 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.337056 23759 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:43.335227 23728 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:05:43.360782 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47531.5us
I20260709 13:05:43.343684 23720 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.348380 23766 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.352075 23647 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.352917 23502 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.336449 23636 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.369172 22000 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:43.391811 22001 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:43.347470 23582 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.396426 22001 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:43.397915 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:43.364939 23718 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.369961 23580 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:43.402529 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71330.6us
I20260709 13:05:43.371122 23749 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.357210 23577 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.398710 23720 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.340540 23763 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:43.408872 23760 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:05:43.411026 23773 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.336416 23764 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.391887 23654 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.362708 23586 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:05:43.428404 23762 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:05:43.413771 23749 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.431337 23749 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:43.390558 23766 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.390560 23716 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:43.372493 23733 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.349730 23540 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.407859 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:43.428026 23765 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.149s	user 0.000s	sys 0.002s
I20260709 13:05:43.407094 23580 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.356316 23592 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:43.437891 23769 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:43.438283 23696 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:43.440156 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.440061 23645 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:43.399951 23502 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:43.443537 23767 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:43.465425 23719 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.465998 23719 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.438535 23725 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:43.446353 23715 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:43.438797 23716 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.451627 23580 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.412971 23647 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:43.449787 23586 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:43.379576 23723 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.419636 23577 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.453717 23604 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.372133 23701 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:43.458470 23696 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:43.483265 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 253275us
I20260709 13:05:43.463701 23654 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:43.470654 23715 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:43.504431 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:43.451450 23728 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.447731 23773 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.522091 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.521231 23714 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:05:43.530373 23645 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.554832 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:43.532652 23702 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.586s	user 0.003s	sys 0.008s
I20260709 13:05:43.555598 22007 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.527228 23719 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.554256 23715 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.567540 23715 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.553860 23636 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.522156 23696 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.525069 23647 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.577960 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.535531 23718 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:43.577688 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 226578us
I20260709 13:05:43.585319 23771 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.585978 23771 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.574679 23720 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:43.604782 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.592968 23696 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.581347 23763 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.595211 23771 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.660080 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:43.646063 23577 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.662595 23773 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.669152 23615 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.670735 23615 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:43.677923 23771 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.678068 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.678536 23771 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.679792 21933 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:43.678843 21927 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
W20260709 13:05:43.683120 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.683660 21934 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:43.684504 23636 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.685076 23636 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.686291 23707 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.686738 23707 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.687255 23636 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.688009 23702 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.689215 23707 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.692081 23577 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.690478 23732 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.693902 23577 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.694088 23732 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.696034 23732 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.697189 21856 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:43.697352 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:43.697997 21854 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.698333 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:43.698885 21858 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.699281 21853 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:43.700349 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.700914 21857 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.702349 21851 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:43.688933 23702 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:43.706470 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.706857 23702 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.708225 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.709151 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.710134 23699 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.715018 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.715080 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.715821 21856 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.716827 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.718094 23720 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.718456 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.717348 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.719061 21859 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.719393 21852 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.716782 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.718588 23720 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.722275 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.669339 23738 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:05:43.723922 21855 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:43.683039 23771 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.725895 23770 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.682873 23773 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.731498 23773 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.687762 23636 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.691286 23716 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.730839 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.733377 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.745313 21857 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.696331 23577 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.730075 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.740761 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:43.722231 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.724659 23732 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.748937 22007 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.746320 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.750375 21858 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:43.748292 21855 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:43.756119 23702 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:43.740773 23714 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:43.761555 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:43.761412 23714 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.762451 23695 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:43.762867 23695 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.763217 23695 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.763250 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.764019 23714 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:43.741017 23718 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.764837 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.756565 23577 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.768699 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.718463 23580 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.769748 23586 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:43.772900 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.756042 23779 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:05:43.745185 23636 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.780966 22007 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:43.766209 23654 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:43.767935 23763 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:43.795464 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:43.756328 23770 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.773613 23714 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.781502 23586 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:43.808705 23586 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.808524 23714 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:43.810155 23717 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.003s
I20260709 13:05:43.810595 23717 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:43.757671 23771 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:05:43.806478 23718 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.813283 22005 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:43.774075 23577 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.814191 23773 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.818862 23502 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.821360 23720 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:43.825644 23712 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 13:05:43.785348 23779 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:43.826838 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:43.772442 23732 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.836323 23636 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.838018 23732 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.828034 21854 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:43.805339 23763 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.757230 23615 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:43.829068 23718 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:43.836163 23779 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.839231 22007 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:43.844977 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:43.838292 23601 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.003s
I20260709 13:05:43.829416 23702 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:05:43.857749 22002 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.032s
W20260709 13:05:43.864915 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 465638us
I20260709 13:05:43.867416 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:43.883515 22002 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.006s	sys 0.032s
I20260709 13:05:43.849735 23763 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.853276 23764 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.897876 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.898087 23577 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.857983 23779 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.912458 21934 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:43.931631 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.935021 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.935846 21855 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:43.897581 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:43.883723 23601 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3.
I20260709 13:05:43.937958 23770 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.939442 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.845203 23712 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:43.947458 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:43.949780 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.951748 21855 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:43.948798 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.907261 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:43.947785 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.954612 21857 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:43.955657 21856 raft_consensus.cc:2368] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3.
I20260709 13:05:43.962852 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:43.965166 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.965860 21852 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:43.966509 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.967453 21854 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:43.960726 23712 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:43.993966 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:43.995362 22000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.032s
W20260709 13:05:43.995818 22000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.012s	sys 0.032s
I20260709 13:05:43.996718 22005 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:43.963546 21934 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:43.954254 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:43.918129 23702 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.012524 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:44.014230 22587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.008s	sys 0.023s
W20260709 13:05:44.025794 22587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.008s	sys 0.023s
I20260709 13:05:44.026010 21854 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:44.025918 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:44.021502 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.026402 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:43.961169 22585 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:44.014789 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.019s	sys 0.016s
W20260709 13:05:44.027139 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.020s	sys 0.016s
I20260709 13:05:44.028398 21933 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:44.028844 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 361523us
W20260709 13:05:44.014634 23540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.011s	sys 0.019s
W20260709 13:05:44.011508 23766 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.005s
I20260709 13:05:44.033420 21854 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.036006 23782 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.021535 23580 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.021231 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:44.042017 23716 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:44.042505 23577 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.043520 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:44.078472 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:44.049714 23720 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:43.975682 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:44.052724 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.075304 22005 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:44.092399 21852 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:44.093355 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:44.052614 23540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.011s	sys 0.019s
I20260709 13:05:44.081341 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:44.034132 23712 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:44.037442 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.094928 22006 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:43.958657 23718 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:44.077237 23645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.014s	sys 0.016s
I20260709 13:05:44.102541 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.082158 21851 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:44.106734 21933 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:44.091528 21927 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:44.124007 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 355307us
I20260709 13:05:44.128023 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:44.129606 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:44.132359 21852 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:44.080127 23578 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:05:44.134775 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.135932 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.140527 21852 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:44.141178 21933 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:44.104391 23645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.015s	sys 0.017s
I20260709 13:05:44.141602 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.114939 23712 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:44.076025 23766 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.186136 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:44.090550 23577 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.100040 23719 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 13:05:44.190477 22556 leader_election.cc:336] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.601s (SENT)
W20260709 13:05:44.147640 23582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.015s	sys 0.016s
I20260709 13:05:44.184443 23732 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:44.207329 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:44.214754 21930 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:44.147775 23645 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:44.187484 23580 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.195578 21853 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:44.189854 23770 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.189538 23707 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.241237 23770 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.241891 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 348) took 1055 ms. Trace:
I20260709 13:05:44.241364 23707 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.187171 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:44.207854 23582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.016s	sys 0.016s
I20260709 13:05:44.195199 23636 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:44.146387 23561 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:05:44.243973 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:44.245620 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 474543us
I20260709 13:05:44.250232 21930 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:44.206655 23778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.033s	sys 0.062s
I20260709 13:05:44.214673 23714 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.240675 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.090991 23782 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.258713 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.257033 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.237815 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.261685 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.257416 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:44.260255 21854 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:44.267437 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:44.267982 21928 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.263283 21930 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:44.269158 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.269695 21930 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:44.258457 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.262593 21931 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:44.237862 23580 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.242256 22002 rpcz_store.cc:276] 0709 13:05:43.186229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.186340 (+   111us) service_pool.cc:224] Handling call
0709 13:05:43.226686 (+ 40346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.226913 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:43.902411 (+675498us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.241518 (+339107us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.241564 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.241625 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.241692 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.241810 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22801,"prepare.queue_time_us":825063,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":45543,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":939634,"threads_started":2}
I20260709 13:05:44.275508 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.245481 23732 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:44.280344 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:44.281522 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.277381 21930 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:44.253140 23778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.033s	sys 0.062s
W20260709 13:05:44.277163 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 198759us
I20260709 13:05:44.279237 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.243582 23586 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.282941 21935 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:44.282979 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:44.284590 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94141.3us
I20260709 13:05:44.298801 21931 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:05:44.299551 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.831s (SENT)
I20260709 13:05:44.299989 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.254623 23636 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:44.300765 21928 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:44.301304 23636 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.279610 21933 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:05:44.275815 23699 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 13:05:44.272804 23580 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.297217 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:44.276305 23715 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.247862 23645 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.311933 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:44.286726 23647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.026s
I20260709 13:05:44.283859 23771 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.310439 21851 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:44.306249 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.304097 23702 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.319424 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:44.309041 23604 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.307170 23720 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:44.316484 23647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.005s	sys 0.026s
I20260709 13:05:44.390645 21934 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:44.321705 23578 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.310637 23714 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 13:05:44.358978 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 59426.1us
I20260709 13:05:44.359489 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:44.318996 23771 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:44.372490 23791 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:05:44.313485 23725 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.381973 23797 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:05:44.325947 23792 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.373903 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.322395 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.374496 21932 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:44.373180 23800 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:05:44.392480 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.192776 22586 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:44.357867 23803 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.393832 23578 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.358549 23728 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:44.393985 21898 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.001s (SENT) [suppressed 74 similar messages]
I20260709 13:05:44.369020 23592 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.397217 23771 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.395323 21934 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.433344 23578 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.403561 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:44.452291 23809 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 13:05:44.441464 23763 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 13:05:44.391211 23604 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.438112 23764 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.440176 23586 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.404081 23725 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.421967 23766 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:44.465569 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 127819us
W20260709 13:05:44.467362 23790 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:05:44.454636 21851 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
W20260709 13:05:44.464722 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 72357.5us
I20260709 13:05:44.422055 21853 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:44.439889 23577 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.433445 23592 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.442401 23796 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:44.408391 23792 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.485842 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:44.489775 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:44.490597 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:44.491113 23807 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:05:44.480887 22585 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:44.496006 21860 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:44.463989 23728 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:44.501827 22556 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.003s (SENT)
I20260709 13:05:44.502620 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:44.505461 23786 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.506940 23792 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:44.498392 23720 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:44.508277 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.510738 23725 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.518805 23766 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.521384 23707 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.483421 23764 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.570690 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:44.572369 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.405491 23732 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:44.533322 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 139349us
I20260709 13:05:44.469681 23763 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.532058 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.500007 23592 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:44.524834 23695 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.001s
W20260709 13:05:44.530787 23808 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:05:44.562667 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60128.6us
I20260709 13:05:44.592825 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:44.493999 23779 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.000s
I20260709 13:05:44.555197 23728 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:44.574719 23819 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:44.558084 23577 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:44.577152 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.577468 21927 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:44.579744 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:44.620980 21852 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.558374 23786 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:44.563561 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 462075us
W20260709 13:05:44.601758 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99968.6us
I20260709 13:05:44.604636 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:44.573472 23766 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.574234 23778 raft_consensus.cc:2764] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:05:44.615020 23615 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.618070 21857 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:44.667116 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.583981 23725 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:44.655234 22556 leader_election.cc:336] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.164s (SENT)
I20260709 13:05:44.655356 21852 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:44.669816 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:44.670547 21852 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.601640 23714 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:44.668707 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.620812 21858 raft_consensus.cc:2368] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4.
I20260709 13:05:44.605856 23540 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.609961 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.601343 23815 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.667913 21857 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:44.606698 23823 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.653908 23786 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.675992 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.663519 23766 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.667910 23718 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.607511 23695 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.680550 21852 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:44.681473 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:44.683596 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73763.6us
I20260709 13:05:44.683653 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.688570 21857 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:44.617628 23733 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.689544 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.690025 23733 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.691740 23733 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:44.684319 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50236.4us
I20260709 13:05:44.677482 21926 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:44.693084 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.694623 21852 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:44.697381 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:44.697983 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:44.614745 23779 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.650134 23577 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.698021 21852 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.682747 23813 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.685274 23645 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:44.706363 23792 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.706981 23792 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:44.707475 23792 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.686013 23815 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.694898 23718 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.737792 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:44.699203 21927 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:44.716775 23805 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.360s	user 0.000s	sys 0.003s
W20260709 13:05:44.718978 23817 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:05:44.704272 23813 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.757030 23796 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.684165 23540 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:44.727115 23818 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:05:44.746876 23763 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:44.779610 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 89316us
I20260709 13:05:44.734293 23695 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.784632 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:44.778261 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:44.785959 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 130755us
I20260709 13:05:44.786360 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:44.718483 23803 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.743731 23577 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.787959 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:44.787981 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:44.718205 23580 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.741789 23771 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.729367 23792 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.702090 23778 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:05:44.738957 23615 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:44.693637 23823 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.776113 23578 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.795701 23803 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:44.797211 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:44.799365 21932 raft_consensus.cc:606] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:44.800596 21932 raft_consensus.cc:614] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:44.791335 23647 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:44.810815 21852 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:44.799490 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:44.821332 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:44.825577 23802 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.825875 23792 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.826081 23802 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:44.826571 23802 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.826978 22002 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:44.828244 22006 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.823038 21929 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:44.823966 23778 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.837412 23732 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.864042 23814 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.864547 23783 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:44.841270 23749 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.864681 23789 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:05:44.867229 23789 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.841637 23832 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:44.868211 23732 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.873726 23732 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.881173 23783 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:44.853310 23580 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.882450 23786 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.867725 23789 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.864578 23814 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.889647 23647 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:44.894411 23814 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.895566 23647 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.885421 23766 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.886227 23779 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.903973 23771 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.884601 23832 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.883916 23749 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.902590 23645 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:44.911517 23815 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.916733 23813 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.915933 21853 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:44.918660 23792 raft_consensus.cc:476] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:05:44.885279 23783 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.922307 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.923146 22007 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:44.923319 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.923920 22004 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:44.923944 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.924288 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.914853 23725 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.922505 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.925832 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.915596 23766 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.925765 23728 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.929872 23789 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:44.932173 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.933313 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.921111 23577 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.937474 22002 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:44.937875 23577 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.937040 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.940106 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.916572 23540 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.942143 22001 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:44.943536 23577 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.809546 23803 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.947611 23649 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:44.951792 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.952661 22005 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:44.946923 23615 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:44.919593 23831 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:44.950511 23649 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:44.949488 23592 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:44.977253 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1538) took 1029 ms. Trace:
I20260709 13:05:44.980234 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:44.981498 21856 rpcz_store.cc:276] 0709 13:05:43.947946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.948544 (+   598us) service_pool.cc:224] Handling call
0709 13:05:44.976992 (+1028448us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b540025ab54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:44.977166 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6414080}
I20260709 13:05:44.984082 21859 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:44.973628 23749 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:44.986974 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 560) took 1143 ms. Trace:
I20260709 13:05:44.987381 22590 rpcz_store.cc:276] 0709 13:05:43.844037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.844177 (+   140us) service_pool.cc:224] Handling call
0709 13:05:44.986241 (+1142064us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:44.986441 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.986623 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.986651 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.986694 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:44.986730 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.986786 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.986895 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56602368}
I20260709 13:05:44.989122 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1537) took 1041 ms. Trace:
I20260709 13:05:44.983765 23786 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:44.989605 21859 rpcz_store.cc:276] 0709 13:05:43.947645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.947779 (+   134us) service_pool.cc:224] Handling call
0709 13:05:44.980064 (+1032285us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:44.989035 (+  8971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10561792}
W20260709 13:05:44.954879 23834 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:05:44.995661 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.998915 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:44.999722 22004 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:44.998251 21856 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:45.001358 21856 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.981861 23803 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.009531 21857 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:45.012530 21856 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.013768 21859 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.961985 23728 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:44.964259 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:44.956296 23832 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.008153 23592 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.022953 22005 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:45.023665 22005 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:45.017874 23502 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.996752 23831 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.026239 22005 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.023499 21926 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.989316 23615 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.047312 22005 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.037592 23837 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:45.048942 23728 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.048760 23837 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.049451 23728 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.052220 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.054365 23837 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:45.056025 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 258839us
I20260709 13:05:45.057955 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:45.037781 23764 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.039611 23831 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.068903 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.069597 22004 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:45.039937 23502 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.054253 23802 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.050557 23814 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.065807 23837 raft_consensus.cc:2764] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:05:45.086179 23615 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.090340 23770 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.093822 23502 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.121912 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.122793 22004 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.137527 22009 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.073911 23649 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.141116 23802 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.145855 22008 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:45.162968 22007 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:45.169680 21972 leader_election.cc:400] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3.
I20260709 13:05:45.170821 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:45.180922 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:45.133806 23766 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.190922 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.200356 21932 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:45.210520 23783 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:45.211459 22003 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:45.203441 22005 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:45.211253 23783 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:45.209622 23649 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:05:45.214210 23841 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:45.214869 21851 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:45.215257 23841 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:45.215636 23649 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.216070 23783 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:45.194100 23561 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.216545 23783 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:45.216924 23561 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.191058 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.221110 23561 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.164688 23786 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:45.193470 23832 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.222088 23786 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.222291 23832 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.148952 23837 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.225343 23832 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.226851 23814 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.212955 23766 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.215682 23841 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:45.211905 23719 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:05:45.219120 23716 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:05:45.230369 21926 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:45.165555 23802 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.235198 23782 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.235527 23786 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.237077 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.237751 22004 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:45.242345 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1535) took 1295 ms. Trace:
I20260709 13:05:45.242821 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.243979 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.275240 23802 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.275992 22587 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:45.245591 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.263118 23749 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.245404 23837 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.241101 23796 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:45.279266 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.246662 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:45.199450 23728 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.248893 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.249202 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:45.252197 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:45.260699 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.261873 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.287678 22589 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:45.287964 21932 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:45.288249 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:45.288622 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:45.289232 22593 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.264349 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:45.273591 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:45.266829 21855 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:45.216156 23649 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.285516 22590 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.293488 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.303092 22004 raft_consensus.cc:2368] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:45.304216 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.294453 21935 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:45.296274 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44173.3us
I20260709 13:05:45.307000 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.307044 23829 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:45.267319 21858 rpcz_store.cc:276] 0709 13:05:43.947077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.947215 (+   138us) service_pool.cc:224] Handling call
0709 13:05:45.242070 (+1294855us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b5400016b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.242240 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538765184}
I20260709 13:05:45.297796 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:45.277453 23749 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.308336 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.308557 23829 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:45.308925 21858 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.309059 23829 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:45.309957 23829 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.310307 23749 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.297354 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.312409 23636 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:45.314203 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:45.284531 23832 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.315270 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.315286 23636 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:45.315878 23636 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.315914 22004 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:45.316491 23636 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:45.315814 23832 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.317361 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.317576 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.318547 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:45.321877 21972 leader_election.cc:400] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4.
I20260709 13:05:45.321775 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.315469 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:45.277472 23716 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:45.284624 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.330840 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.286818 23782 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.296510 23592 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:45.300052 23701 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.329370 23841 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.335269 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:45.339919 22589 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.340626 21932 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.328088 23714 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.300823 23786 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.336179 23702 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:45.321067 23577 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.336607 23802 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.337157 23592 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.295830 23601 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.333155 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.340559 23701 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.335081 23782 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.331610 23843 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.328523 23716 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:45.355278 23837 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.300221 23502 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:45.361183 22589 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:05:45.363741 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.328729 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.344959 23702 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.366910 22588 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.337054 23831 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.381506 23585 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.381412 23843 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.343925 23714 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:45.368559 23701 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.370618 22003 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:45.337050 23764 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:05:45.375299 22005 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:45.357499 23592 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.382303 23585 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.370174 21932 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:45.344784 23786 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:45.389793 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.390982 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.391726 21932 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:45.397661 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.398959 22589 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 1.
I20260709 13:05:45.405365 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.406263 22005 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.383709 23702 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:45.409405 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.360086 23601 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.412119 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.413030 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.413133 21858 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:45.385325 23714 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.418587 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.409876 23786 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.419430 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.411298 22593 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:45.419545 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:45.389658 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:45.415789 23766 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.428843 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:45.429495 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.430123 21932 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:45.430411 21854 raft_consensus.cc:2368] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:45.430979 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 213) took 1393 ms. Trace:
I20260709 13:05:45.427727 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.431370 21854 rpcz_store.cc:276] 0709 13:05:44.037889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.429176 (+1391287us) service_pool.cc:224] Handling call
0709 13:05:45.430899 (+  1723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.435645 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.427176 21926 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.447247 22005 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:45.441713 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.449803 21854 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:45.431926 23766 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:45.455359 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.456030 22593 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:45.416090 23716 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.465703 22005 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.462289 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.457135 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.467725 21932 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:45.468808 21854 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:45.470322 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.464982 23649 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.441270 23802 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:45.479396 23802 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:45.480448 23802 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:45.483059 23848 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:05:45.475658 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.485301 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1714) took 1289 ms. Trace:
I20260709 13:05:45.485785 21856 rpcz_store.cc:276] 0709 13:05:44.195808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.027733 (+831925us) service_pool.cc:224] Handling call
0709 13:05:45.028157 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.028911 (+   754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.028954 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.029518 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.368184 (+338666us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:45.368228 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.368305 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.485019 (+116714us) raft_consensus.cc:1685] finished
0709 13:05:45.485046 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.485230 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":4691,"thread_start_us":306526,"threads_started":1,"wal-append.queue_time_us":369207}
I20260709 13:05:45.478051 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.491863 21854 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:45.493213 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.492735 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.498129 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.499845 21854 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:45.501062 22005 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.502434 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.504235 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.471745 23843 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.507581 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.508160 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:45.508491 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 190008us
I20260709 13:05:45.512850 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.515774 21854 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 2: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:45.483085 23716 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.518666 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:45.513841 21856 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:45.526856 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.528048 22005 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:45.498191 23647 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.530121 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.531728 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.532209 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.519851 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 215) took 1481 ms. Trace:
I20260709 13:05:45.535553 21932 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:45.533037 21856 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:45.538017 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.517194 23837 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:45.540910 21856 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.532938 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.516897 23782 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.562000 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.563320 21932 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.533493 21852 rpcz_store.cc:276] 0709 13:05:44.038458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.841246 (+802788us) service_pool.cc:224] Handling call
0709 13:05:44.855103 (+ 13857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.855555 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.855597 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.856177 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.079890 (+223713us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:45.404349 (+324459us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.519440 (+115091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.519541 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.519598 (+    57us) raft_consensus.cc:1685] finished
0709 13:05:45.519615 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.519775 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":388464,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":141,"raft.queue_time_us":99864,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":22773,"thread_start_us":556624,"threads_started":3,"wal-append.queue_time_us":343633}
I20260709 13:05:45.526579 23589 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:45.534338 23846 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 13:05:45.533981 23647 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.564123 21854 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:45.567903 22005 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:45.572327 22005 raft_consensus.cc:606] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:05:45.572851 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.573443 21854 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:45.574333 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.576069 21852 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:45.559957 23837 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.562808 23585 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.583096 21852 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:45.538865 23766 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:05:45.574331 23647 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:45.584702 23845 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:05:45.585103 23749 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:45.585494 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1548) took 1389 ms. Trace:
I20260709 13:05:45.590966 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.571115 23856 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:45.593854 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1715) took 1386 ms. Trace:
I20260709 13:05:45.600281 21852 rpcz_store.cc:276] 0709 13:05:44.207959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.582431 (+1374472us) service_pool.cc:224] Handling call
0709 13:05:45.582813 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.593559 (+ 10746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.593794 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.564553 23855 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:45.595026 21856 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:45.581809 22588 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:05:45.573958 23601 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.598229 21854 rpcz_store.cc:276] 0709 13:05:44.196300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.573035 (+1376735us) service_pool.cc:224] Handling call
0709 13:05:45.573296 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.585244 (+ 11948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.585436 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.598117 23749 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.568955 23829 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:05:45.591924 23837 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.610450 22588 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 2
I20260709 13:05:45.621282 23841 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.626515 23589 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:45.627017 23589 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.628221 23702 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.604683 23814 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:45.629181 23814 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.629554 23814 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.630004 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 210) took 1593 ms. Trace:
I20260709 13:05:45.630419 21851 rpcz_store.cc:276] 0709 13:05:44.036945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.490164 (+453219us) service_pool.cc:224] Handling call
0709 13:05:45.629766 (+1139602us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54000596d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.629937 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1618221952}
I20260709 13:05:45.633308 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1547) took 1491 ms. Trace:
W20260709 13:05:45.634119 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.614404 23749 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.633682 21859 rpcz_store.cc:276] 0709 13:05:44.141515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.014623 (+873108us) service_pool.cc:224] Handling call
0709 13:05:45.015102 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.023740 (+  8638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.023780 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.038041 (+ 14261us) log.cc:844] Serialized 564 byte log entry
0709 13:05:45.261060 (+223019us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:45.261097 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.261155 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.633105 (+371950us) raft_consensus.cc:1685] finished
0709 13:05:45.633129 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.633257 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":205830,"threads_started":1,"wal-append.queue_time_us":321023}
I20260709 13:05:45.596818 23636 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.616080 23856 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:45.641566 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1550) took 1380 ms. Trace:
I20260709 13:05:45.641824 23856 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.642004 21859 rpcz_store.cc:276] 0709 13:05:44.261073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.640607 (+1379534us) service_pool.cc:224] Handling call
0709 13:05:45.640941 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.641223 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.641254 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.641298 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:45.641334 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.641390 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.641509 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.647444 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.616767 22588 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:05:45.648751 22588 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:05:45.649149 22588 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.655192 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1720) took 1415 ms. Trace:
I20260709 13:05:45.655709 21854 rpcz_store.cc:276] 0709 13:05:44.239361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.609230 (+1369869us) service_pool.cc:224] Handling call
0709 13:05:45.609469 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.649795 (+ 40326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.649823 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.654908 (+  5085us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:45.654954 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.655025 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.655139 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24721,"raft.queue_time_us":24875,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7860}
I20260709 13:05:45.658386 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1726) took 1334 ms. Trace:
I20260709 13:05:45.658810 21854 rpcz_store.cc:276] 0709 13:05:44.323537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.657390 (+1333853us) service_pool.cc:224] Handling call
0709 13:05:45.657775 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.657988 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.658024 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.658079 (+    55us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:45.658133 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.658201 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.658321 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.666682 21859 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:05:45.669169 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1552) took 1356 ms. Trace:
I20260709 13:05:45.669463 21859 rpcz_store.cc:276] 0709 13:05:44.313040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.646492 (+1333452us) service_pool.cc:224] Handling call
0709 13:05:45.662386 (+ 15894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.667235 (+  4849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.669123 (+  1888us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.649603 23649 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:45.675133 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.676651 22005 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:45.637876 23714 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.685782 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1729) took 1104 ms. Trace:
I20260709 13:05:45.686362 21859 rpcz_store.cc:276] 0709 13:05:44.581765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.682279 (+1100514us) service_pool.cc:224] Handling call
0709 13:05:45.685282 (+  3003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.685470 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.685498 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.685539 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:45.685569 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.685620 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.685738 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.687850 21859 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:45.680745 23649 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.690834 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.692032 23718 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.693328 23713 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:05:45.693753 23713 raft_consensus.cc:1068] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Service unavailable: leader transfer in progress
I20260709 13:05:45.694427 23713 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:45.694350 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.694904 23713 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:45.695644 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.696156 22003 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.695798 23713 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:45.697701 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.699081 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.702363 21856 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.692797 23649 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.706111 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.701315 23835 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.707532 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.707140 23835 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:45.708209 22000 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:45.709125 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.709671 22000 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:45.710733 22000 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.711961 22000 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:45.713603 23749 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.715375 22003 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:45.715705 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1719) took 1476 ms. Trace:
I20260709 13:05:45.716135 21852 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.716473 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1792) took 1120 ms. Trace:
I20260709 13:05:45.717061 21932 rpcz_store.cc:276] 0709 13:05:44.596529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.564067 (+967538us) service_pool.cc:224] Handling call
0709 13:05:45.570074 (+  6007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.570420 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.570447 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.570888 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.666549 (+ 95661us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.671249 (+  4700us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.671921 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.672004 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.716292 (+ 44288us) raft_consensus.cc:1685] finished
0709 13:05:45.716311 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.716422 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4528,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":2721,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":11540,"thread_start_us":68163,"threads_started":1,"wal-append.queue_time_us":107619}
I20260709 13:05:45.716114 21857 rpcz_store.cc:276] 0709 13:05:44.239049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.689286 (+450237us) service_pool.cc:224] Handling call
0709 13:05:45.715497 (+1026211us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b540018ecd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.715646 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734001920}
I20260709 13:05:45.725088 21857 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:05:45.725490 21857 pending_rounds.cc:85] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 4
I20260709 13:05:45.725790 21857 pending_rounds.cc:107] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:05:45.726055 21857 raft_consensus.cc:2889] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.729076 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1793) took 1131 ms. Trace:
I20260709 13:05:45.696980 23713 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:45.714308 23717 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:05:45.729493 21933 rpcz_store.cc:276] 0709 13:05:44.597289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.607547 (+1010258us) service_pool.cc:224] Handling call
0709 13:05:45.621403 (+ 13856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.622218 (+   815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.622248 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.622692 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.671719 (+ 49027us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:45.671788 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.671872 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.728855 (+ 56983us) raft_consensus.cc:1685] finished
0709 13:05:45.728876 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.729019 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1283,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":48632,"threads_started":1,"wal-append.queue_time_us":58202}
I20260709 13:05:45.738571 21857 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.742096 23714 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:45.744881 23853 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:05:45.746352 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 301) took 1465 ms. Trace:
I20260709 13:05:45.746702 21930 rpcz_store.cc:276] 0709 13:05:44.280841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.287726 (+  6885us) service_pool.cc:224] Handling call
0709 13:05:44.298866 (+ 11140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.299732 (+   866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.299866 (+   134us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.245834 (+945968us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:05:45.250992 (+  5158us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.457263 (+206271us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.596960 (+139697us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.597234 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.597314 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.744299 (+146985us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:05:45.746129 (+  1830us) raft_consensus.cc:1685] finished
0709 13:05:45.746150 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.746272 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188001,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":4121,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":6621,"spinlock_wait_cycles":14746880,"thread_start_us":298469,"threads_started":2,"wal-append.queue_time_us":267993}
I20260709 13:05:45.749343 21930 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:45.707587 21926 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
W20260709 13:05:45.750663 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 54303us
I20260709 13:05:45.749864 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 305) took 1093 ms. Trace:
I20260709 13:05:45.712729 23837 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.730026 23835 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.768714 22006 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:45.720048 23589 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.744915 23585 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.761497 21928 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.777704 23592 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.779654 23701 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.763793 21930 rpcz_store.cc:276] 0709 13:05:44.656328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.748452 (+1092124us) service_pool.cc:224] Handling call
0709 13:05:45.748754 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.749661 (+   907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.749815 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.889556 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1796) took 1133 ms. Trace:
W20260709 13:05:45.811832 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.893795 21933 rpcz_store.cc:276] 0709 13:05:44.755978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.749194 (+993216us) service_pool.cc:224] Handling call
0709 13:05:45.888966 (+139772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.889185 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.889214 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.889273 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:45.889310 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.889376 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.889499 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.895447 21933 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:45.895818 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:45.896534 23841 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:45.896672 21933 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:45.782203 23814 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.900162 21933 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:45.892501 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 589185us
I20260709 13:05:45.902185 21933 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.788496 22591 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:45.781131 23827 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.099s	user 0.000s	sys 0.011s
I20260709 13:05:45.903833 21930 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.921293 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.909610 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:45.822566 23786 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.929538 23718 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:45.929714 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.930045 23718 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:45.930598 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:45.930967 23718 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.931473 23858 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:45.931943 23858 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:45.933168 23699 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:45.933549 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1716) took 1717 ms (client timeout 1991 ms). Trace:
I20260709 13:05:45.900882 23814 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.924749 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.788627 21926 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:45.933864 23647 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:45.934032 21853 rpcz_store.cc:269] 0709 13:05:44.215829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.507957 (+292128us) service_pool.cc:224] Handling call
0709 13:05:45.933348 (+1425391us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.933486 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798778752}
I20260709 13:05:45.935509 21853 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.925515 21858 raft_consensus.cc:2368] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:45.935945 23647 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.893074 23714 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.936702 22004 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:45.937333 23786 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.922878 23585 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.786887 23702 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:45.939175 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1685) took 3740 ms (client timeout 2001 ms). Trace:
I20260709 13:05:45.939468 23585 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:45.939558 21860 rpcz_store.cc:269] 0709 13:05:42.198657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.198887 (+   230us) service_pool.cc:224] Handling call
0709 13:05:45.938959 (+3740072us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b5400259c54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.939080 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5249556736}
I20260709 13:05:45.941042 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:45.941560 23829 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:45.941807 23854 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:45.889312 23749 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.942523 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1549) took 1702 ms. Trace:
I20260709 13:05:45.942883 21852 rpcz_store.cc:276] 0709 13:05:44.240426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.614017 (+1373591us) service_pool.cc:224] Handling call
0709 13:05:45.715919 (+101902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.716723 (+   804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.942350 (+225627us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b540018ecd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.942465 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183417088}
I20260709 13:05:45.926442 22585 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:45.926786 23837 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:45.927794 23592 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:45.928305 21928 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:05:45.765236 23577 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:45.951575 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1749) took 1012 ms. Trace:
I20260709 13:05:45.932802 23858 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:45.943224 23636 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.946007 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:45.947013 21930 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:45.925402 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.948163 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.961851 21930 pending_rounds.cc:85] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 5
I20260709 13:05:45.961848 21928 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:45.962271 22590 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:45.962425 21930 pending_rounds.cc:107] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:45.903565 21933 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:45.952812 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1746) took 1029 ms. Trace:
I20260709 13:05:45.956430 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:45.960024 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:45.963768 22009 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:45.949044 21932 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:05:45.965459 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.966526 21928 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:05:45.957482 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.969087 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.963061 22008 rpcz_store.cc:276] 0709 13:05:44.923539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.931840 (+  8301us) service_pool.cc:224] Handling call
0709 13:05:45.952645 (+1020805us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540045abd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.952759 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482204416}
I20260709 13:05:45.972652 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.973675 21853 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:45.974218 23578 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:45.974536 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.974978 22586 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:45.975850 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.914247 23720 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.977202 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:45.978009 21860 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:45.981376 21854 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:45.941593 23851 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.507s	user 0.001s	sys 0.000s
I20260709 13:05:45.985875 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.936682 23714 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:45.951963 22001 rpcz_store.cc:276] 0709 13:05:44.939631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.939885 (+   254us) service_pool.cc:224] Handling call
0709 13:05:45.951424 (+1011539us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b540042c2d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:45.951521 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3269888}
I20260709 13:05:45.962749 21930 raft_consensus.cc:2889] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.956943 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.994619 21858 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:45.996860 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.997504 22588 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:45.990813 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:45.961807 23766 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:45.957679 23702 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.972723 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.961476 23776 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.001368 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.008971 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.001917 21858 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:46.012553 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:45.957813 23585 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:45.924170 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.006343 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.025189 22557 leader_election.cc:400] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:46.025776 22557 leader_election.cc:403] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:46.007467 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.026446 22002 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:46.001951 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:46.027784 21852 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:46.036763 22009 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:05:45.982438 23720 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:46.015667 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.039999 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.041028 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.212476 22009 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.041766 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:45.965219 23649 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:46.024067 23702 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.047294 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.217826 21860 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:45.982208 23716 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:05:46.053581 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.998364 23636 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.075171 21852 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:46.053071 21858 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.006184 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:45.992007 23764 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:05:46.049252 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:46.039700 23585 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.229933 23771 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:46.230320 23771 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:46.030212 23776 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.231135 23771 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.023655 23766 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:46.097590 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.098531 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1501) took 1739 ms. Trace:
I20260709 13:05:46.099493 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.105885 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1556) took 1038 ms. Trace:
I20260709 13:05:46.239321 22005 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:46.239077 21859 rpcz_store.cc:276] 0709 13:05:45.067040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.688992 (+621952us) service_pool.cc:224] Handling call
0709 13:05:45.689344 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.104544 (+415200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.104590 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.104660 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:46.104713 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.105675 (+   962us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b540025c454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:46.105713 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.105831 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954237184}
W20260709 13:05:46.205165 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 262294us
I20260709 13:05:46.101195 21934 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:46.095440 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 133300us
I20260709 13:05:46.109391 23831 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.238058 21935 rpcz_store.cc:276] 0709 13:05:44.358656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.358864 (+   208us) service_pool.cc:224] Handling call
0709 13:05:44.359165 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.305284 (+946119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.098188 (+792904us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b540017eb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:46.098431 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150498560}
I20260709 13:05:46.244602 22009 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:46.247359 22001 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:46.255333 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.219435 22008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:05:46.269557 22005 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:46.269865 22008 raft_consensus.cc:606] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.270533 22008 raft_consensus.cc:614] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:46.219758 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:46.271677 22005 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:46.272008 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52329.9us
I20260709 13:05:46.225307 21852 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:05:46.227124 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 267171us
I20260709 13:05:46.116242 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.275146 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.276460 22007 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:05:46.276660 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:46.232707 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 221769us
I20260709 13:05:46.242448 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:46.100469 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1782) took 1844 ms (client timeout 2153 ms). Trace:
I20260709 13:05:46.277711 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:46.231206 23585 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.278332 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:46.281632 22007 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:46.087991 23715 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:46.287349 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 303) took 1988 ms. Trace:
I20260709 13:05:46.242353 22586 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.241820 23855 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:05:46.289875 23733 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.233464 23589 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.219688 23636 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.286656 21858 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:05:46.301123 21858 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 2
I20260709 13:05:46.242962 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.294932 21931 rpcz_store.cc:276] 0709 13:05:44.299338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.299456 (+   118us) service_pool.cc:224] Handling call
0709 13:05:44.299800 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.300105 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.300140 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.719191 (+1419051us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:05:45.723049 (+  3858us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.957547 (+234498us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.957599 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.957663 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.250209 (+292546us) raft_consensus.cc:1685] finished
0709 13:05:46.250223 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.287280 (+ 37057us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2633,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":59,"spinlock_wait_cycles":9966976,"thread_start_us":229875,"threads_started":1,"wal-append.queue_time_us":320410}
I20260709 13:05:46.301467 21858 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:05:46.278059 23718 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.305955 21858 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
W20260709 13:05:46.307040 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 343638us
I20260709 13:05:46.291395 23578 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.322175 22007 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:46.294409 22006 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:46.292510 23831 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.292752 23701 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:46.325294 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.302234 23766 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:46.300046 23713 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 13:05:46.291702 21926 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:46.301033 23585 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:46.335062 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56827.6us
I20260709 13:05:46.300524 23733 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.303684 23636 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.280499 22009 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:46.339192 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.340036 22007 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:46.291373 23854 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:05:46.308439 23776 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.324087 22000 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
W20260709 13:05:46.369026 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.335219 23585 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.350076 23733 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.386147 23604 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:46.287349 21929 rpcz_store.cc:269] 0709 13:05:44.256339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.261944 (+  5605us) service_pool.cc:224] Handling call
0709 13:05:46.100158 (+1838214us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:46.100366 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070365952}
I20260709 13:05:46.316131 23819 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:05:46.394248 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:46.328648 23578 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:46.399500 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 120848us
I20260709 13:05:46.399644 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:46.386788 23869 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:05:46.395251 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.404312 21859 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:46.334020 23831 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:46.406320 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 88938.5us
I20260709 13:05:46.402384 21852 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.331995 23813 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 13:05:46.402432 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66457.2us
I20260709 13:05:46.356427 23701 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:46.350873 23718 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:05:46.369395 23854 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:05:46.410089 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 132432us
I20260709 13:05:46.420514 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:46.401717 23867 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:05:46.396026 23585 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.408270 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:46.421326 22002 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:05:46.426434 22001 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:05:46.403748 23776 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:46.367815 23877 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:05:46.402053 23884 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:46.458326 23872 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:46.459877 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 226) took 1239 ms. Trace:
I20260709 13:05:46.418572 23832 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.429522 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.412120 23701 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:46.432019 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.432861 21852 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:46.430538 23702 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.435148 23771 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.436668 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1795) took 1837 ms. Trace:
W20260709 13:05:46.437352 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.443290 23578 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:46.479470 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53700.4us
I20260709 13:05:46.450693 23886 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:46.452044 23714 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.453558 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1723) took 2206 ms. Trace:
I20260709 13:05:46.460484 21857 rpcz_store.cc:276] 0709 13:05:45.220096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.759448 (+539352us) service_pool.cc:224] Handling call
0709 13:05:45.763078 (+  3630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.763474 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.763505 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.784943 (+ 21438us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.086934 (+301991us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:46.115208 (+ 28274us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.115453 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.115529 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.459633 (+344104us) raft_consensus.cc:1685] finished
0709 13:05:46.459651 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.459806 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4437,"prepare.queue_time_us":28597,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1586,"raft.queue_time_us":2937,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":74069,"thread_start_us":182837,"threads_started":1,"wal-append.queue_time_us":551207}
I20260709 13:05:46.461776 21860 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:46.430284 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:46.423529 23854 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:46.501672 23854 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.503104 23854 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.501108 21851 rpcz_store.cc:276] 0709 13:05:44.247238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.631359 (+1384121us) service_pool.cc:224] Handling call
0709 13:05:45.631772 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.632186 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.632226 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.662015 (+ 29789us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.760853 (+ 98838us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.924416 (+163563us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.924900 (+   484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.925012 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.453083 (+528071us) raft_consensus.cc:1685] finished
0709 13:05:46.453122 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.453453 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151224,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":680,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":190,"thread_start_us":54104,"threads_started":1,"wal-append.queue_time_us":354156}
I20260709 13:05:46.503579 23733 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.480923 21932 rpcz_store.cc:276] 0709 13:05:44.599237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.733765 (+1134528us) service_pool.cc:224] Handling call
0709 13:05:45.737374 (+  3609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.964342 (+226968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.436364 (+472022us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54003543d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:46.436576 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486254336}
I20260709 13:05:46.418184 23873 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.505949 23873 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.506431 23885 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.508271 23873 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.430940 23776 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.508868 23776 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.483333 23702 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.498116 23578 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:46.516865 23578 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.519096 23578 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.520947 21851 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:46.521292 23823 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.521629 21860 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:46.522869 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:46.524183 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:46.513659 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 83470.4us
I20260709 13:05:46.516301 23702 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.476588 23701 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.526726 21852 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:46.478981 23786 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.558178 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1717) took 2342 ms. Trace:
W20260709 13:05:46.559681 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 151480us
I20260709 13:05:46.560046 21858 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:46.560760 21856 rpcz_store.cc:276] 0709 13:05:44.217891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.607015 (+1389124us) service_pool.cc:224] Handling call
0709 13:05:45.607330 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.702866 (+ 95536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.557877 (+855011us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b540008acd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:46.558102 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218018688}
I20260709 13:05:46.561082 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.562088 21852 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.424932 23879 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:05:46.431437 23874 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 13:05:46.564566 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.566891 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.565752 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.601414 22007 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:46.588968 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.594465 23884 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:46.564857 23873 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:46.572192 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.603917 21856 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.603511 23884 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.566943 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.606012 21858 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:46.607896 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.608157 21857 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:46.548375 23702 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:46.613588 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 84588.1us
I20260709 13:05:46.527242 23883 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:46.614744 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48944.2us
I20260709 13:05:46.615721 21856 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:05:46.621176 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.609200 23714 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.572556 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.631110 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.634995 21852 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
W20260709 13:05:46.633042 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60803.9us
I20260709 13:05:46.637367 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:46.633702 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 177299us
I20260709 13:05:46.639777 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:46.640080 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:46.641404 23879 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:46.641880 23879 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:46.642333 23879 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.649857 22588 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:46.652856 22556 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:46.653913 23589 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.654300 23589 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.654652 23589 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.655776 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.648440 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.651818 23879 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.660167 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:46.661052 22585 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.661895 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:46.663375 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 46650.8us
I20260709 13:05:46.664369 22588 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:46.663631 23888 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:46.664932 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:46.666091 23888 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.667781 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.669930 23888 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.670917 23879 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.671617 23879 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:46.672034 23879 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:46.672427 23879 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.679876 23715 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:46.680394 23715 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:46.680744 23715 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.615309 23649 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.615456 23873 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:46.682873 23649 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.683913 23873 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.621186 23728 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:05:46.686017 23733 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.629374 23714 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:46.686671 23714 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.688027 23831 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.687871 23715 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.690025 23715 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:46.691185 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1554) took 2347 ms. Trace:
I20260709 13:05:46.691702 21854 rpcz_store.cc:276] 0709 13:05:44.343538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.659938 (+1316400us) service_pool.cc:224] Handling call
0709 13:05:45.660304 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.981892 (+321588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.690890 (+708998us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54003d3e54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:46.691110 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737854720}
I20260709 13:05:46.586851 23720 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:46.700758 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 63481us
I20260709 13:05:46.576629 23786 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.713995 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.715755 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.718416 22587 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:46.718633 22593 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:46.724967 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.726439 22591 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:46.726820 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.727723 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.727939 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:46.728217 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:46.758909 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:46.729319 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.676599 22585 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:46.747622 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1503) took 2323 ms. Trace:
I20260709 13:05:46.748097 22589 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:46.748293 23601 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:46.748844 23589 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.749679 22000 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:46.709604 23717 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:46.750761 23880 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.271s	user 0.000s	sys 0.000s
I20260709 13:05:46.752175 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.798992 22590 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:46.766799 21934 rpcz_store.cc:276] 0709 13:05:44.412287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.435885 (+ 23598us) service_pool.cc:224] Handling call
0709 13:05:46.100922 (+1665037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.240599 (+139677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.735095 (+494496us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540003be50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:46.735242 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3828323584}
I20260709 13:05:46.632090 23883 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:46.771440 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:46.773491 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 111634us
I20260709 13:05:46.782544 21753 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 2, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 2 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.785413 23815 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:46.809875 22009 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:46.805119 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:46.811321 21854 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:46.805119 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:46.814468 22004 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:46.792695 21757 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 304) took 5787 ms. Trace:
I20260709 13:05:46.803733 21753 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 3, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.825289 23831 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:46.729048 23888 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:46.754743 23796 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:05:46.832796 23813 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:46.809373 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.631490 23833 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.164s
I20260709 13:05:46.717555 22586 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:46.769358 23649 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:46.769269 23592 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:46.770291 23771 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:46.871361 23771 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.871531 23855 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:46.845022 22588 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:46.795027 23882 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:46.872145 23585 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.770624 23601 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:46.725610 23749 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 13:05:46.717258 23876 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:05:46.752903 23892 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.127s
I20260709 13:05:46.810046 23815 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:46.854918 23636 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.857300 21853 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:46.754110 23837 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:05:46.621246 23783 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.808972 23835 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:05:46.799356 23604 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:46.711798 23889 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.219s
I20260709 13:05:46.871860 23771 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.839692 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:46.926770 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 87550.8us
W20260709 13:05:46.875936 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 259557us
I20260709 13:05:46.929066 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:46.930233 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:46.879457 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 149225us
I20260709 13:05:46.932847 23701 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.844338 23718 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.934077 23718 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.934448 23718 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.935389 23701 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.935958 23701 raft_consensus.cc:2764] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:05:46.886232 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 456752us
I20260709 13:05:46.939316 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.817428 21757 rpcz_store.cc:276] 0709 13:05:40.993455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.993726 (+   271us) service_pool.cc:224] Handling call
0709 13:05:46.780916 (+5787190us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:43.324001 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:43.420400 (+ 96399us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:43.433045 (+ 12645us) write_op.cc:435] Acquired schema lock
0709 13:05:43.436089 (+  3044us) tablet.cc:662] Decoding operations
0709 13:05:43.455848 (+ 19759us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:43.455911 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:43.455946 (+    35us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:05:43.456259 (+   313us) tablet.cc:701] Row locks acquired
0709 13:05:43.456281 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:05:43.456444 (+   163us) write_op.cc:270] Start()
0709 13:05:43.456564 (+   120us) write_op.cc:276] Timestamp: P: 1783602343456396 usec, L: 0
0709 13:05:43.456589 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:05:43.457309 (+   720us) log.cc:844] Serialized 987 byte log entry
0709 13:05:44.403160 (+945851us) op_driver.cc:464] REPLICATION: finished
0709 13:05:44.610280 (+207120us) write_op.cc:301] APPLY: starting
0709 13:05:44.617621 (+  7341us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:44.658161 (+ 40540us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:44.684372 (+ 26211us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:44.743986 (+ 59614us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:44.744192 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:05:44.744241 (+    49us) write_op.cc:312] APPLY: finished
0709 13:05:44.744463 (+   222us) log.cc:844] Serialized 29 byte log entry
0709 13:05:44.744644 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.744740 (+    96us) write_op.cc:454] Released schema lock
0709 13:05:44.744788 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:44.745725 (+   937us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.745766 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":460460,"reported_tablets":7,"spinlock_wait_cycles":306944,"child_traces":[["op",{"apply.queue_time_us":184904,"apply.run_cpu_time_us":2621,"apply.run_wall_time_us":140878,"num_ops":2,"prepare.queue_time_us":294991,"prepare.run_cpu_time_us":26664,"prepare.run_wall_time_us":566416,"raft.queue_time_us":412153,"raft.run_cpu_time_us":18105,"raft.run_wall_time_us":131323,"replication_time_us":946360,"thread_start_us":977899,"threads_started":4,"wal-append.queue_time_us":527842,"wal-append.run_cpu_time_us":31402,"wal-append.run_wall_time_us":1763786}]]}
I20260709 13:05:46.849035 23885 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:46.909646 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 244916us
I20260709 13:05:46.808530 23699 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.942209 23699 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.832823 23888 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.874038 23592 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.919019 23876 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:46.909152 23887 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.063s	user 0.003s	sys 0.000s
I20260709 13:05:46.921542 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1770) took 1308 ms. Trace:
I20260709 13:05:46.944891 23876 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.946657 23897 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:46.945441 22003 rpcz_store.cc:276] 0709 13:05:45.613568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.716004 (+102436us) service_pool.cc:224] Handling call
0709 13:05:46.437906 (+721902us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:46.438107 (+   201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.438464 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.438494 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.438984 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.609549 (+170565us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:46.609590 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.609654 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.921199 (+311545us) raft_consensus.cc:1685] finished
0709 13:05:46.921220 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.921465 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1783,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":9730944,"thread_start_us":170203,"threads_started":1,"wal-append.queue_time_us":421009}
W20260709 13:05:46.877938 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:46.947923 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.948328 23876 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.942101 23885 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.950480 23885 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.944773 23592 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:46.951179 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.951200 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:46.944566 23699 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:46.943797 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.953737 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.954219 23885 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:46.944566 23888 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.955056 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.955780 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.950373 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:46.955964 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:46.957392 21926 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:46.957737 23885 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.956413 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.958760 22591 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:05:46.917287 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 110752us
I20260709 13:05:46.959914 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:46.960071 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.960485 22587 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.961241 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.961872 23592 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.963119 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.964089 23592 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:46.965034 23885 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:46.965051 23897 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.965662 23897 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.965720 23885 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:46.966123 23897 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.966729 23885 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:46.967478 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:46.968371 21926 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:46.969393 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.970584 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:46.971755 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:46.971899 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:46.972499 22588 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:46.973377 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:46.974087 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:46.975404 21927 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:46.976181 21927 pending_rounds.cc:85] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 5
I20260709 13:05:46.976438 21927 pending_rounds.cc:107] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:46.976679 21927 raft_consensus.cc:2889] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:46.976856 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46697.4us
I20260709 13:05:46.977902 23592 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:46.964053 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.978360 23592 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:46.980798 23592 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:46.979820 23873 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:46.981750 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:46.978895 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:46.997130 22593 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:47.001379 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:46.916741 23855 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:47.026784 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 92873.1us
W20260709 13:05:47.005119 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45165.4us
W20260709 13:05:47.004411 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.032521 21974 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:46.920434 23604 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:46.986750 23783 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.037624 23883 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:46.999887 23713 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:47.039144 23589 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:47.038944 23904 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:47.042567 23636 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:47.004676 23802 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:05:47.044154 23854 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.002s
I20260709 13:05:47.092905 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.031628 23855 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:47.053432 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1743) took 1096 ms. Trace:
I20260709 13:05:47.054596 23886 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:47.056159 23874 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:47.030731 23717 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.064190 23879 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.072252 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.115435 22009 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:46.998965 23858 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:47.086045 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:47.030826 23889 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.064929 23905 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:47.069723 23907 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:47.109400 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.035385 23578 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.080420 23604 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.097517 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:47.029184 23764 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:05:47.094141 23749 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:47.150377 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:47.167768 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70356.9us
I20260709 13:05:47.067976 23868 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:05:47.110301 23886 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:47.170430 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:47.170171 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:47.171269 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 166243us
I20260709 13:05:47.194020 22003 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:47.029613 23897 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.029610 23601 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.122414 23903 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:47.235338 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.203135 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 399143us
I20260709 13:05:47.068244 23815 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:47.204623 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 222959us
I20260709 13:05:47.085242 23835 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:05:47.211216 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:47.253541 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 83208.3us
W20260709 13:05:47.236418 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 234890us
I20260709 13:05:47.270771 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.285579 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:47.285723 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:47.272740 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:47.287181 21930 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:47.272904 21935 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:47.142282 23906 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:47.143241 23585 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:47.296222 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.167438 23720 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:47.235469 23702 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.001s
I20260709 13:05:47.289448 22003 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:47.296030 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.296128 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.167879 23786 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:47.298586 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 266149us
I20260709 13:05:47.397297 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:47.197861 23782 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:47.279520 23712 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:47.277493 23819 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.174s
I20260709 13:05:47.269894 23858 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:47.418012 21932 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:47.400501 23873 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:47.430176 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.433619 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 142965us
I20260709 13:05:47.189646 23854 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.408001 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:47.479952 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.454408 23903 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:47.149501 23907 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.402927 23601 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.465440 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:47.429687 23832 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.018s	sys 0.020s
I20260709 13:05:47.522908 23782 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.110132 21853 rpcz_store.cc:276] 0709 13:05:45.956487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.975580 (+ 19093us) service_pool.cc:224] Handling call
0709 13:05:47.053152 (+1077572us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54000fbc54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:47.053347 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818685312}
I20260709 13:05:47.444888 23877 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.523695 23899 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:47.522523 23712 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.168730 23578 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:47.268543 23855 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:47.526544 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 68963.3us
W20260709 13:05:47.429235 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 362621us
I20260709 13:05:47.527426 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.529611 21926 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:47.592664 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.587157 23868 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:47.562665 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 273169us
I20260709 13:05:47.526041 21930 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:47.653406 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:47.656111 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.662997 21934 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:47.663509 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:47.664043 23902 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:05:47.663890 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.664636 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:47.667429 23713 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:47.667985 23713 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.494489 23702 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.660976 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:47.658573 21935 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:47.458244 23585 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:47.645697 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 360051us
W20260709 13:05:47.625880 23714 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.939s	user 0.012s	sys 0.000s
I20260709 13:05:47.675918 22001 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:47.677191 21855 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.565s	user 0.002s	sys 0.007s
I20260709 13:05:47.608335 23589 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:47.714530 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.520699 23906 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:47.627388 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.539925 21932 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:47.687971 21855 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:05:47.620582 23832 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.021s	sys 0.023s
I20260709 13:05:47.630534 23712 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.691933 22001 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:47.692747 21929 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:47.695060 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.700726 23912 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:47.707151 22000 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:47.714504 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:47.712479 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.787927 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.787770 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73317.6us
I20260709 13:05:47.520366 23701 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:47.722226 22592 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEARNER]: Advancing to term 3
I20260709 13:05:47.722730 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:47.678622 23776 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.169s	user 0.004s	sys 0.006s
I20260709 13:05:47.732344 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:47.789855 21932 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.455756 23886 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.757242 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.775586 23936 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:05:47.791481 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.792493 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:47.679629 23636 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.711859 23855 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:47.761848 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:47.760968 21901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.010s	sys 0.024s
W20260709 13:05:47.763149 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.764597 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.670828 23903 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.766837 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.813584 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.659752 23905 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:05:47.726643 23749 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:05:47.830847 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:47.767941 23916 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:05:47.726945 23589 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.773207 23922 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:47.774585 23931 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:47.842422 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:47.843238 21860 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.679281 23874 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:05:47.776189 23912 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:47.793654 22591 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:47.801620 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:47.778251 22000 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:47.803155 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:47.804509 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:47.851864 21859 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:47.797870 23636 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.809262 23932 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:05:47.888234 23907 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:47.816298 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:47.816511 23782 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:47.764735 23873 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.798159 23776 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.765513 23925 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.712462 23714 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:47.822484 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 203571us
W20260709 13:05:47.766033 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.800024 21901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.010s	sys 0.024s
I20260709 13:05:47.847914 21852 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.901831 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:47.899714 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:47.906915 21929 raft_consensus.cc:2368] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:47.907792 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.908349 21929 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:47.680275 23868 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.629889 23899 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.911123 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:47.884020 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.911682 21932 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:47.912074 21935 raft_consensus.cc:2368] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:47.885401 22592 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:05:47.817929 23879 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:47.853560 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 91865.9us
I20260709 13:05:47.886863 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.883129 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.814720 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:47.915400 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:47.915717 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:47.916007 22001 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 1.
I20260709 13:05:47.765813 23858 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.891899 23636 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.847324 23897 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:47.913648 22592 pending_rounds.cc:85] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 2
I20260709 13:05:47.689890 23585 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.882203 23886 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.905251 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:47.918208 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:47.927143 21859 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.923218 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:47.928790 21856 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:05:47.905790 23882 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.033s	user 0.009s	sys 0.002s
I20260709 13:05:47.929768 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.930040 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:47.924942 21930 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:47.815058 23701 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:47.925585 22592 pending_rounds.cc:107] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:05:47.934306 21852 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:47.920655 21860 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:47.932435 22592 raft_consensus.cc:2889] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:47.937094 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:47.922910 23771 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.920s	user 0.003s	sys 0.008s
I20260709 13:05:47.904417 23925 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:47.931309 23766 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:47.929366 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.914331 23832 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:48.003970 23728 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:48.005069 23728 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:48.005671 23728 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:48.006218 23728 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.921917 22000 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.010473 22000 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:48.012645 23592 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.007s	user 0.002s	sys 0.005s
I20260709 13:05:48.013293 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.944962 23766 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.014514 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:48.015550 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:48.015821 23766 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.016525 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:47.817342 23906 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:47.924731 23823 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:48.067710 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46721.3us
I20260709 13:05:47.938578 23854 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:47.938714 23782 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.942555 23882 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.947320 23925 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.945230 21929 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:48.133052 22590 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:48.131597 23921 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:48.143182 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.091410 23915 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:48.081715 23854 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:48.146883 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.154745 22006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.152s	user 0.000s	sys 0.028s
W20260709 13:05:48.155315 22006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.000s	sys 0.029s
I20260709 13:05:48.094102 23823 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:47.964890 23771 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.064505 23592 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.207317 23854 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.108682 23782 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:47.933183 23912 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:48.281025 22593 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.032s	sys 0.052s
W20260709 13:05:48.281567 22593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.032s	sys 0.052s
I20260709 13:05:47.942154 23938 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:48.008723 23713 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.276852 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:48.302400 21927 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:48.273507 23883 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:48.324002 23713 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:48.332160 23855 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.000s
I20260709 13:05:48.337203 23823 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.340137 23771 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:48.342674 23718 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.408s	user 0.003s	sys 0.009s
I20260709 13:05:48.423440 22009 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:48.345293 22000 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:48.305711 23912 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:48.421880 23578 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:48.421815 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:48.408015 23783 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.326s	user 0.003s	sys 0.009s
I20260709 13:05:48.447153 22008 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:48.456024 23589 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:48.405229 23813 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.424803 23604 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.251s	user 0.009s	sys 0.003s
I20260709 13:05:48.419746 23585 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.464802 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1755) took 1899 ms. Trace:
I20260709 13:05:48.473472 22002 rpcz_store.cc:276] 0709 13:05:46.565748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.565875 (+   127us) service_pool.cc:224] Handling call
0709 13:05:46.566265 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.600520 (+ 34255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.600551 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.600993 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.845430 (+244437us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b18003c3d60. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:05:47.082548 (+237118us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:47.082594 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.082660 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.422571 (+1339911us) raft_consensus.cc:1685] finished
0709 13:05:48.422592 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.464711 (+ 42119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7835,"raft.queue_time_us":34552,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":12047,"spinlock_wait_cycles":9961216,"thread_start_us":236908,"threads_started":1,"wal-append.queue_time_us":1236135}
W20260709 13:05:48.470898 23929 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.435s	user 0.000s	sys 0.001s
I20260709 13:05:48.424330 23883 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:48.423895 23701 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.426021 23776 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.425738 23702 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:48.465476 23778 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:05:48.453747 23796 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.457852 23868 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:48.444934 23718 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.474041 23604 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.478361 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1761) took 1602 ms. Trace:
W20260709 13:05:48.497190 21859 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
W20260709 13:05:48.502408 23890 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
W20260709 13:05:48.503410 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 81686.7us
W20260709 13:05:48.503636 21934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.024s	sys 0.063s
W20260709 13:05:48.505280 21934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.025s	sys 0.063s
I20260709 13:05:48.505518 21859 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:05:48.500783 23601 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.876s	user 0.006s	sys 0.006s
I20260709 13:05:48.472934 23813 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.500290 23899 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:48.506505 23712 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.000s
I20260709 13:05:48.499946 21929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:48.507318 23776 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:48.503532 23604 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.553277 23717 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:48.553782 23717 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:48.554682 23717 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.493949 23796 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.542635 23714 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:48.544183 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1527) took 1574 ms. Trace:
W20260709 13:05:48.546262 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 888817us
W20260709 13:05:48.548487 23728 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.002s
W20260709 13:05:48.540798 23908 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.132s	user 0.007s	sys 0.023s
W20260709 13:05:48.563975 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.515580 23592 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.529474 21851 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:05:48.590955 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.591172 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 473) took 1784 ms. Trace:
I20260709 13:05:48.591557 22590 rpcz_store.cc:276] 0709 13:05:46.806331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.806462 (+   131us) service_pool.cc:224] Handling call
0709 13:05:46.844832 (+ 38370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.133822 (+1288990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.590918 (+457096us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:48.591102 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2962027392}
I20260709 13:05:48.565554 23604 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:48.530485 23771 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:05:48.603097 23815 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:48.538229 23712 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.500648 22000 rpcz_store.cc:276] 0709 13:05:46.871907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.884509 (+ 12602us) service_pool.cc:224] Handling call
0709 13:05:47.025200 (+140691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.706962 (+681762us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b54002d5120. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:05:48.445937 (+738975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.473808 (+ 27871us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10071936}
I20260709 13:05:48.606185 23728 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:48.651427 22003 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.012s	sys 0.012s
W20260709 13:05:48.652282 21931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.033s	sys 0.036s
W20260709 13:05:48.652724 21931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.034s	sys 0.036s
W20260709 13:05:48.692512 21854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.016s	sys 0.008s
W20260709 13:05:48.693220 21854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.016s	sys 0.009s
W20260709 13:05:48.696344 22003 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.013s	sys 0.012s
W20260709 13:05:48.721158 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.006s	sys 0.023s
W20260709 13:05:48.722381 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.007s	sys 0.023s
I20260709 13:05:48.724568 23601 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:48.605999 23604 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.585314 21926 rpcz_store.cc:276] 0709 13:05:46.969514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.970336 (+   822us) service_pool.cc:224] Handling call
0709 13:05:48.543908 (+1573572us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540017fa54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:48.544106 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985101568}
W20260709 13:05:48.727290 21901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.042s	sys 0.059s
I20260709 13:05:48.725593 23578 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.718863 23913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.023s	sys 0.032s
I20260709 13:05:48.567667 23883 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:48.739616 21901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.042s	sys 0.059s
I20260709 13:05:48.726225 23601 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.740413 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1758) took 2113 ms. Trace:
W20260709 13:05:48.728580 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 937159us
I20260709 13:05:48.744736 22007 rpcz_store.cc:276] 0709 13:05:46.613705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.614043 (+   338us) service_pool.cc:224] Handling call
0709 13:05:46.614351 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.614703 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.614743 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.618729 (+  3986us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.927180 (+308451us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:46.927219 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.927283 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.536604 (+1609321us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:05:48.711938 (+175334us) raft_consensus.cc:1685] finished
0709 13:05:48.711961 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.727108 (+ 15147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4309,"raft.queue_time_us":1935,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":3915,"spinlock_wait_cycles":69793024,"thread_start_us":173325,"threads_started":1,"wal-append.queue_time_us":463398}
I20260709 13:05:48.712301 23592 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.604534 23925 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:48.701555 23936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.008s	sys 0.021s
W20260709 13:05:48.721607 23932 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.015s	sys 0.015s
W20260709 13:05:48.748061 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.027s	sys 0.016s
W20260709 13:05:48.743323 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.017s	sys 0.030s
I20260709 13:05:48.833945 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1817) took 1314 ms. Trace:
W20260709 13:05:48.743359 23913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.023s	sys 0.032s
W20260709 13:05:48.828599 22559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.016s	sys 0.008s
I20260709 13:05:48.741485 23855 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.838323 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.017s	sys 0.036s
W20260709 13:05:48.744073 23831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.018s	sys 0.035s
W20260709 13:05:48.838200 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.027s	sys 0.021s
W20260709 13:05:48.846812 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 931088us
I20260709 13:05:48.837098 21931 rpcz_store.cc:276] 0709 13:05:47.520005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.522734 (+  2729us) service_pool.cc:224] Handling call
0709 13:05:47.523196 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.523529 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.523557 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.621979 (+ 98422us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.548212 (+926233us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4800241ed8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:05:48.653252 (+105040us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:48.653311 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.653410 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.814819 (+161409us) raft_consensus.cc:1685] finished
0709 13:05:48.814848 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.833842 (+ 18994us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":189669,"raft.run_cpu_time_us":2954,"raft.run_wall_time_us":834438,"spinlock_wait_cycles":64270592,"thread_start_us":941514,"threads_started":1,"wal-append.queue_time_us":950747}
W20260709 13:05:48.838889 22559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.016s	sys 0.008s
I20260709 13:05:48.750828 23718 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:48.851450 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.006s	sys 0.016s
I20260709 13:05:48.805466 23876 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.788981 23712 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:48.740777 23854 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:48.813055 23921 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.000s	sys 0.015s
W20260709 13:05:48.828863 23874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.012s	sys 0.012s
W20260709 13:05:48.876297 23874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.012s	sys 0.014s
I20260709 13:05:48.878240 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:48.859820 23728 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:48.840278 23932 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.016s	sys 0.021s
W20260709 13:05:48.852082 23831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.021s	sys 0.038s
W20260709 13:05:48.840220 23936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.015s	sys 0.028s
W20260709 13:05:48.868561 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.008s	sys 0.011s
I20260709 13:05:48.744807 23578 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:48.869884 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.007s	sys 0.016s
W20260709 13:05:48.871399 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 941346us
I20260709 13:05:48.895239 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:48.895480 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 958397us
I20260709 13:05:48.850832 23813 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.879518 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:48.883169 23829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.007s	sys 0.024s
I20260709 13:05:48.753378 23592 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:48.785424 23888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.000s	sys 0.025s
I20260709 13:05:48.882251 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:48.845171 23855 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.888669 22584 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:48.889227 23953 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:05:48.871086 23921 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.001s	sys 0.018s
I20260709 13:05:48.893748 22591 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:48.901469 21859 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:05:48.938597 21859 pending_rounds.cc:85] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 2
I20260709 13:05:48.938877 21859 pending_rounds.cc:107] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:05:48.939108 21859 raft_consensus.cc:2889] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:48.895275 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.008s	sys 0.011s
I20260709 13:05:48.902334 22587 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:48.939724 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 871217us
W20260709 13:05:48.904953 23829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.007s	sys 0.025s
I20260709 13:05:48.869519 23876 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:48.890113 23944 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:05:48.904500 22556 connection.cc:256] RPC call timeout handler was delayed by -1.9550302559999864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 13:05:48.880405 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:48.902045 23712 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.922994 23888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.000s	sys 0.026s
I20260709 13:05:48.926731 23813 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.934113 23714 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.965483 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 86102.4us
I20260709 13:05:48.969033 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:48.971715 23899 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.934302 23856 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:48.972328 23899 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:48.949707 23921 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:49.017993 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61530.8us
W20260709 13:05:49.049746 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.045660 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:49.095095 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.035934 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:49.092141 22556 connection.cc:256] RPC call timeout handler was delayed by -1.9149436540000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
I20260709 13:05:49.031044 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.092113 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1781) took 1889 ms. Trace:
W20260709 13:05:49.097232 22556 connection.cc:256] RPC call timeout handler was delayed by -1.2813436540000112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2801s for a response
W20260709 13:05:49.098222 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1001858000000766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 13:05:49.097445 22003 rpcz_store.cc:276] 0709 13:05:47.202774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.814493 (+611719us) service_pool.cc:224] Handling call
0709 13:05:47.814810 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.716010 (+901200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.716049 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.741310 (+ 25261us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.903845 (+162535us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:48.903894 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.903962 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.091827 (+187865us) raft_consensus.cc:1685] finished
0709 13:05:49.091858 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.092039 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19968,"thread_start_us":987626,"threads_started":2,"wal-append.queue_time_us":324188}
I20260709 13:05:48.961123 23718 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.105342 22556 leader_election.cc:400] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:49.147282 22556 leader_election.cc:403] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:05:49.148566 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117626us
I20260709 13:05:49.149557 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:48.990044 22587 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:48.975505 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:48.966626 23873 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.079622 23874 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.153338 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1762) took 2278 ms. Trace:
I20260709 13:05:49.172071 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.172922 22587 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:49.153728 22591 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:49.158867 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.176577 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.178970 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.180240 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:49.088704 23886 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.157s	user 0.011s	sys 0.001s
I20260709 13:05:48.973404 23932 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:49.087765 23636 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.111s	user 0.005s	sys 0.005s
I20260709 13:05:49.169600 23924 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:49.045189 21853 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:48.977600 23946 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:05:49.090200 23888 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.221616 23888 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.224364 23888 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.172216 22006 rpcz_store.cc:276] 0709 13:05:46.874511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.875602 (+  1091us) service_pool.cc:224] Handling call
0709 13:05:46.876002 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.876401 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.876441 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.893104 (+ 16663us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.422904 (+1529800us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:48.422949 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.423018 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.153102 (+730084us) raft_consensus.cc:1685] finished
0709 13:05:49.153129 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.153273 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2696,"raft.queue_time_us":8400,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":18442,"thread_start_us":1152674,"threads_started":1,"wal-append.queue_time_us":1827421}
I20260709 13:05:49.083532 23601 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.225822 23601 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:49.225981 23831 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.226819 23831 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.226480 23601 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.227252 23831 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.227706 23831 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.021473 21929 raft_consensus.cc:606] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:49.229238 21929 raft_consensus.cc:614] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:49.229565 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.230162 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 528) took 1510 ms. Trace:
I20260709 13:05:49.231693 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.230660 22593 rpcz_store.cc:276] 0709 13:05:47.720220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.721088 (+   868us) service_pool.cc:224] Handling call
0709 13:05:47.721497 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.721907 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.721946 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.727827 (+  5881us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.282018 (+554191us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:48.282074 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.282165 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.229868 (+947703us) raft_consensus.cc:1685] finished
0709 13:05:49.229896 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.230084 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30622,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7468,"thread_start_us":553787,"threads_started":1,"wal-append.queue_time_us":1013233}
I20260709 13:05:49.086912 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.080967 23578 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.086400 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:49.085052 23855 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:48.976888 23714 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.182933 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 398) took 1987 ms. Trace:
I20260709 13:05:49.250260 22004 rpcz_store.cc:276] 0709 13:05:47.195489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.195709 (+   220us) service_pool.cc:224] Handling call
0709 13:05:47.196067 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.653320 (+457253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.653357 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:47.653403 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:47.653432 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.182650 (+1529218us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400168854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:49.182709 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.182860 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051079424}
W20260709 13:05:49.190716 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 221790us
W20260709 13:05:49.159597 23950 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
W20260709 13:05:49.154297 23823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.009s	sys 0.016s
I20260709 13:05:49.020548 23604 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.045430 23925 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:48.966189 23829 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:49.160178 23966 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:05:49.160167 23961 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:05:49.212654 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:49.172701 23945 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:05:49.145933 23716 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:49.160205 23959 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:05:49.170890 23948 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:05:48.955544 23953 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.161068 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.160156 23943 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.219729 23958 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.219894 23881 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:49.220394 23962 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.139s
I20260709 13:05:48.964860 23712 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.289323 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 307) took 1573 ms. Trace:
I20260709 13:05:49.283840 23829 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.289824 23592 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.201401 23879 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.289737 21934 rpcz_store.cc:276] 0709 13:05:47.715832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.715957 (+   125us) service_pool.cc:224] Handling call
0709 13:05:47.716230 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.716584 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.716620 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.754682 (+ 38062us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.505682 (+751000us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:48.505723 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.505791 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.289115 (+783324us) raft_consensus.cc:1685] finished
0709 13:05:49.289135 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.289267 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36186,"raft.queue_time_us":60286,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":27529,"thread_start_us":714421,"threads_started":1,"wal-append.queue_time_us":1009992}
I20260709 13:05:49.290925 23714 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.291360 23829 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:49.237802 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 263524us
I20260709 13:05:49.291664 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.293431 23942 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:49.293969 23942 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:49.294224 23892 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.294486 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:49.294759 23892 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.295131 23907 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.161617 23728 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:49.245012 22584 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 2: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:05:49.244760 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 276822us
I20260709 13:05:49.290946 23712 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.305657 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.252429 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:49.306942 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 54639.7us
I20260709 13:05:49.307818 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:49.255754 23921 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:49.309655 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 717644us
W20260709 13:05:49.309752 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 563960us
I20260709 13:05:49.310567 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:49.258674 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 314) took 1489 ms (client timeout 1618 ms). Trace:
I20260709 13:05:49.151532 23856 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.161101 23718 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:49.238301 23912 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.780s	user 0.011s	sys 0.002s
I20260709 13:05:49.217290 23932 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.322697 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 400) took 1918 ms. Trace:
I20260709 13:05:49.273361 21931 raft_consensus.cc:2368] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
W20260709 13:05:49.265398 23823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.010s	sys 0.017s
I20260709 13:05:49.217782 23886 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:05:49.216328 23636 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.182210 23873 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.241848 23888 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:05:49.283406 23716 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.287891 23954 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:49.286610 23958 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.288117 22585 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:49.288902 23943 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:48.973815 23813 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.298588 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1786) took 1625 ms. Trace:
I20260709 13:05:49.346599 22001 rpcz_store.cc:276] 0709 13:05:47.673513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.920829 (+247316us) service_pool.cc:224] Handling call
0709 13:05:47.921197 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.921436 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:47.932138 (+ 10702us) log.cc:844] Serialized 14 byte log entry
0709 13:05:49.298259 (+1366121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:49.298310 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:49.298350 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.298406 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.298520 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1405226,"prepare.queue_time_us":10536,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":1367001,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":49}
I20260709 13:05:49.298916 23876 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:49.299726 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 201894us
I20260709 13:05:49.299861 23899 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.300071 23924 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:49.298277 23903 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:49.301510 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.302011 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.360850 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.301862 23953 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.296120 23907 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:49.302958 23964 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:05:49.292825 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.365343 21934 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:49.308383 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1770) took 1290 ms. Trace:
I20260709 13:05:49.313366 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.237159 23854 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.366405 22009 rpcz_store.cc:276] 0709 13:05:48.017940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.424562 (+406622us) service_pool.cc:224] Handling call
0709 13:05:48.424906 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.425190 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:48.502600 (+ 77410us) log.cc:844] Serialized 14 byte log entry
0709 13:05:49.308036 (+805436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:49.308090 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:49.308129 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.308173 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.308306 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":804534,"prepare.queue_time_us":76985,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":806006,"raft.run_cpu_time_us":598,"raft.run_wall_time_us":10641}
I20260709 13:05:49.331291 23932 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.324448 22005 rpcz_store.cc:276] 0709 13:05:47.404035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.819580 (+415545us) service_pool.cc:224] Handling call
0709 13:05:47.819903 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.820147 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:47.849805 (+ 29658us) log.cc:844] Serialized 14 byte log entry
0709 13:05:49.286898 (+1437093us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b1800418a80. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:05:49.322305 (+ 35407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:49.322367 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:49.322417 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.322481 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.322623 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1012,"prepare.queue_time_us":29111,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1449089,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":69808896,"thread_start_us":35117,"threads_started":1}
I20260709 13:05:49.308821 23962 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.309769 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.315258 23712 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:49.315092 23874 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:05:49.312095 21927 rpcz_store.cc:269] 0709 13:05:47.769108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.883784 (+114676us) service_pool.cc:224] Handling call
0709 13:05:49.258435 (+1374651us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:49.258606 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766745984}
W20260709 13:05:49.340863 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 160705us
I20260709 13:05:49.329519 22586 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:49.382762 23907 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:05:49.334087 23815 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.004s
I20260709 13:05:49.351331 21935 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:49.294866 23942 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.372834 23813 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.398214 22556 leader_election.cc:400] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:49.393864 23877 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:49.398746 22556 leader_election.cc:403] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:49.398942 23877 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:49.376564 21860 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:49.377595 21857 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:49.377982 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:49.402099 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:49.379036 21858 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:49.404458 21858 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:49.379683 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.405435 22590 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:49.376610 21853 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:49.393850 23856 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.393718 23899 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:49.410780 23873 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.410964 23899 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:49.411327 23899 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:05:49.407984 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.801s (SENT) [suppressed 8 similar messages]
I20260709 13:05:49.408489 21852 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:49.411667 23952 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:05:49.349821 23892 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.418501 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.412398 23903 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:49.419289 22589 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:49.419420 23716 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:49.348141 23951 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:05:49.419977 23903 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.421471 23884 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:05:49.424540 23964 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:05:49.385370 23924 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:05:49.425611 23601 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.426961 21930 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:49.383158 23954 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:49.412207 23899 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.441804 23782 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.409093 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.452456 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 321) took 1681 ms. Trace:
I20260709 13:05:49.458930 21930 rpcz_store.cc:276] 0709 13:05:47.771111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.562859 (+791748us) service_pool.cc:224] Handling call
0709 13:05:49.426539 (+863680us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:49.426743 (+   204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.452079 (+ 25336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.452383 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6644096}
I20260709 13:05:49.454553 23899 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.454421 23954 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.408129 23881 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3.
I20260709 13:05:49.468281 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 312) took 1699 ms. Trace:
I20260709 13:05:49.469070 21935 rpcz_store.cc:276] 0709 13:05:47.768479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.016297 (+247818us) service_pool.cc:224] Handling call
0709 13:05:49.468093 (+1451796us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:49.468224 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3053963008}
I20260709 13:05:49.457595 23636 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.449575 23964 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:49.399972 23877 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.473662 23964 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.476758 23718 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.476895 23964 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.485342 23943 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:49.487054 23985 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:05:49.507368 23717 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:49.511490 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1578) took 2923 ms (client timeout 1863 ms). Trace:
W20260709 13:05:49.511976 21855 rpcz_store.cc:269] 0709 13:05:46.588443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.588629 (+   186us) service_pool.cc:224] Handling call
0709 13:05:49.511288 (+2922659us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b540008a2d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:49.511411 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171245440}
I20260709 13:05:49.512818 23831 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.517326 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 320) took 1746 ms. Trace:
I20260709 13:05:49.532121 21932 rpcz_store.cc:276] 0709 13:05:47.770786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.559748 (+788962us) service_pool.cc:224] Handling call
0709 13:05:48.560166 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.560524 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.560553 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.563241 (+  2688us) log.cc:844] Serialized 26 byte log entry
0709 13:05:49.017613 (+454372us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:49.017655 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.017740 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.516934 (+499194us) raft_consensus.cc:1685] finished
0709 13:05:49.516955 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.517102 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5679,"raft.queue_time_us":2086,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":20622,"thread_start_us":420210,"threads_started":1,"wal-append.queue_time_us":890439}
I20260709 13:05:49.519344 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48000 (request call id 16) took 1733 ms. Trace:
I20260709 13:05:49.521411 23907 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.544684 23907 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:49.525041 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1759) took 2831 ms. Trace:
W20260709 13:05:49.540746 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.549762 22008 rpcz_store.cc:276] 0709 13:05:46.693625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.694874 (+  1249us) service_pool.cc:224] Handling call
0709 13:05:46.695316 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.447619 (+1752303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.524709 (+1077090us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400167954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:49.524960 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4027813760}
I20260709 13:05:49.540620 21929 rpcz_store.cc:276] 0709 13:05:47.786376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.478394 (+692018us) service_pool.cc:224] Handling call
0709 13:05:49.519113 (+1040719us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b5400097ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707c9853 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691 00007f3b6ed1d60c 00007f3b6ed1c8c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:49.519259 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047635840}
I20260709 13:05:49.539914 23932 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:49.557386 23973 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.311s	user 0.001s	sys 0.001s
I20260709 13:05:49.562031 23796 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:49.563880 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1752) took 1759 ms. Trace:
I20260709 13:05:49.564447 21851 rpcz_store.cc:276] 0709 13:05:47.803820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.813923 (+ 10103us) service_pool.cc:224] Handling call
0709 13:05:47.887040 (+ 73117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.594994 (+707954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.563181 (+968187us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54000596d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:49.563578 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420020608}
W20260709 13:05:49.592840 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 184858us
W20260709 13:05:49.594417 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.594659 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 281351us
I20260709 13:05:49.595338 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.596158 23712 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.596828 23712 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:49.596189 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 220782us
I20260709 13:05:49.598269 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:49.599532 22592 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:49.599462 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:49.601225 23720 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:49.601887 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:49.602172 23577 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:49.602494 22593 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.602674 23577 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:49.603214 23577 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:49.605039 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:49.606204 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 295679us
I20260709 13:05:49.606967 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.608436 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.610680 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.612329 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.613490 22590 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:49.600677 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:49.620371 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:49.621217 22589 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:49.622416 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:49.632042 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.628266 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:49.633749 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.633874 22004 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:05:49.638150 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 163071us
W20260709 13:05:49.642608 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.632630 23835 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:49.601909 23720 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:49.615329 23712 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.649353 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:49.630693 23974 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:49.615197 23879 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.630548 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:49.625272 23718 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:05:49.636839 23831 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.637970 23936 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.638188 23782 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:49.717912 22585 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:49.637904 23601 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.630726 23912 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:05:49.722158 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.696163 22586 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:49.648504 23876 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:05:49.678967 23749 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.701170 23933 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:49.701608 23980 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:49.703624 23714 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.699839 23932 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260709 13:05:49.706034 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 390501us
W20260709 13:05:49.707317 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.731003 23925 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.644676 23926 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:05:49.696015 23990 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:49.717161 23835 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:49.681598 23636 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.693475 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46727.4us
I20260709 13:05:49.731323 22593 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:49.731374 21934 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:49.728178 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:49.725521 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.663496 23988 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.735972 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.737465 21935 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:49.749392 23717 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:49.718514 23905 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.717208 23911 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.707672 23802 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:49.719013 23702 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:49.732280 23819 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.734019 23720 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.762518 23981 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.762948 23927 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.763314 23941 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:49.736755 23783 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:49.793020 23717 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:49.740583 23749 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.720882 23829 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:05:49.723013 23899 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.834128 23899 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.834669 23899 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.709205 23879 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.793555 23713 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:05:49.757694 23951 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.752100 23933 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:49.761190 23796 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.790781 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.768857 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1748) took 1967 ms. Trace:
I20260709 13:05:49.794998 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:49.796072 22009 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:49.844350 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.790864 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.791601 22591 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:49.791368 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:49.843653 21856 rpcz_store.cc:276] 0709 13:05:47.801845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.801962 (+   117us) service_pool.cc:224] Handling call
0709 13:05:47.802335 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.929297 (+126962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.768589 (+1839292us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400058cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:49.768792 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290235264}
W20260709 13:05:49.845770 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 54498.5us
W20260709 13:05:49.792595 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64546.2us
I20260709 13:05:49.790962 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.792177 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.789498 23922 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:49.789827 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.804504 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.733772 23601 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.796464 23988 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.764585 23990 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:49.826249 23949 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:05:49.764348 23952 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:49.717828 23936 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.765378 23925 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.792516 23989 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.772716 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.843295 23876 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.801451 23835 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.851054 22593 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:49.853498 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.854167 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.873061 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.855458 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.856973 21930 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:49.875221 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:49.803435 23802 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.807017 23702 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.798894 23636 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.860154 23937 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:05:49.829278 23783 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:49.868093 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 95471.8us
I20260709 13:05:49.882149 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.801226 23905 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.858171 23932 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:49.813934 23819 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.876623 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:49.852546 23922 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:49.880226 21934 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:49.883708 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:49.863655 23988 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.843786 23951 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.856774 23912 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:05:49.857424 23796 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.894641 21933 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:49.869432 23749 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:49.863551 23717 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.896075 23932 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:49.881228 23989 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.881209 23837 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:49.903276 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:49.887687 23937 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:49.878979 23876 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.869355 23952 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.822100 23911 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:49.842188 23927 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:49.949923 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46752.9us
I20260709 13:05:49.952167 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:49.912072 23577 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.913139 23749 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.915681 23937 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:49.934309 23713 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.916644 23989 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.975445 23951 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:49.905284 23922 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.835152 23941 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:49.964826 23717 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:49.989046 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.016s	sys 0.036s
I20260709 13:05:49.831291 23981 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:49.979691 21932 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:49.897231 23933 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:49.991681 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.016s	sys 0.037s
W20260709 13:05:49.995927 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 646712us
I20260709 13:05:49.998878 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:49.973194 23949 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:05:49.988260 23749 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:49.995837 23713 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:50.036103 21928 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:05:49.993655 23951 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:50.037556 23925 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:49.902048 23990 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:50.004742 23954 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.015s	sys 0.028s
I20260709 13:05:50.027295 23937 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:50.030826 23879 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:50.006001 23921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.009s	sys 0.032s
W20260709 13:05:50.052610 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.035995 23802 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:50.036332 23749 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:50.004201 23903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.019s	sys 0.029s
I20260709 13:05:50.030623 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:50.028272 23952 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:50.028350 23876 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:49.998884 23922 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:50.008395 23831 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:49.997130 23964 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:05:50.112607 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:50.114439 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:50.119613 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:50.044943 23713 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:50.124771 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:50.128172 21855 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:50.047583 23951 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:05:50.050633 23921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.009s	sys 0.033s
W20260709 13:05:50.048033 23954 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.016s	sys 0.029s
I20260709 13:05:50.118718 23906 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.118034 23702 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:50.122718 23937 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:50.121652 23831 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:50.011111 23932 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:50.133910 23990 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:50.153160 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.142729 23783 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:50.157085 23927 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:50.157966 23905 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:50.138723 23636 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:50.125952 23881 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:05:50.164273 23881 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:50.164672 23881 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:50.121115 23986 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.115216 23903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.021s	sys 0.031s
I20260709 13:05:50.134756 23601 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:50.181166 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.137038 23958 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:50.151108 23831 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.172070 23713 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:50.172541 23749 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:50.148093 23906 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:50.301252 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.307559 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:50.317685 23981 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:50.326141 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:50.312568 23954 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:05:50.338629 23954 raft_consensus.cc:1068] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:05:50.189622 23986 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:50.360558 23877 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:50.278782 23819 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:50.381954 21855 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:50.384409 21859 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:50.392321 22005 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.017s	sys 0.036s
W20260709 13:05:50.397342 22005 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.017s	sys 0.036s
I20260709 13:05:50.349226 23954 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.314369 23911 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:50.421038 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.291173 23951 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:50.429445 23899 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.007s
I20260709 13:05:50.424487 23981 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.416334 22585 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:50.432765 23858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.041s	sys 0.071s
I20260709 13:05:50.348177 23881 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:50.408715 23988 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:50.440493 23942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.041s	user 0.020s	sys 0.042s
W20260709 13:05:50.483640 23926 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.028s	sys 0.056s
W20260709 13:05:50.525900 23926 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.029s	sys 0.057s
I20260709 13:05:50.526816 23868 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:50.480968 23858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.041s	sys 0.071s
W20260709 13:05:50.503069 23717 consensus_meta.cc:287] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.511s	user 0.017s	sys 0.001s
I20260709 13:05:50.425508 23954 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:50.459524 23981 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:50.504006 23980 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.765s	user 0.009s	sys 0.001s
W20260709 13:05:50.531879 23942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.021s	sys 0.042s
I20260709 13:05:50.453155 23899 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:50.544435 23997 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.078s	user 0.003s	sys 0.003s
I20260709 13:05:50.534921 23986 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:50.544840 23906 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:50.554490 23955 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.164s	user 0.005s	sys 0.028s
I20260709 13:05:50.544204 23980 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:50.667879 23999 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:05:50.878296 23989 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.887s	user 0.013s	sys 0.000s
I20260709 13:05:50.880384 23981 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:50.878909 23989 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:50.842597 23995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.000s	sys 0.008s
W20260709 13:05:50.908133 23995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.000s	sys 0.011s
W20260709 13:05:50.858878 23977 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.004s	sys 0.004s
W20260709 13:05:50.919601 23977 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.004s	sys 0.005s
I20260709 13:05:51.075214 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260709 13:05:51.080919 21823 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.391s	user 0.015s	sys 0.030s
W20260709 13:05:51.081566 21823 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.392s	user 0.015s	sys 0.030s
W20260709 13:05:51.082162 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.39315e+06us
W20260709 13:05:51.083459 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.086236 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.137550 21974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.002s	sys 0.015s
W20260709 13:05:51.136205 21931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.000s	sys 0.004s
W20260709 13:05:51.158896 21974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.356s	user 0.002s	sys 0.022s
W20260709 13:05:51.159183 21931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.000s	sys 0.011s
W20260709 13:05:51.159494 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 1.40615e+06us
I20260709 13:05:51.087013 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:51.141311 22592 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:51.153939 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:51.161288 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:51.162539 21929 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:51.163344 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1835) took 1429 ms (client timeout 1832 ms). Trace:
W20260709 13:05:51.141270 22586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.007s	sys 0.010s
W20260709 13:05:51.164764 22586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.120s	user 0.014s	sys 0.010s
I20260709 13:05:51.098850 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:51.137247 21973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.387s	user 0.000s	sys 0.035s
W20260709 13:05:51.166102 21973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.416s	user 0.000s	sys 0.042s
I20260709 13:05:51.166672 21858 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:51.168673 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1843) took 1368 ms (client timeout 1704 ms). Trace:
W20260709 13:05:51.170243 23720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.260s	user 0.015s	sys 0.041s
W20260709 13:05:51.170773 23720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.261s	user 0.015s	sys 0.041s
I20260709 13:05:51.171293 23771 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:51.173199 23819 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.174721 23813 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:51.159711 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1810) took 3885 ms. Trace:
I20260709 13:05:51.168552 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:51.163889 21929 rpcz_store.cc:269] 0709 13:05:49.733654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.763391 (+ 29737us) service_pool.cc:224] Handling call
0709 13:05:51.160742 (+1397351us) spinlock_profiling.cc:243] Waited 957 us on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:51.163266 (+  2524us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201472}
I20260709 13:05:51.203437 21928 rpcz_store.cc:276] 0709 13:05:47.273804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.273970 (+   166us) service_pool.cc:224] Handling call
0709 13:05:47.274385 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.036682 (+2762297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.159007 (+1122325us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b54003075d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:51.159501 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6350569728}
I20260709 13:05:51.161312 21852 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:51.174368 23819 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:51.163621 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:51.256948 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:51.107911 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:51.181777 21933 rpcz_store.cc:269] 0709 13:05:49.800200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.844095 (+ 43895us) service_pool.cc:224] Handling call
0709 13:05:51.168202 (+1324107us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b54004290d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:51.168470 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114132736}
I20260709 13:05:51.090516 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:51.271162 23855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.200s	user 0.008s	sys 0.037s
I20260709 13:05:51.271385 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:51.271818 23855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.008s	sys 0.038s
I20260709 13:05:51.272888 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:51.274331 21859 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:51.277159 23835 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.007s	sys 0.001s
W20260709 13:05:51.277823 23835 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.007s	sys 0.001s
I20260709 13:05:51.278934 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:51.280045 21928 raft_consensus.cc:2376] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5. Re-sending same reply.
I20260709 13:05:51.156983 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:51.265923 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50433.2us
I20260709 13:05:51.306234 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:51.089663 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:51.295205 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 130351us
W20260709 13:05:51.301755 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 65100.7us
W20260709 13:05:51.320945 22559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.003s	sys 0.006s
I20260709 13:05:51.153733 23577 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:51.297930 23912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.345s	user 0.014s	sys 0.048s
I20260709 13:05:51.300325 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1827) took 1579 ms. Trace:
I20260709 13:05:51.332057 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:51.327051 22559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.004s	sys 0.006s
I20260709 13:05:51.289999 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:51.328043 21858 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 1.
I20260709 13:05:51.332158 23771 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.336777 23912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.429s	user 0.015s	sys 0.048s
I20260709 13:05:51.363188 21926 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:51.349496 22556 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.001s	sys 0.015s
I20260709 13:05:51.361881 23771 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:51.368470 23903 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 13:05:51.364028 22005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.002s	sys 0.009s
W20260709 13:05:51.363575 21851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.000s	sys 0.030s
W20260709 13:05:51.364549 23873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.636s	user 0.043s	sys 0.090s
W20260709 13:05:51.364933 22593 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.005s	sys 0.003s
I20260709 13:05:51.363139 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:51.354262 21857 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:51.361797 23911 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:51.370723 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:51.363034 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:51.379853 21860 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:05:51.387511 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.390504 22593 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.007s	sys 0.003s
W20260709 13:05:51.382709 22556 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.002s	sys 0.016s
I20260709 13:05:51.367828 23819 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:51.398716 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.398984 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 122077us
I20260709 13:05:51.399763 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:51.400170 24026 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:51.401238 24026 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:51.404405 24026 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:51.405140 23994 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:51.406059 23994 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:51.409441 21853 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:51.411041 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 605) took 1108 ms. Trace:
I20260709 13:05:51.411509 22009 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:51.411662 22585 rpcz_store.cc:276] 0709 13:05:50.302516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.312244 (+  9728us) service_pool.cc:224] Handling call
0709 13:05:51.410336 (+1098092us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:51.410694 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172143872}
W20260709 13:05:51.412825 21935 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.015s	sys 0.004s
W20260709 13:05:51.413379 21935 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.015s	sys 0.004s
I20260709 13:05:51.425088 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:51.426174 22001 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:51.428500 23903 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:51.430302 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.430524 23903 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:51.431051 23903 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:51.433198 23835 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:51.434134 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:51.434634 23835 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:51.435067 22004 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:51.435211 23835 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:51.332769 21927 rpcz_store.cc:276] 0709 13:05:49.720707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.723452 (+  2745us) service_pool.cc:224] Handling call
0709 13:05:51.156451 (+1432999us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:51.300243 (+143792us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19010944}
I20260709 13:05:51.364274 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:51.437592 23835 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:51.394789 22005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.003s	sys 0.009s
W20260709 13:05:51.439221 23873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.831s	user 0.044s	sys 0.091s
I20260709 13:05:51.339218 23855 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 13:05:51.439043 21851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.308s	user 0.000s	sys 0.032s
I20260709 13:05:51.438709 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:51.474349 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 74407.3us
W20260709 13:05:51.476907 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 156086us
I20260709 13:05:51.432403 23903 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:51.513345 22007 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:05:51.504662 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.09835e+06us
I20260709 13:05:51.529258 22003 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:05:51.551568 21859 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
W20260709 13:05:51.550951 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59336.3us
I20260709 13:05:51.568584 22591 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:51.577601 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:51.570776 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:51.587491 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 56864.8us
I20260709 13:05:51.555790 23899 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:51.600612 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:51.568758 23835 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:51.532881 24026 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 13:05:51.612931 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.613423 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 405) took 1987 ms. Trace:
I20260709 13:05:51.613729 23771 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:51.614552 22003 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:51.614822 23912 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:51.615725 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65045.9us
I20260709 13:05:51.616290 22003 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:51.616722 22002 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.617295 23868 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.618752 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1632228009998471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:05:51.619246 23951 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:51.663960 23951 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:51.625358 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 550) took 1492 ms. Trace:
I20260709 13:05:51.674592 22593 rpcz_store.cc:276] 0709 13:05:50.133046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.133181 (+   135us) service_pool.cc:224] Handling call
0709 13:05:50.133463 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.563393 (+1429930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.567988 (+  4595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.577850 (+  9862us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:51.589914 (+ 12064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.600266 (+ 10352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.625270 (+ 25004us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15527,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":14883,"thread_start_us":1257138,"threads_started":1}
I20260709 13:05:51.666275 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1770) took 1552 ms. Trace:
I20260709 13:05:51.625772 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:51.677415 23868 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:51.677986 22591 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:51.626966 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:51.679593 22587 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:51.679893 23868 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:51.680526 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1128228009998793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 13:05:51.682041 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.649s (SENT)
I20260709 13:05:51.627281 22004 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:51.628988 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:51.617304 23942 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:51.641776 22009 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:51.642275 24015 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:51.645916 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:51.598706 23994 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:51.645267 21852 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:51.647716 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:51.649016 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:51.650470 22002 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:05:51.650645 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:51.651473 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:51.653301 21855 raft_consensus.cc:2955] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:51.655977 22000 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:51.577646 23911 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:51.656556 23990 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.489s	user 0.009s	sys 0.013s
I20260709 13:05:51.613839 22005 rpcz_store.cc:276] 0709 13:05:49.626278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.627399 (+  1121us) service_pool.cc:224] Handling call
0709 13:05:49.627623 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.627833 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:50.397751 (+769918us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.536277 (+1138526us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.563698 (+ 27421us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:51.604292 (+ 40594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.613211 (+  8919us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.613352 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3095,"prepare.queue_time_us":1949643,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":4582,"raft.queue_time_us":13146,"raft.run_cpu_time_us":716,"raft.run_wall_time_us":109127,"thread_start_us":1808781,"threads_started":2}
W20260709 13:05:51.613579 24019 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:05:51.663174 22588 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:51.625097 23713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.298s	user 0.095s	sys 0.077s
I20260709 13:05:51.645259 23903 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:51.665766 24006 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:05:51.671823 23831 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.400s	user 0.014s	sys 0.005s
W20260709 13:05:51.730741 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45813.2us
W20260709 13:05:51.732116 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.900774 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 218794us
I20260709 13:05:51.924768 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:51.925227 22008 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:51.926265 22003 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:05:51.926966 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 284839us
I20260709 13:05:51.720986 22001 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:51.901572 22002 pending_rounds.cc:85] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 2
I20260709 13:05:51.908241 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1845) took 2094 ms. Trace:
I20260709 13:05:51.926510 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 332) took 1497 ms. Trace:
W20260709 13:05:51.854518 24036 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:05:51.934759 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 271006us
I20260709 13:05:51.935763 22006 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:51.904301 23713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.653s	user 0.097s	sys 0.086s
I20260709 13:05:51.928372 22002 pending_rounds.cc:107] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:05:51.938413 22002 raft_consensus.cc:2889] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:51.736145 23922 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.618s	user 0.010s	sys 0.010s
W20260709 13:05:51.860762 23937 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.705s	user 0.015s	sys 0.005s
I20260709 13:05:51.926311 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:51.683169 23972 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 2.437s	user 0.005s	sys 0.001s
I20260709 13:05:51.902359 24006 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:51.929960 21931 rpcz_store.cc:276] 0709 13:05:49.807569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.886140 (+ 78571us) service_pool.cc:224] Handling call
0709 13:05:49.886491 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.886919 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.886949 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:49.891186 (+  4237us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.160394 (+1269208us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:51.160554 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.160751 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.901372 (+740621us) raft_consensus.cc:1685] finished
0709 13:05:51.901394 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.901673 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2692,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":9369,"thread_start_us":1210258,"threads_started":1,"wal-append.queue_time_us":1929221}
I20260709 13:05:51.955787 21860 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:51.973910 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:51.941742 24033 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:05:51.980870 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 547) took 2130 ms. Trace:
I20260709 13:05:51.982677 23716 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:51.926915 23942 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:51.926612 22585 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:52.014715 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.017597 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:52.017913 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.019362 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.020146 23932 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:51.986541 23776 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:52.020874 23776 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:52.021242 24029 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:51.987074 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.022342 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.022755 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 64094.3us
I20260709 13:05:52.006336 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:51.932147 21935 rpcz_store.cc:276] 0709 13:05:50.429439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.429567 (+   128us) service_pool.cc:224] Handling call
0709 13:05:50.429894 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.430146 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:51.413922 (+983776us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.904209 (+490287us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.925708 (+ 21499us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:51.926129 (+   421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.926237 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.926451 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11027,"prepare.queue_time_us":1302612,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":319151,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":168,"thread_start_us":1148527,"threads_started":2,"wal-append.queue_time_us":447827}
I20260709 13:05:52.023780 23874 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:52.024153 24028 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.024454 23874 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:52.024026 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 87980.4us
I20260709 13:05:52.011983 21928 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:52.014318 23716 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:51.921180 23831 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.025894 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:52.002435 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.031344 21859 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:52.010394 23937 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.006424 22586 rpcz_store.cc:276] 0709 13:05:49.850648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.853152 (+  2504us) service_pool.cc:224] Handling call
0709 13:05:49.878763 (+ 25611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.902652 (+ 23889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.907598 (+  4946us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:49.983955 (+ 76357us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.166063 (+1182108us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:51.234249 (+ 68186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.265693 (+ 31444us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.980509 (+714816us) raft_consensus.cc:1685] finished
0709 13:05:51.980544 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.980760 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1230,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":13418,"thread_start_us":1122453,"threads_started":1,"wal-append.queue_time_us":1981095}
I20260709 13:05:52.054064 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:52.054404 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.028239 23874 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.055410 22585 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:52.063227 23886 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:52.063870 23764 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:52.065553 21852 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:52.065675 23733 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:52.066174 21858 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:51.942188 24015 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.012553 24006 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.024852 24028 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.044950 23951 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.076660 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.072386 23937 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:51.676651 21851 rpcz_store.cc:276] 0709 13:05:50.114014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.117670 (+  3656us) service_pool.cc:224] Handling call
0709 13:05:50.118025 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.124700 (+  6675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.124735 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.130776 (+  6041us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.600408 (+1469632us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:51.625736 (+ 25328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.625878 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.658122 (+ 32244us) raft_consensus.cc:1685] finished
0709 13:05:51.664839 (+  6717us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.666123 (+  1284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1120,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":7099,"thread_start_us":1344572,"threads_started":1,"wal-append.queue_time_us":1478782}
I20260709 13:05:52.206846 21853 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:52.202104 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1755) took 3302 ms. Trace:
I20260709 13:05:52.082813 22590 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:52.183889 22006 raft_consensus.cc:2410] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:52.216212 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.082609 24027 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 13:05:52.095996 23933 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:52.089283 24038 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.156152 23886 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.104432 24032 raft_consensus.cc:2764] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:05:52.208935 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 141168us
I20260709 13:05:52.201164 23954 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.108893 23831 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.216359 21860 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:05:52.198319 21926 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.777s	user 0.012s	sys 0.000s
I20260709 13:05:52.221992 21926 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:52.222630 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 337) took 1111 ms. Trace:
I20260709 13:05:52.222873 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.218364 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1777) took 1829 ms. Trace:
I20260709 13:05:52.223621 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.224364 21856 rpcz_store.cc:276] 0709 13:05:50.389271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.389410 (+   139us) service_pool.cc:224] Handling call
0709 13:05:50.389872 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.394284 (+  4412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.394330 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.394945 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.452655 (+1057710us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:51.486693 (+ 34038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.508545 (+ 21852us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.217976 (+709431us) raft_consensus.cc:1685] finished
0709 13:05:52.217998 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.218199 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3897,"raft.queue_time_us":10141,"spinlock_wait_cycles":8961280,"thread_start_us":235402,"threads_started":1,"wal-append.queue_time_us":1583864}
I20260709 13:05:52.226953 21856 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:52.227041 21851 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:52.228345 21856 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:52.089285 23932 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.217206 21854 rpcz_store.cc:276] 0709 13:05:48.899460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.899641 (+   181us) service_pool.cc:224] Handling call
0709 13:05:48.899965 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.904251 (+  4286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.904306 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.914612 (+ 10306us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.977768 (+ 63156us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:48.978099 (+   331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.978509 (+   410us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.682055 (+2703546us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:05:52.201633 (+519578us) raft_consensus.cc:1685] finished
0709 13:05:52.201663 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.201914 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3926,"raft.queue_time_us":22487,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":13856,"spinlock_wait_cycles":45972352,"thread_start_us":61158,"threads_started":1,"wal-append.queue_time_us":331136}
I20260709 13:05:52.218901 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.216382 24006 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.218880 23764 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.232214 23764 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.219805 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.219730 24038 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.217679 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.233946 21858 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:52.095065 23911 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.235472 24038 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.222988 21926 rpcz_store.cc:276] 0709 13:05:51.111182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.288481 (+177299us) service_pool.cc:224] Handling call
0709 13:05:52.222538 (+934057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.235896 23932 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:52.083690 23877 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.681s	user 0.011s	sys 0.007s
I20260709 13:05:52.241222 24026 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.257826 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 236162us
W20260709 13:05:52.259769 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.259796 23714 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.260483 23714 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:52.257933 22556 leader_election.cc:336] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.618s (SENT)
I20260709 13:05:52.262272 21860 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:05:52.264714 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 217217us
I20260709 13:05:52.262633 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:52.251123 24017 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:52.274422 24038 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.258347 24026 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.279776 23873 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:52.281888 22009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41320
I20260709 13:05:52.290211 22009 raft_consensus.cc:606] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 13:05:52.290445 22009 raft_consensus.cc:614] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:52.263831 23933 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:05:52.284001 21933 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:52.286692 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.288242 23881 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.284804 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1825) took 2564 ms. Trace:
I20260709 13:05:52.294368 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 604) took 2185 ms. Trace:
I20260709 13:05:52.272138 23716 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.293701 21932 rpcz_store.cc:276] 0709 13:05:49.720182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.720298 (+   116us) service_pool.cc:224] Handling call
0709 13:05:49.738488 (+ 18190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.993224 (+254736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.284306 (+2291082us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540042a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:52.284626 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538745088}
I20260709 13:05:52.301362 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.293308 21853 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:52.304639 21852 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 3. Ops: []
I20260709 13:05:52.301440 23835 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:52.306090 21856 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:52.307004 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:52.309731 24037 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:05:52.301195 24017 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.314086 23874 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.317570 21931 raft_consensus.cc:2410] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:52.319990 23942 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:52.332463 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 86692.4us
I20260709 13:05:52.334002 21927 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:52.334869 21933 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:52.325892 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:52.336472 21926 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:52.339922 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:52.341472 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.341701 21935 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:52.294824 22584 rpcz_store.cc:276] 0709 13:05:50.107550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.110359 (+  2809us) service_pool.cc:224] Handling call
0709 13:05:50.117914 (+  7555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.126091 (+  8177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.127677 (+  1586us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.934437 (+1806760us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b2c001465b8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:05:51.940107 (+  5670us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.244251 (+304144us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:52.244316 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.249963 (+  5647us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.278497 (+ 28534us) raft_consensus.cc:1685] finished
0709 13:05:52.289049 (+ 10552us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.292735 (+  3686us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10614528,"thread_start_us":208950,"threads_started":1,"wal-append.queue_time_us":223276}
I20260709 13:05:52.348078 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.348537 23578 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:52.349239 21928 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:52.349702 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1790) took 1022 ms. Trace:
I20260709 13:05:52.350394 21929 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:52.326709 23873 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:52.351401 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.351866 23835 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.353165 23886 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.354177 23835 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.354594 23936 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.338876 23907 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:52.355203 23907 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:05:52.351868 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.357201 23873 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.356379 23907 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.350098 22006 rpcz_store.cc:276] 0709 13:05:51.332094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.362329 (+ 30235us) service_pool.cc:224] Handling call
0709 13:05:52.349457 (+987128us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400437bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:52.349637 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235200512}
I20260709 13:05:52.359084 21858 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:52.347169 23874 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.347091 24028 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.357959 23813 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:52.358351 23932 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:52.370769 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.368861 23813 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.382798 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:52.375087 23937 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.377526 23813 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.383534 23942 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:52.390376 23578 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:52.391489 23942 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.391988 23907 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:52.393898 23942 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.396881 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 411) took 2602 ms. Trace:
I20260709 13:05:52.397322 22008 rpcz_store.cc:276] 0709 13:05:49.794030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.794205 (+   175us) service_pool.cc:224] Handling call
0709 13:05:49.794579 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.925829 (+2131250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.396565 (+470736us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b5400425ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:52.396769 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4895279360}
I20260709 13:05:52.410116 23720 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:52.410903 23720 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.411507 23813 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.413028 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.413373 23718 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:52.413937 23718 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.415628 23989 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.416008 23718 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.416488 23837 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:52.386242 23835 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.417117 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:52.417402 23835 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.419420 23776 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.423811 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.416911 23837 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:05:52.434041 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.436324 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.425524 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:52.512853 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.514542 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.515008 23837 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.446205 23907 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.540804 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 151328us
I20260709 13:05:52.544720 23907 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.516533 23837 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.415602 23813 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:52.550913 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162110us
I20260709 13:05:52.423588 23936 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:52.553851 22009 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:52.553687 23776 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.563026 23786 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:52.563338 21853 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:52.564884 22002 raft_consensus.cc:2410] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:52.565460 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 176078us
I20260709 13:05:52.541466 23720 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.567009 22588 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:52.575856 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.567929 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:52.568594 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.577387 22007 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:52.548966 23907 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.569381 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.571848 24015 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.566692 23936 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.582261 24041 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.583033 23936 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.582937 24041 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.575049 24017 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.577764 22008 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:52.587088 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:52.587996 22584 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.589533 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.590261 21852 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:52.554970 23835 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.595556 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.606683 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 606) took 2237 ms. Trace:
I20260709 13:05:52.618546 22589 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:52.619413 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 483) took 1001 ms. Trace:
I20260709 13:05:52.619803 22589 rpcz_store.cc:276] 0709 13:05:51.618124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.657121 (+ 38997us) service_pool.cc:224] Handling call
0709 13:05:52.618006 (+960885us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:52.618239 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.619013 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.619324 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17661696}
I20260709 13:05:52.607592 22592 rpcz_store.cc:276] 0709 13:05:50.360253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.360432 (+   179us) service_pool.cc:224] Handling call
0709 13:05:50.360801 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.161262 (+800461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.592702 (+1431440us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b5400530cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:52.598026 (+  5324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793251712}
I20260709 13:05:52.630019 23778 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:52.630551 23778 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:52.629061 23720 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:52.631803 23778 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.632809 23720 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.633270 23771 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.634281 23932 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.634076 23771 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.633389 23720 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.650034 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.649535 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.615998 21926 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:52.667150 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.653103 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.673697 22008 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:52.674431 22007 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:05:52.675734 24049 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:05:52.651669 23778 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:52.682121 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 46269.3us
W20260709 13:05:52.683606 22556 leader_election.cc:336] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.884s (SENT)
W20260709 13:05:52.683789 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 200795us
I20260709 13:05:52.684077 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:52.646265 23771 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.685911 23813 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:52.689237 22005 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:52.689477 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.690791 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.691200 22000 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:05:52.691668 22000 pending_rounds.cc:85] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:05:52.691718 22008 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:52.691972 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:52.692735 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.693163 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.693439 24017 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.693864 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.689736 22005 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:52.694818 22004 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:52.695153 22005 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.695665 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:52.696311 22005 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.696611 22004 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:52.693473 22001 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:52.697276 22001 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:52.698246 22001 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.699229 22001 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.692005 22000 pending_rounds.cc:107] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:05:52.701608 22000 raft_consensus.cc:2889] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:52.702454 23720 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.691087 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:52.704351 24008 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.342s	user 0.005s	sys 0.001s
I20260709 13:05:52.704210 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.705077 22003 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:52.708323 23988 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.702481 23927 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:52.709491 23927 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:52.710528 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:52.710363 23927 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.711807 23927 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.711790 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.713642 24026 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:52.716667 24017 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.717203 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:52.719115 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.719826 22004 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:52.720690 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:52.721262 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.722075 22007 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:52.721766 24017 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:52.722316 24041 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.722648 24017 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:52.723652 24017 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.724004 21754 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 4, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 4 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:52.725843 21754 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:05:52.726962 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.727401 21754 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:52.729728 22584 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:52.720649 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.731421 22004 raft_consensus.cc:2410] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:52.732370 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47588.1us
I20260709 13:05:52.704897 23720 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:52.732870 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.732906 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.693971 22009 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:05:52.737323 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.738379 23601 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:52.738817 23601 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.739329 23601 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.739497 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.738579 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.742040 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:52.735901 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 139407us
I20260709 13:05:52.743923 22006 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:52.742740 22007 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:52.634827 23932 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.740731 21930 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:52.740285 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.747974 22592 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:52.735302 21754 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 6, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 6 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:52.703030 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:52.756524 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.760878 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.761746 21857 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:52.761888 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.762602 21926 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:52.754174 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.763191 22589 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:52.763442 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:52.760962 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.764004 21935 raft_consensus.cc:2376] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5. Re-sending same reply.
I20260709 13:05:52.764433 23796 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.736263 23813 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:52.769106 21928 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:05:52.770321 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.764781 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.769316 21753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 281) took 6628 ms. Trace:
I20260709 13:05:52.765128 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.772895 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:52.777599 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.778340 22001 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:52.780258 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.782030 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.782889 22007 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:52.785044 22589 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:52.754844 23778 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.764873 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.766145 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.780015 21754 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:52.765352 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.794885 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:52.796033 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:52.766601 23941 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.800294 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.801999 23879 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.802449 23879 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.802798 23879 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.802887 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.804008 23899 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:52.804464 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.805444 24015 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.805500 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:52.806052 24015 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.809278 24015 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.813488 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.815018 21931 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:52.817054 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:52.799254 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:52.823431 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.820744 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.826086 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.806902 21929 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:52.826195 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.826911 22006 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:52.819449 23988 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:52.808532 23778 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.827152 23986 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:52.820706 22585 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:52.810263 21927 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:52.827430 21754 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:52.832508 22003 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:52.825609 23881 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:52.819525 23941 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:52.833820 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:52.836640 22001 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.836242 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:52.846853 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.847922 22003 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:52.849040 23974 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:52.830606 23988 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:52.849712 23974 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:52.849771 23988 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:52.850358 23778 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:52.850896 23974 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.830709 23986 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:52.847265 23835 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:52.853328 23986 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.853608 23835 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.854055 23835 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.853824 21754 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 3, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 3 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:52.856711 24041 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:52.856993 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.857817 22006 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:52.858903 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:52.859956 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.857213 24041 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:52.861332 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.856726 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:52.819258 23720 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:05:52.870118 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.866812 24041 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:52.839459 23941 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:52.866729 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.879597 23716 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:52.881117 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.879987 23964 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:52.881487 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:52.888136 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.871524 23989 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:52.880244 23716 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:52.889267 23716 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.875485 23941 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:52.865986 23813 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.881330 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.880350 23796 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:52.884553 23837 raft_consensus.cc:2764] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:52.897768 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.889568 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.887221 23964 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:52.905632 22001 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:52.774019 21753 rpcz_store.cc:276] 0709 13:05:46.105951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.106193 (+   242us) service_pool.cc:224] Handling call
0709 13:05:52.716873 (+6610680us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:47.608664 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:47.628042 (+ 19378us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:47.628101 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:05:47.628129 (+    28us) tablet.cc:662] Decoding operations
0709 13:05:47.628634 (+   505us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:47.628738 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:47.628781 (+    43us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:05:47.680073 (+ 51292us) tablet.cc:701] Row locks acquired
0709 13:05:47.682168 (+  2095us) write_op.cc:260] PREPARE: finished
0709 13:05:47.753550 (+ 71382us) write_op.cc:270] Start()
0709 13:05:47.766294 (+ 12744us) write_op.cc:276] Timestamp: P: 1783602347753483 usec, L: 0
0709 13:05:47.776982 (+ 10688us) op_driver.cc:348] REPLICATION: starting
0709 13:05:47.898614 (+121632us) log.cc:844] Serialized 919 byte log entry
0709 13:05:49.397659 (+1499045us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800335f80. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1
0709 13:05:49.420133 (+ 22474us) op_driver.cc:464] REPLICATION: finished
0709 13:05:50.043794 (+623661us) write_op.cc:301] APPLY: starting
0709 13:05:50.043904 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:50.048447 (+  4543us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:50.048488 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:50.171856 (+123368us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:50.172049 (+   193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:05:50.172094 (+    45us) write_op.cc:312] APPLY: finished
0709 13:05:50.172305 (+   211us) log.cc:844] Serialized 29 byte log entry
0709 13:05:50.910001 (+737696us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:50.910373 (+   372us) write_op.cc:454] Released schema lock
0709 13:05:50.910573 (+   200us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:50.912190 (+  1617us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:50.990412 (+ 78222us) write_op.cc:454] Released schema lock
0709 13:05:52.341745 (+1351333us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cc32a 00007f3b70751f14 00007f3b70751ec3 00007f3b70751e38 00007f3b70751df3 00007f3b707510dc 00007f3b7077ae16 00007f3b70762cbb 00007f3b7074d39d 00007f3b70750d75
Metrics: {"mutex_wait_us":667975,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":620210,"apply.run_cpu_time_us":14500,"apply.run_wall_time_us":871195,"num_ops":2,"prepare.queue_time_us":673568,"prepare.run_cpu_time_us":90997,"prepare.run_wall_time_us":1868328,"raft.queue_time_us":343852,"raft.run_cpu_time_us":33397,"raft.run_wall_time_us":3120627,"replication_time_us":1635979,"spinlock_wait_cycles":427601024,"thread_start_us":2529899,"threads_started":5,"wal-append.queue_time_us":2212258,"wal-append.run_cpu_time_us":43309,"wal-append.run_wall_time_us":1185645}]]}
I20260709 13:05:52.907294 23833 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.910393 23833 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.910768 23833 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.910966 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:52.911612 21858 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:52.909718 23932 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.913393 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.913815 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.914146 21931 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:52.914258 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.914539 21929 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:52.918200 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.894671 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.899055 23989 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:52.896164 23796 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:52.930193 23796 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:52.931104 21806 auto_rebalancer.cc:893] Removing replace marker failed: Replica 7d07aa0dcdd1484ab87925c9a4604b81 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:05:52.931545 21806 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:05:52.931823 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:05:52.921533 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:52.924849 24017 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.926302 21931 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:52.926894 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.907265 23942 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.927490 23989 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.927906 23883 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:52.928040 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:52.939688 21932 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.929714 23714 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.893739 23778 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:05:52.948021 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.960462 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.972693 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.942396 23883 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:52.978130 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:52.941807 23989 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.987922 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:52.989573 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:52.990289 21928 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.990489 24015 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:52.948137 23942 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:52.992606 22007 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:52.993446 24015 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:52.994022 23942 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:52.944468 24012 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.205s	user 0.002s	sys 0.008s
I20260709 13:05:52.961619 24017 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.041716 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:53.029485 23899 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:53.031908 23942 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:05:53.048836 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:53.048913 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 127434us
I20260709 13:05:53.057139 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.057916 22003 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:53.059401 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1596) took 1043 ms. Trace:
I20260709 13:05:53.063778 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.049875 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:53.047502 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 124902us
I20260709 13:05:53.059947 21860 rpcz_store.cc:276] 0709 13:05:52.016430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.222456 (+206026us) service_pool.cc:224] Handling call
0709 13:05:53.059155 (+836699us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:53.059322 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87912192}
I20260709 13:05:53.040611 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.043320 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.050618 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.052268 23786 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:53.106201 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.052857 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.069766 24006 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:53.107630 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.123991 21855 raft_consensus.cc:2368] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
W20260709 13:05:53.103906 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.042577 23835 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:53.128193 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.047608 23936 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
W20260709 13:05:53.132623 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 175061us
I20260709 13:05:53.041288 23829 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.155s
I20260709 13:05:53.074139 23899 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:53.137281 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.141393 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.061407 23951 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
W20260709 13:05:53.145978 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.081874 23907 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:53.082324 23881 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:53.159237 22002 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:05:53.162111 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.121215 23786 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.162923 21855 raft_consensus.cc:2376] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5. Re-sending same reply.
I20260709 13:05:53.168327 22001 raft_consensus.cc:2368] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:53.169427 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 607) took 2799 ms. Trace:
I20260709 13:05:53.157940 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.127633 24006 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.174526 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.171053 22590 rpcz_store.cc:276] 0709 13:05:50.369371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.369556 (+   185us) service_pool.cc:224] Handling call
0709 13:05:50.369904 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.217715 (+1847811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.143869 (+926154us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54006702d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:53.169093 (+ 25224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3922075520}
I20260709 13:05:53.180814 21928 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:53.140864 21929 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:53.190086 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.189703 23881 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.191442 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.195747 23716 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.173761 23786 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.200004 23937 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:53.082240 23883 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.201296 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.202097 21855 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:53.203029 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.183826 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.203992 21851 raft_consensus.cc:2376] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4. Re-sending same reply.
I20260709 13:05:53.204079 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.204706 21855 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:53.204948 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.214035 21860 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:53.205542 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.215492 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.221748 22002 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:53.216568 21932 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:53.227780 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.232040 21856 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:53.179840 24006 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.241142 22003 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:53.242496 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:53.086030 23964 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.201340 23883 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:53.252420 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 188742us
I20260709 13:05:53.210916 23835 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:53.206086 23833 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.263463 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.259398 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.281854 21856 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:53.257514 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.283012 23964 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.283226 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:53.283833 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.286968 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.288015 23988 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.213375 23796 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.222492 23879 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:53.280723 22005 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
W20260709 13:05:53.388927 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.303684 23899 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:53.418007 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:53.418474 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 149140us
I20260709 13:05:53.256615 24059 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:53.421393 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:53.446089 21928 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:53.446177 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.447034 22008 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:53.422075 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 138941us
W20260709 13:05:53.423507 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 152628us
I20260709 13:05:53.454514 22005 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:05:53.437280 21851 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:53.455008 22005 pending_rounds.cc:85] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:05:53.458140 22005 pending_rounds.cc:107] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:05:53.458405 22005 raft_consensus.cc:2889] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:53.421970 23835 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.439147 23941 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.009s
I20260709 13:05:53.435498 23951 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:53.467312 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.468231 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.469151 21856 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:53.437774 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.475884 21935 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:53.474272 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 346134us
W20260709 13:05:53.462601 23601 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.719s	user 0.018s	sys 0.007s
I20260709 13:05:53.480986 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.482532 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:53.466858 24063 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.168s	user 0.000s	sys 0.007s
W20260709 13:05:53.489372 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.473835 24059 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:53.461478 24017 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:53.472127 24040 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.228s	user 0.007s	sys 0.000s
I20260709 13:05:53.496929 22004 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.509541 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.510037 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.511014 21855 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:53.511233 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.502738 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:53.514930 22590 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.444250 24064 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 13:05:53.515700 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 330890us
I20260709 13:05:53.510233 21931 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:53.473134 23951 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:53.524228 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.528256 21928 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:53.529603 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:53.530313 21928 raft_consensus.cc:2376] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5. Re-sending same reply.
W20260709 13:05:53.522987 24058 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.367s	user 0.001s	sys 0.007s
I20260709 13:05:53.486477 23716 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.535290 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.474857 23941 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.541450 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.471777 23835 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.555025 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.555166 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.555440 22556 leader_election.cc:400] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:53.557433 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.560930 21857 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:53.563757 22009 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.535858 23951 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:53.564402 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.564611 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:53.534993 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.569823 22004 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:53.496042 23601 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.580721 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.581523 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.582672 21928 raft_consensus.cc:2410] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:53.584314 21857 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:53.595265 21859 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:53.567365 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.578732 24071 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:53.598269 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.604636 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.606992 21857 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:53.589491 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.588272 21853 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:53.607798 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.610404 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.614961 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.615772 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.616802 22004 raft_consensus.cc:2368] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:53.600960 23936 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.603644 23833 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:53.601248 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:53.620991 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 60463.1us
I20260709 13:05:53.618162 21928 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:53.628402 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.630306 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.630519 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.613085 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:53.642064 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.634405 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.634243 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:53.645104 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.649909 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.650272 21932 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:53.624691 23879 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:53.653856 22592 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:05:53.652566 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:53.645833 24068 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:05:53.632143 24062 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:53.667565 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.607858 24071 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.669695 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.658910 23833 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:53.674259 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.674639 23835 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:53.675101 23988 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:53.676393 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.622720 23786 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:53.645526 24069 raft_consensus.cc:670] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:53.624719 23936 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.682585 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.627780 21927 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
W20260709 13:05:53.699352 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:53.699719 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 74332.8us
I20260709 13:05:53.700978 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.701464 21856 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:05:53.610549 24059 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.709765 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.712544 22001 raft_consensus.cc:2368] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:53.713475 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.714200 23856 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:05:53.714532 23986 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.861s	user 0.009s	sys 0.003s
I20260709 13:05:53.714672 23856 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:53.715095 23986 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.715731 23856 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.717196 23833 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:53.718245 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.718952 22589 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6.
I20260709 13:05:53.719817 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:53.713812 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.713289 23835 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:53.715371 22556 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:53.714849 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.712592 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.730381 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.731429 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.678507 24006 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.697216 24062 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:53.739003 22004 raft_consensus.cc:2368] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:53.739322 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.716843 23856 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.707934 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.747411 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.748682 22589 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:53.751592 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:53.711369 23941 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:53.760084 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:53.747303 24041 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.857s	user 0.006s	sys 0.007s
I20260709 13:05:53.744628 23835 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:53.766219 21856 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:05:53.773304 24060 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:05:53.773717 22585 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:05:53.780301 23936 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.789247 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.790239 22584 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:53.766865 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.794337 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:53.795110 21935 raft_consensus.cc:2368] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:53.770952 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.747603 23813 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.748732 23601 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:53.799440 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.800163 22589 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:05:53.795845 24072 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:53.802604 24072 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:53.802997 24072 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.806059 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.759521 21853 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:53.795329 24041 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.807144 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.838734 22006 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:53.839157 22008 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:53.839775 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.807561 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.841382 22004 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.839845 22590 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:53.808035 22591 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.808699 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:53.809522 23989 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.004s
I20260709 13:05:53.798245 23986 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.811120 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.800690 24083 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:05:53.806252 23786 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.843876 22006 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:53.769918 23881 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:53.821566 23813 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.862058 23881 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:53.863214 22591 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.863142 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:53.864344 22585 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 3: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:53.846726 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.849179 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1775) took 1146 ms. Trace:
I20260709 13:05:53.865543 22587 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.865743 22008 rpcz_store.cc:276] 0709 13:05:52.692738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.693625 (+   887us) service_pool.cc:224] Handling call
0709 13:05:53.446039 (+752414us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaeee 0000000000375939 00007f3b7571e1e0 00007f3b757e70c4 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:53.839587 (+393548us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8691456}
I20260709 13:05:53.848603 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:53.852329 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 135133us
W20260709 13:05:53.812964 23883 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.548s	user 0.003s	sys 0.006s
I20260709 13:05:53.865233 21856 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:53.853098 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:53.855363 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.856146 22592 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:53.856513 22589 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.856163 23716 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.870600 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.871273 21856 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:05:53.871706 24073 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:05:53.872117 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:53.872689 21856 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:53.813081 24072 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.875433 24072 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.860957 22593 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:53.863025 23881 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.845616 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.869585 22588 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:53.866945 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.866967 22007 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:53.892532 22001 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.814491 24071 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:53.868661 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.869973 23813 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:53.870400 23889 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:05:53.878091 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:53.902637 22006 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.876819 23601 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:53.881249 24062 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.883184 24041 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:53.883818 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.884352 24006 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.888651 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:53.867367 24070 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:53.867189 23941 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.907431 23941 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.903740 23601 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:53.909113 23889 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.910022 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.889372 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:53.911876 24070 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:05:53.913580 24055 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.064s	user 0.000s	sys 0.007s
I20260709 13:05:53.914103 24070 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:53.849323 23989 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:53.916054 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:53.916244 23989 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:53.890283 21851 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:05:53.890739 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.890905 23716 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:53.895196 22587 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:53.867492 24087 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.904733 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 192200us
I20260709 13:05:53.904739 22002 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:53.905372 23837 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:53.910137 23601 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.919057 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.920567 22004 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:53.916921 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:53.922291 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.924021 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:53.916797 23989 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:53.897301 22000 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:53.925032 23837 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:53.935458 23813 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:53.910607 23941 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:53.944442 22006 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:53.945232 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44911.1us
I20260709 13:05:53.937860 24083 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:53.947196 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.938606 23937 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:53.955459 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:53.945391 23813 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:53.946767 24083 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:53.949640 23883 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:53.961031 23813 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.970292 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:53.957361 23937 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:53.972545 23601 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.961583 23837 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:53.961215 24029 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:53.936353 24087 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.034343 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.041884 21856 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:05:54.042789 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.041381 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:54.046635 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.048169 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.048297 24083 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.049912 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.050781 21854 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 2: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:54.051426 21824 leader_election.cc:336] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 1.587s (SENT)
I20260709 13:05:54.051632 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.052320 21854 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:54.053496 24094 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:05:54.048883 21855 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.057890 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:54.059139 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 107985us
I20260709 13:05:53.960320 23951 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.034998 23883 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.062491 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.035156 23937 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.061980 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.066772 21860 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:05:54.081801 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.082304 21856 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.083046 21854 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.083796 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.084406 21854 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:54.085268 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1778) took 1196 ms. Trace:
I20260709 13:05:54.085989 24087 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:54.086627 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45305.3us
I20260709 13:05:54.087368 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:54.087496 23796 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.087687 23813 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.088351 23796 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:54.089504 24096 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:05:54.085678 21859 rpcz_store.cc:276] 0709 13:05:52.869655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.869792 (+   137us) service_pool.cc:224] Handling call
0709 13:05:54.066143 (+1196351us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b540028b754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:54.066338 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665518336}
I20260709 13:05:54.094151 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.095454 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.096457 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.096969 21856 raft_consensus.cc:2368] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:54.107884 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.109807 21854 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:54.109833 21857 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.088908 23796 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.111965 23796 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:54.112599 23813 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.090524 24029 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:05:54.112701 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.114250 21859 pending_rounds.cc:85] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 4
I20260709 13:05:54.114604 21859 pending_rounds.cc:107] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:05:54.114895 21859 raft_consensus.cc:2889] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:54.090780 24097 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:05:54.080570 23883 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:54.119024 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.121929 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60151.6us
I20260709 13:05:54.040894 23964 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:05:54.123095 21854 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:05:54.120510 21857 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:54.080041 21851 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:54.106426 24099 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.127144 21857 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:54.117419 24084 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:54.130576 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.135217 21933 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.124240 24064 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.119745 23941 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
W20260709 13:05:54.143891 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 101207us
I20260709 13:05:54.130414 23813 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.150068 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.132658 24084 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:54.152676 23829 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:54.159765 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.160369 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.160560 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.164803 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.165645 22002 raft_consensus.cc:2410] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:54.166944 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.168349 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:54.170171 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.170092 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.172400 21854 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:54.176174 22002 raft_consensus.cc:2368] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6.
W20260709 13:05:54.185262 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 62556.1us
I20260709 13:05:54.186952 22004 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:54.195542 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.194000 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:54.193562 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94815us
I20260709 13:05:54.196280 21928 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:54.197739 24086 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.198027 21824 leader_election.cc:400] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:54.196689 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:54.198729 23951 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.199362 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:54.199668 24100 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.158s	user 0.002s	sys 0.000s
I20260709 13:05:54.201069 21853 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:54.194638 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.202607 21853 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:54.202873 21927 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.206435 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.207039 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.207580 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.208856 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.209424 21932 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:54.209622 24084 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.210316 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.210465 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.210912 24084 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.207686 21934 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
I20260709 13:05:54.209563 21930 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.157896 23813 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.216167 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.217033 21926 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:54.217952 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.218489 21926 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.219161 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.219812 21934 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.219650 21853 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:54.175356 24081 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:54.221385 24081 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:54.221877 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:54.222947 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.148449 24064 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.185963 23856 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:54.251713 23856 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:05:54.253149 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55257.1us
I20260709 13:05:54.254094 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:54.251754 21932 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:05:54.254309 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.257104 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.257853 22003 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:54.263535 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.264303 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:05:54.265118 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.265671 22001 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.266390 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.267988 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.269119 22006 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:05:54.222545 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:54.252614 23856 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.272414 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.272876 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.290196 22006 raft_consensus.cc:2368] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:54.273707 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:54.292421 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.282313 21825 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:54.288276 21824 leader_election.cc:400] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:54.253149 24064 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.281312 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:54.294905 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.299156 22006 raft_consensus.cc:2368] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:54.295779 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.299994 22009 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.162051 24105 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:05:54.297796 22002 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:54.299304 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:54.269821 24107 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:05:54.215802 24103 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:05:54.312477 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:54.315999 22001 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:54.321724 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.322494 21932 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.326582 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.327318 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.328662 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:54.328907 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:54.329022 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:54.329768 22585 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:54.329921 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:54.330484 21853 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:54.330636 22593 raft_consensus.cc:2368] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:54.330863 24110 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:54.295104 24083 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.331853 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.332548 22007 raft_consensus.cc:2376] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4. Re-sending same reply.
I20260709 13:05:54.328662 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:54.333995 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.334952 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.335167 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.335707 22008 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:54.335800 21929 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.336524 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.300303 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.337190 24108 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:54.337874 24108 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.337982 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:54.338297 24108 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.339593 21927 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:54.340008 23813 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:54.331914 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43888us
I20260709 13:05:54.297106 24069 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.341585 22008 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:54.344923 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.317425 23936 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:54.341507 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:54.352394 22007 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:54.345165 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:54.362123 22556 leader_election.cc:400] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:54.362546 21931 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:54.351405 21934 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
W20260709 13:05:54.365046 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 232628us
W20260709 13:05:54.369390 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.362672 22556 leader_election.cc:403] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:05:54.370800 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 113917us
I20260709 13:05:54.340976 21927 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:05:54.298389 24086 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:05:54.381755 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:54.342294 23856 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.379504 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:54.324878 23796 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.418388 23796 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:54.421180 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:54.421887 23796 raft_consensus.cc:2764] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:05:54.422190 23716 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:54.425052 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60434.8us
I20260709 13:05:54.425616 23716 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.425940 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:54.426146 23716 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.428058 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:54.428730 21855 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:54.380967 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67613.5us
I20260709 13:05:54.431578 23796 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:54.435711 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:54.439126 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 67461.6us
I20260709 13:05:54.451315 23796 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:54.452114 23796 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.452569 23796 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.351658 24064 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:05:54.349733 24083 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:05:54.381259 23936 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:54.346930 24069 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.384569 24059 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:54.353231 23813 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:05:54.459561 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:54.467222 21974 leader_election.cc:400] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:54.473214 21974 leader_election.cc:403] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:05:54.478549 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:54.493438 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.509739 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47183.1us
I20260709 13:05:54.511433 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:54.459935 23829 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.528779 22005 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:54.533299 24083 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:54.534173 24083 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:54.538905 22008 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:54.552008 22009 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:54.552251 21933 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.523810 24110 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:54.521298 24106 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.613690 24106 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.617134 22001 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:05:54.617630 22001 pending_rounds.cc:85] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:05:54.617966 22001 pending_rounds.cc:107] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:05:54.618249 22001 raft_consensus.cc:2889] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:54.619509 23936 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.621572 24083 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.627175 24099 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:54.627741 24099 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:05:54.633035 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.633774 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.636830 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1794) took 1043 ms. Trace:
I20260709 13:05:54.637274 21859 rpcz_store.cc:276] 0709 13:05:53.597053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.113798 (+516745us) service_pool.cc:224] Handling call
0709 13:05:54.114095 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.115573 (+  1478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.115621 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.116094 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.352046 (+235952us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:54.352082 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.352147 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.636616 (+284469us) raft_consensus.cc:1685] finished
0709 13:05:54.636634 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.636753 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":235315,"threads_started":1,"wal-append.queue_time_us":414011}
I20260709 13:05:54.638585 24081 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:54.640226 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:54.640918 21935 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:54.646258 23837 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:54.648698 23937 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.004s
I20260709 13:05:54.559870 21930 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:54.526178 24087 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.574450 22005 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:54.697537 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:05:54.707504 22000 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.003s
I20260709 13:05:54.597667 24113 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:54.763952 24113 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:54.541669 23829 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:54.599627 24109 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:05:54.766424 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1792) took 1263 ms. Trace:
I20260709 13:05:54.766886 21856 rpcz_store.cc:276] 0709 13:05:53.503000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.098635 (+595635us) service_pool.cc:224] Handling call
0709 13:05:54.099304 (+   669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.113476 (+ 14172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.113504 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.113946 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.190920 (+ 76974us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:54.190961 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.191025 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.766147 (+575122us) raft_consensus.cc:1685] finished
0709 13:05:54.766173 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.766349 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7935,"raft.queue_time_us":16855,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":5241,"thread_start_us":76606,"threads_started":1,"wal-append.queue_time_us":196854}
I20260709 13:05:54.564459 23786 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:54.658545 22003 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:05:54.769178 22003 pending_rounds.cc:85] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 4
I20260709 13:05:54.769444 22003 pending_rounds.cc:107] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:05:54.769671 22003 raft_consensus.cc:2889] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:54.659163 23837 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.664696 23964 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:54.565840 24110 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:54.771723 24110 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.520407 23941 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:54.773330 23941 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.775640 23941 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:54.777809 23829 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.780892 24113 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.782472 24113 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:54.671830 23883 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.003s
I20260709 13:05:54.786724 23883 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.788280 21928 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:54.793105 23883 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:54.542173 24062 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.794435 24062 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.559299 24116 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.797395 24116 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.800473 24116 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.802474 24062 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:54.713760 24106 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:54.628795 24099 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:54.804487 24099 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.813408 22000 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:54.719781 24108 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:54.722883 23601 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.684s	user 0.004s	sys 0.006s
I20260709 13:05:54.822067 21852 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:54.823498 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1795) took 1202 ms. Trace:
I20260709 13:05:54.823879 21852 rpcz_store.cc:276] 0709 13:05:53.620989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.680336 (+ 59347us) service_pool.cc:224] Handling call
0709 13:05:53.909870 (+229534us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:54.823274 (+913404us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:54.823425 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108475904}
I20260709 13:05:54.825644 21853 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:54.730796 23937 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.829571 23937 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:54.801561 24116 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:05:54.734371 21926 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
W20260709 13:05:54.737476 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.833447 21926 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
W20260709 13:05:54.833798 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.578620 21855 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:54.834774 21858 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [3.5-4.6]
W20260709 13:05:54.737685 24084 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.006s
I20260709 13:05:54.762532 21929 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:54.763240 21932 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:54.673364 24069 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:54.772925 24110 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.716163 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1622) took 1215 ms. Trace:
I20260709 13:05:54.842165 21860 rpcz_store.cc:276] 0709 13:05:53.500943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.908736 (+407793us) service_pool.cc:224] Handling call
0709 13:05:54.066454 (+157718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.081226 (+ 14772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.715971 (+634745us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540028b750. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:54.716104 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361840000}
I20260709 13:05:54.751915 23716 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:54.832309 24116 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:54.902047 23989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.034s	sys 0.053s
W20260709 13:05:54.929850 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 488207us
I20260709 13:05:54.938114 21900 leader_election.cc:400] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6.
I20260709 13:05:54.941475 21900 leader_election.cc:403] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:54.953506 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.908689 23989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.035s	sys 0.053s
W20260709 13:05:54.953761 21851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.033s	sys 0.068s
W20260709 13:05:55.001981 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 56284.5us
I20260709 13:05:54.998158 24084 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:54.998729 23883 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6.
I20260709 13:05:54.997682 24106 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:55.022522 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.010s	sys 0.012s
W20260709 13:05:55.024529 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.010s	sys 0.012s
W20260709 13:05:55.008920 21851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.034s	sys 0.068s
W20260709 13:05:55.027148 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 548731us
I20260709 13:05:55.024818 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:55.043262 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.023775 24105 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.072s	sys 0.114s
W20260709 13:05:55.067505 24105 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.072s	sys 0.115s
W20260709 13:05:55.069540 21901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.010s	sys 0.017s
I20260709 13:05:55.019729 24108 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:55.071305 21901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.010s	sys 0.017s
W20260709 13:05:55.071756 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.074522 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.011s	sys 0.018s
W20260709 13:05:55.075663 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.011s	sys 0.018s
W20260709 13:05:55.077152 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 929755us
W20260709 13:05:55.078387 21857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.011s	sys 0.019s
W20260709 13:05:55.082566 21857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.011s	sys 0.019s
I20260709 13:05:55.091818 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1801) took 1374 ms. Trace:
I20260709 13:05:55.093521 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1804) took 1375 ms. Trace:
W20260709 13:05:55.063740 23889 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.004s	sys 0.022s
W20260709 13:05:55.094720 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.107565 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.107968 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.108369 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.108386 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.009s	sys 0.030s
W20260709 13:05:55.110407 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.009s	sys 0.030s
W20260709 13:05:55.096341 21826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.004s
W20260709 13:05:55.098285 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.022755 24084 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:55.108343 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.011s	sys 0.028s
W20260709 13:05:55.113876 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.011s	sys 0.028s
W20260709 13:05:55.114359 21826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.009s	sys 0.004s
W20260709 13:05:55.121544 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.000s	sys 0.026s
W20260709 13:05:55.121934 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.026s
W20260709 13:05:55.122232 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 653039us
I20260709 13:05:55.100471 24086 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:55.125222 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:55.126026 22007 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:55.126053 21855 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:55.059026 24106 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.122952 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.092327 21857 rpcz_store.cc:276] 0709 13:05:53.717201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.133774 (+416573us) service_pool.cc:224] Handling call
0709 13:05:54.134064 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.134267 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:54.459489 (+325222us) log.cc:844] Serialized 14 byte log entry
0709 13:05:55.090482 (+630993us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:55.090755 (+   273us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:55.090899 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.091190 (+   291us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.091645 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33241,"prepare.queue_time_us":251450,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":7263,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":4114,"thread_start_us":930936,"threads_started":2}
W20260709 13:05:55.127323 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.123301 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:55.118803 23889 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.004s	sys 0.022s
I20260709 13:05:55.136260 21860 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:55.125308 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:55.138535 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.015s	sys 0.040s
W20260709 13:05:55.139117 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.001s	sys 0.043s
I20260709 13:05:55.139899 21852 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
W20260709 13:05:55.139986 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.015s	sys 0.040s
W20260709 13:05:55.139938 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.001s	sys 0.043s
I20260709 13:05:55.140484 21852 pending_rounds.cc:85] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 5
I20260709 13:05:55.141026 21852 pending_rounds.cc:107] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:55.141283 21852 raft_consensus.cc:2889] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:55.144493 22559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.000s	sys 0.013s
W20260709 13:05:55.144974 22559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.000s	sys 0.013s
I20260709 13:05:55.134409 22003 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:55.155669 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:55.156666 21856 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:05:55.139226 21823 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.869s (SENT) [suppressed 81 similar messages]
W20260709 13:05:55.159329 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 725781us
I20260709 13:05:55.107692 24126 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:55.164877 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:55.156104 24083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.012s	sys 0.000s
I20260709 13:05:55.168447 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:55.171236 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:55.141273 23989 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.171547 22588 raft_consensus.cc:2368] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:55.168447 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:55.172402 22587 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:55.175504 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.178930 21855 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:55.154662 23883 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.177528 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:55.077278 24105 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
W20260709 13:05:55.171172 24083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.013s	sys 0.000s
I20260709 13:05:55.181285 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.182660 22000 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:55.183415 22586 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:55.183816 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:55.185496 24041 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:05:55.185886 23796 consensus_meta.cc:287] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.000s
I20260709 13:05:55.154435 24108 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:55.187592 22557 leader_election.cc:400] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:05:55.213866 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:55.180052 23941 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.182665 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.196756 24106 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:55.163695 24126 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.220203 22004 raft_consensus.cc:2368] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5.
I20260709 13:05:55.198297 24084 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:55.211231 23936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.019s	sys 0.037s
I20260709 13:05:55.199559 23716 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:55.222733 24108 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.226270 23716 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:55.226929 24086 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.227363 24086 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.227805 24108 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:55.228040 23716 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:55.178876 24059 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:05:55.230028 24059 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:55.230535 24059 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:55.231478 24059 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.200693 23964 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:55.178625 24080 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:05:55.234388 24062 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:55.234967 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.216727 23796 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.176517 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:55.236356 21853 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:55.236301 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.202930 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:55.209862 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:55.211061 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:55.238224 22593 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:55.208817 22000 pending_rounds.cc:85] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:05:55.208513 24069 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.111003 21854 rpcz_store.cc:276] 0709 13:05:53.718132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.173443 (+455311us) service_pool.cc:224] Handling call
0709 13:05:54.189851 (+ 16408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.190040 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:54.632048 (+442008us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b1800431980. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf
0709 13:05:54.632269 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 13:05:55.093099 (+460830us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:55.093167 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:55.093227 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.093302 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.093450 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11619,"prepare.queue_time_us":263729,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":3702,"raft.queue_time_us":28404,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":7582,"spinlock_wait_cycles":8693632,"thread_start_us":687586,"threads_started":2}
I20260709 13:05:55.224375 23829 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:55.207561 23883 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.224961 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.225556 24084 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:55.244773 22000 pending_rounds.cc:107] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:55.237345 23796 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:55.253892 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:55.224056 23936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.019s	sys 0.038s
W20260709 13:05:55.269428 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44555.5us
W20260709 13:05:55.269656 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59881.1us
I20260709 13:05:55.281663 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.281904 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.237286 24029 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:55.244367 24113 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:55.237202 24041 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:55.241199 23813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.005s	sys 0.015s
W20260709 13:05:55.273488 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 183488us
I20260709 13:05:55.285298 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.237154 23988 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:05:55.297843 24113 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:05:55.298063 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44263us
I20260709 13:05:55.298297 24113 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:55.298947 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.299275 24113 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.300420 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.291540 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:55.301318 21856 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.301265 24122 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:55.294798 21972 leader_election.cc:400] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:55.221935 24126 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.295198 23796 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.295589 24144 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.304157 24144 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.273622 23856 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:55.305277 23856 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.305670 23856 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.307080 24144 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:55.307644 23989 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:55.289953 23941 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.301586 24081 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:55.308382 21857 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:55.308552 24081 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.309092 24081 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.292021 23833 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:55.309324 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.309701 23833 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:55.310791 23833 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.300805 24113 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.265882 24069 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.280676 23716 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:55.314718 23716 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:55.315040 24069 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:55.315765 24148 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:55.315686 23716 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.316485 24148 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:55.317539 24148 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.273262 23829 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:55.276405 24099 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.280684 22000 raft_consensus.cc:2889] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:55.335310 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.335353 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.286190 24029 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:55.346628 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:55.341355 21856 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:55.347538 24083 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:55.361449 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.347302 21824 leader_election.cc:400] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:55.332796 23829 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:55.338510 23941 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:55.322604 23813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.006s	sys 0.015s
I20260709 13:05:55.321311 24041 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:55.367403 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.383256 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.349471 24086 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.370750 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.345958 23989 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:55.386598 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.352396 24081 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.361055 24139 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:55.396750 24146 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:55.397863 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.398823 24146 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.400427 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.400646 24144 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:55.401258 24138 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.401376 24144 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.401605 24041 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.402675 24138 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:55.403160 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.403707 24144 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:55.404029 24138 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.404397 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:55.366660 23937 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.405246 23937 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.407483 23937 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:55.402402 24146 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.410889 24081 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:55.413365 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.414358 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1802) took 1696 ms. Trace:
I20260709 13:05:55.414814 21851 rpcz_store.cc:276] 0709 13:05:53.717508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.141775 (+424267us) service_pool.cc:224] Handling call
0709 13:05:54.171560 (+ 29785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.222726 (+ 51166us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:54.384952 (+162226us) log.cc:844] Serialized 14 byte log entry
0709 13:05:55.070440 (+685488us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:55.075090 (+  4650us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:55.090483 (+ 15393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.414091 (+323608us) spinlock_profiling.cc:243] Waited 50.9 ms on lock 0x7b5400259c54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:55.414147 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.414297 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":301797,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":4652,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4237,"spinlock_wait_cycles":117075840,"thread_start_us":761444,"threads_started":2,"wal-append.queue_time_us":816072}
I20260709 13:05:55.418043 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.418566 24029 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.419484 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.420044 24029 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.374930 23941 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:55.425436 23941 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.427613 24138 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:55.428503 24138 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:55.429927 24138 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:55.431056 24138 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.432713 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:55.386322 23989 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.380801 24084 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:55.375162 24099 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:55.361167 24105 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:05:55.435096 23796 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:55.392248 24136 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:55.437198 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.435997 24139 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:55.454392 24150 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:55.476966 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:55.455869 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 316645us
I20260709 13:05:55.460858 23989 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:55.459717 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 284896us
I20260709 13:05:55.473340 22593 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:55.473987 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:55.368629 24126 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.459090 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:55.464241 23813 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:55.486357 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.493304 23813 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.493919 23813 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.471848 24099 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.493342 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.477828 24139 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.474972 24120 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.481882 24084 raft_consensus.cc:2764] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:05:55.509470 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.485227 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:55.483898 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.543797 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.488636 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.483846 23883 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:55.506948 24072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.012s	sys 0.010s
I20260709 13:05:55.483763 24134 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:55.503646 24146 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:55.540143 24099 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.537973 24133 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.497412 24069 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:05:55.491271 24144 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:05:55.503113 23937 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 13:05:55.552814 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67670.4us
I20260709 13:05:55.587280 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:55.564814 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 351017us
W20260709 13:05:55.561102 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70183.1us
W20260709 13:05:55.569773 24072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.012s	sys 0.010s
I20260709 13:05:55.504446 24136 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:05:55.590607 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:55.591704 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.556365 24083 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:55.553359 23796 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.571491 24086 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:55.573173 24146 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:55.618427 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 198995us
W20260709 13:05:55.619454 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.624419 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.625296 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:55.626364 21930 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:55.635018 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47846.8us
I20260709 13:05:55.640781 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.643069 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:55.643712 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 177849us
I20260709 13:05:55.648169 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:55.596156 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:55.622627 24134 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.648716 21930 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:05:55.665966 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.674021 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:55.622651 24146 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:55.677701 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.595383 23856 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.646003 24084 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:55.633670 24143 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.693559 21852 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:55.682092 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:55.686484 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.648298 24113 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.646353 24150 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:55.645629 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:55.666863 21851 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.585167 24133 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.711105 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:55.640328 23833 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:05:55.645448 24059 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:05:55.617821 24132 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:05:55.745321 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:55.693089 23883 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:55.681641 24041 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:55.730690 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.712519 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:55.692976 24110 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.684430 23988 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.663302 24103 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:05:55.688301 23796 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.745702 21855 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:55.798707 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:55.754875 21935 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:55.799492 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:55.713219 24106 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:55.755369 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.755626 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:55.756174 22007 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:55.756273 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:55.701819 23889 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:55.710278 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 118161us
I20260709 13:05:55.801039 22001 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:55.813112 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:55.814646 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:55.681156 23786 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:55.777323 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 90916us
I20260709 13:05:55.816115 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:55.681061 23936 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:05:55.747020 21853 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:55.746824 24064 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:55.642544 24131 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:05:55.800068 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.842895 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:55.698396 24157 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.144s
I20260709 13:05:55.785486 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.852428 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:55.829128 22003 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:55.839776 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1842) took 1647 ms. Trace:
I20260709 13:05:55.812377 23889 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.841073 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:55.843552 21935 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:55.677438 24134 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.843075 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.820819 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.843361 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.881251 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:55.882021 21933 raft_consensus.cc:2368] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:55.882082 21931 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:55.846382 21825 leader_election.cc:400] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:55.883108 21825 leader_election.cc:403] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:05:55.883764 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47219.8us
I20260709 13:05:55.798389 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.760028 24041 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.854430 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:55.798864 23988 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.910418 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:55.869786 21854 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:55.817508 23786 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:55.712093 24113 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.754164 23829 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.785390 24165 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.853667 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:55.913025 22003 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.872623 23889 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.874073 22588 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:55.786484 24110 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.854707 22592 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:56.142097 24162 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.274s	user 0.000s	sys 0.009s
I20260709 13:05:55.849202 24168 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:56.142937 24168 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:56.143337 24168 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:56.143991 24110 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:55.701191 24143 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.845701 23833 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:55.853416 22004 rpcz_store.cc:276] 0709 13:05:54.192461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.192620 (+   159us) service_pool.cc:224] Handling call
0709 13:05:54.697339 (+504719us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:55.839399 (+1142060us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5400169254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:55.839687 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202484864}
W20260709 13:05:55.906083 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 90060.9us
I20260709 13:05:55.908159 22005 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.839973 22006 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:05:55.850402 24148 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:55.811772 24126 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:55.854588 24084 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:55.850219 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:55.833716 24064 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:55.912364 23856 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:55.819607 23796 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:55.822899 24106 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:55.944125 24159 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 13:05:55.944978 24164 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:55.945783 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.848977 23937 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:05:55.947242 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:55.912591 23786 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:55.950523 24169 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:05:55.951486 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:55.974478 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:55.986780 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.144963 24029 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.724s	user 0.005s	sys 0.018s
I20260709 13:05:56.146639 24143 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:55.914647 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.146850 23833 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:56.133733 24173 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:56.176553 23833 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.176827 24173 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.177278 24173 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:56.178467 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.179770 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.181852 22003 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:56.183568 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:05:56.187353 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.148180 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:56.148650 24148 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:56.189704 24148 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.190739 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:56.192152 21899 leader_election.cc:400] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6.
I20260709 13:05:56.192423 21899 leader_election.cc:403] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:05:56.149823 24084 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:56.197973 24161 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.437s	user 0.007s	sys 0.000s
I20260709 13:05:56.198825 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1809) took 1055 ms. Trace:
I20260709 13:05:56.208361 21857 rpcz_store.cc:276] 0709 13:05:55.143163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.143703 (+   540us) service_pool.cc:224] Handling call
0709 13:05:55.144133 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.308822 (+164689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.198581 (+889759us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003d3454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:56.198764 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377309568}
I20260709 13:05:56.163246 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:56.171196 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.211699 22007 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.212093 22001 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:56.148527 22006 pending_rounds.cc:85] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 2
I20260709 13:05:56.153676 21931 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:05:56.139285 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 427031us
I20260709 13:05:56.214953 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:56.138268 24146 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:55.871301 24081 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:56.140185 23813 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.646s	user 0.015s	sys 0.006s
W20260709 13:05:56.175555 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.221233 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:56.148411 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.199340 24122 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:56.198931 24148 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:56.199868 24136 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:56.150350 22000 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:56.153231 23796 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.151261 24064 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.148813 24176 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.202476 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:56.134258 24133 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.135109 24180 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:05:56.117758 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 296279us
I20260709 13:05:56.423125 24166 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:05:56.134234 23988 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.213907 22006 pending_rounds.cc:107] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:05:56.464160 22006 raft_consensus.cc:2889] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:56.138064 23989 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.012s
W20260709 13:05:56.160588 23941 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.735s	user 0.006s	sys 0.018s
I20260709 13:05:56.468381 23941 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.469787 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.470585 23941 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.176101 24143 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:56.471925 22009 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:56.222159 22004 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:56.223403 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:56.232856 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 54610.6us
I20260709 13:05:56.233244 24189 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.473639 24189 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.474414 24166 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:56.474871 24166 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:56.475317 24171 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.309s	user 0.000s	sys 0.002s
I20260709 13:05:56.476027 24189 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.475728 24166 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:56.154096 24155 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.378s	user 0.000s	sys 0.009s
W20260709 13:05:56.476941 24172 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.343s	user 0.001s	sys 0.002s
I20260709 13:05:56.477118 24166 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.463752 24176 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.478412 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 581) took 1018 ms. Trace:
I20260709 13:05:56.479609 24176 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.241829 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.478789 22586 rpcz_store.cc:276] 0709 13:05:55.459515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.460958 (+  1443us) service_pool.cc:224] Handling call
0709 13:05:55.491334 (+ 30376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.553706 (+ 62372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.556309 (+  2603us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.617288 (+ 60979us) log.cc:844] Serialized 26 byte log entry
0709 13:05:55.914450 (+297162us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:55.927202 (+ 12752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.946407 (+ 19205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.476632 (+530225us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b240031bf98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d72109d 00007f3b7035310e 00007f3b7036a88c 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221
0709 13:05:56.478116 (+  1484us) raft_consensus.cc:1685] finished
0709 13:05:56.478139 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.478348 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1695,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":4493,"spinlock_wait_cycles":63812864,"thread_start_us":208514,"threads_started":1,"wal-append.queue_time_us":473102}
I20260709 13:05:56.461580 24149 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:56.481076 22000 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:56.390964 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.482594 22592 raft_consensus.cc:2410] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:56.217818 24081 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.483474 24081 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:56.486935 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 263605us
I20260709 13:05:56.488345 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:56.174201 24183 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.489519 24190 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:56.489362 24183 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.490082 24190 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:56.346006 24148 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.364447 22007 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:56.239708 24165 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.155670 24106 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.149957 24134 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.147503 24041 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.172821 24080 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:56.172772 24182 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:05:56.322681 24139 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.004s
I20260709 13:05:56.164319 23856 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.425442 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 204277us
I20260709 13:05:56.433740 23786 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.424124 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.517576 22587 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:05:56.156973 24175 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.156162 24168 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.349566 23829 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.364559 24131 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:56.300009 23796 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:56.221244 24099 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.628s	user 0.010s	sys 0.011s
I20260709 13:05:56.260774 23833 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.140491 21851 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:56.221135 23964 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:56.146548 24113 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.327783 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.433982 22588 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:56.366044 24146 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:56.347034 24153 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.180s	user 0.001s	sys 0.001s
W20260709 13:05:56.258481 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.532963 24099 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.539804 24099 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:56.416863 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 237736us
W20260709 13:05:56.543154 24156 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.767s	user 0.005s	sys 0.009s
W20260709 13:05:56.350250 24163 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.564s	user 0.007s	sys 0.010s
I20260709 13:05:56.328513 24108 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6.
I20260709 13:05:56.242231 23813 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:56.412789 23889 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.448638 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:56.372638 24086 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.370805 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:56.456562 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 254108us
I20260709 13:05:56.463483 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:56.466387 24174 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.467275 24160 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.707s	user 0.000s	sys 0.012s
I20260709 13:05:56.241576 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.565573 22002 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:56.565922 21899 leader_election.cc:400] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5.
I20260709 13:05:56.566421 21899 leader_election.cc:403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:05:56.567019 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:56.568284 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:56.568919 21933 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:56.569267 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:56.161805 24138 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.009s
I20260709 13:05:56.242862 22005 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:56.492027 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:56.510258 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:05:56.571807 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61642.6us
I20260709 13:05:56.568862 22006 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:56.514745 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.574131 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.574533 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.575284 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:56.468139 23989 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.494500 24084 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:56.582298 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:56.518404 24139 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.582901 21859 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:56.495010 24132 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.459146 24110 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:05:56.495796 24189 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:56.595726 21933 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:56.529234 23796 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:56.510571 24080 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:56.497846 24176 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:05:56.546913 22590 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:56.552248 23813 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.553864 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:56.554563 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 91280.7us
W20260709 13:05:56.621029 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.556026 22584 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:56.557698 24168 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.557391 24190 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.534617 24083 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:56.493392 24180 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.522184 23856 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.559163 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.559099 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.552492 23833 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.560989 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:56.561173 22586 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:56.562793 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:56.569900 21935 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:05:56.571745 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79774.9us
W20260709 13:05:56.520630 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.573581 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:56.577486 24181 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5.
I20260709 13:05:56.570308 24174 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.556620 24086 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.700472 21933 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:56.578485 23786 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.706880 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:56.578946 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:56.579456 23936 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:56.580966 23989 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.581274 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:56.582090 24170 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.437s	user 0.000s	sys 0.003s
I20260709 13:05:56.533505 24131 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:56.570391 24138 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.595512 24041 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:56.518566 24182 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.644131 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:56.555533 24166 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:56.556102 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:56.617692 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:56.649106 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:56.514039 24165 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.654693 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:56.712066 23936 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:56.596282 23889 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.657492 22006 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:56.742766 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:56.743386 22006 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.527626 24175 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.658736 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:56.617311 23796 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.749271 24196 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:56.749763 24196 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:56.750149 24196 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.750818 24175 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:56.752574 24200 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:56.752909 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:56.753328 22003 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:05:56.752501 21758 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 2 to 3, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.753584 21930 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:56.724700 23786 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:05:56.755849 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.756508 23786 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:56.616518 24110 consensus_queue.cc:1243] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:05:56.696455 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.756922 23786 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:56.650360 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:56.617031 24132 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.756628 23796 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:56.697328 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 134590us
I20260709 13:05:56.724746 24080 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.760139 24132 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.759609 24110 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:05:56.774171 23989 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:56.774664 23989 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:56.775622 23989 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.776687 23989 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.778934 24165 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.780383 21926 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:56.641741 24084 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.705722 21754 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 189) took 12455 ms (client timeout 15000 ms). Trace:
W20260709 13:05:56.708465 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 134941us
I20260709 13:05:56.684303 24184 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.614161 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.839390 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:56.708293 21860 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
W20260709 13:05:56.783617 24201 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:05:56.616602 24189 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.722432 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:56.776840 24064 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.712121 23833 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:56.803011 24084 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.727819 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:56.766429 24138 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.737316 21932 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:56.714192 24106 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:56.656266 21859 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [3.5-4.6]
W20260709 13:05:56.738821 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94800.7us
I20260709 13:05:56.711301 24134 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:05:56.714571 24183 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.749065 21928 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:56.934971 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:56.936400 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:56.780864 24182 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.763442 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:56.927935 24165 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.937902 22587 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.782989 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.761795 23796 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.759455 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:56.783771 23936 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.764412 23889 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:05:56.755874 21758 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.948175 21758 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:56.757859 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.763496 24174 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:56.774019 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:56.790490 22006 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
W20260709 13:05:56.702172 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 141212us
I20260709 13:05:56.952636 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:56.954056 22004 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:05:56.791529 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:56.955353 21930 raft_consensus.cc:2368] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:56.793569 21931 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:56.795328 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:56.956425 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:56.795943 24131 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.958076 22585 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 2: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:05:56.739375 24173 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.001s
I20260709 13:05:56.958911 24173 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:56.839453 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1847) took 1062 ms. Trace:
I20260709 13:05:56.960067 22001 rpcz_store.cc:276] 0709 13:05:55.777479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.221719 (+444240us) service_pool.cc:224] Handling call
0709 13:05:56.564109 (+342390us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:05:56.564365 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.582810 (+ 18445us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:56.583091 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.705306 (+122215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.705393 (+    87us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:56.705458 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.839211 (+133753us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b540042b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:56.839264 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.839396 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33952,"prepare.queue_time_us":495,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":53295,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":63872512,"thread_start_us":69386,"threads_started":1,"wal-append.queue_time_us":255022}
I20260709 13:05:56.961664 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:56.962235 22008 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:56.729530 24041 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:56.854188 21933 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.709847 24136 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:56.723546 24081 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:56.855490 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:56.853475 24209 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.121s	user 0.003s	sys 0.005s
I20260709 13:05:56.859339 24180 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.732728 22591 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:56.860985 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:56.712000 24099 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:56.871018 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 279) took 1302 ms. Trace:
I20260709 13:05:56.871706 23856 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:56.862962 24193 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.860484 24175 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:56.860877 23833 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:56.853322 24139 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.860496 24189 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:56.879606 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:56.802999 21754 rpcz_store.cc:269] 0709 13:05:44.245022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.287563 (+ 42541us) service_pool.cc:224] Handling call
0709 13:05:56.699844 (+12412281us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:53.563599 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:53.717629 (+154030us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:53.717689 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:05:53.717709 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:53.718649 (+   940us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:53.718721 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:53.718750 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:05:53.719438 (+   688us) tablet.cc:701] Row locks acquired
0709 13:05:53.719498 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:05:53.719703 (+   205us) write_op.cc:270] Start()
0709 13:05:53.719853 (+   150us) write_op.cc:276] Timestamp: P: 1783602353719632 usec, L: 0
0709 13:05:53.719878 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:05:53.771578 (+ 51700us) log.cc:844] Serialized 2898 byte log entry
0709 13:05:54.423749 (+652171us) op_driver.cc:464] REPLICATION: finished
0709 13:05:55.197370 (+773621us) write_op.cc:301] APPLY: starting
0709 13:05:55.197469 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:55.197750 (+   281us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:55.197876 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:55.198820 (+   944us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:55.199132 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:05:55.199179 (+    47us) write_op.cc:312] APPLY: finished
0709 13:05:55.199516 (+   337us) log.cc:844] Serialized 59 byte log entry
0709 13:05:55.199695 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:55.199807 (+   112us) write_op.cc:454] Released schema lock
0709 13:05:55.199873 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:55.200321 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:55.200352 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":382899,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":773323,"apply.run_cpu_time_us":2897,"apply.run_wall_time_us":2898,"num_ops":7,"prepare.queue_time_us":588278,"prepare.run_cpu_time_us":15364,"prepare.run_wall_time_us":382313,"raft.queue_time_us":222432,"raft.run_cpu_time_us":19389,"raft.run_wall_time_us":96941,"replication_time_us":703702,"thread_start_us":425243,"threads_started":4,"wal-append.queue_time_us":152759,"wal-append.run_cpu_time_us":12705,"wal-append.run_wall_time_us":2248165}]]}
I20260709 13:05:56.883868 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:56.939069 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:05:56.937417 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 364) took 1370 ms. Trace:
I20260709 13:05:56.938265 21928 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:56.961283 24183 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:56.721963 21858 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:56.963418 22006 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:56.739735 24135 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.975346 22587 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:56.873893 24080 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.854790 24184 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.000247 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:57.163061 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.178907 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.208806 21928 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:05:57.209745 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.210234 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.181373 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:57.087770 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:57.181731 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 242733us
I20260709 13:05:57.183262 22009 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:56.941094 24146 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.222993 24180 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.223637 24180 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.189069 22000 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:57.226413 24184 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.227211 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.228055 21928 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:05:57.190094 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 334674us
I20260709 13:05:57.215348 24062 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:57.229394 24062 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.229750 24062 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:57.231791 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.183656 21856 rpcz_store.cc:276] 0709 13:05:55.568687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.572126 (+  3439us) service_pool.cc:224] Handling call
0709 13:05:55.572549 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.572909 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.572944 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.573444 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:05:55.749120 (+175676us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:55.749159 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.749251 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.581760 (+832509us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b2400130988. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d72109d 00007f3b7035310e 00007f3b7036a88c 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221
0709 13:05:56.868360 (+286600us) raft_consensus.cc:1685] finished
0709 13:05:56.868384 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.870924 (+  2540us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42319,"raft.queue_time_us":766,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":14176,"spinlock_wait_cycles":41699200,"thread_start_us":103919,"threads_started":1,"wal-append.queue_time_us":248507}
I20260709 13:05:57.195945 21933 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:05:57.233177 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.235782 22006 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:57.237124 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76002us
I20260709 13:05:57.233212 22005 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.098685 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:57.178993 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 226472us
I20260709 13:05:57.227330 24180 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:57.179919 21932 rpcz_store.cc:276] 0709 13:05:55.567232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.567403 (+   171us) service_pool.cc:224] Handling call
0709 13:05:55.567778 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.873839 (+1306061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.937180 (+ 63341us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002e95d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:56.937362 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2689193728}
I20260709 13:05:57.242259 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.010808 24136 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.271783 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.244740 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.272763 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:57.249981 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:57.274575 21932 raft_consensus.cc:2410] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:56.998347 24041 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.243482 21857 raft_consensus.cc:2410] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:57.267518 24199 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.132s	user 0.003s	sys 0.006s
I20260709 13:05:57.268127 24166 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:05:57.269232 24138 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.187956 23833 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.018296 24083 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:57.182569 24099 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:57.278228 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:57.282552 21934 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:05:57.214813 24178 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.010s	sys 0.017s
I20260709 13:05:57.291929 22005 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:05:56.946178 24165 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.212404 24086 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.277045 24136 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:57.269289 24207 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:05:57.296198 24189 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:05:57.303308 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.211694 24072 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:57.238929 24132 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:57.319651 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 591836us
I20260709 13:05:57.295502 24139 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.297353 23989 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:57.328243 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:57.305852 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.329171 21932 raft_consensus.cc:2376] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3. Re-sending same reply.
I20260709 13:05:57.337522 24072 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:57.337891 24202 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:05:57.338222 24072 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:57.339310 24196 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
I20260709 13:05:57.339792 24196 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.212805 24193 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.277395 23889 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:05:57.342712 24193 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:57.212170 24175 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.345044 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.271163 23856 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.049s
W20260709 13:05:57.318619 24178 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.010s	sys 0.026s
I20260709 13:05:57.305449 24083 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:57.312737 24084 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.300750 24110 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:05:57.347909 24086 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.305414 24138 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.355548 24174 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:57.228833 23786 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.383985 24083 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.383705 24132 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:57.383147 24180 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.380697 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.387071 23856 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.356699 24139 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.415207 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.430831 21932 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.387059 24099 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.427950 21926 raft_consensus.cc:2376] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5. Re-sending same reply.
I20260709 13:05:57.424818 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:57.354492 24189 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.413374 24174 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:57.436419 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:57.449251 24099 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.450412 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:57.452497 21972 leader_election.cc:400] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:05:57.455858 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:57.425549 24193 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.413686 24086 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.423182 23786 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:57.453286 24216 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.433228 23856 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.453761 24084 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:57.446357 24139 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:57.447834 22584 consensus_meta.cc:287] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.747s	user 0.008s	sys 0.000s
I20260709 13:05:57.461447 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.488513 21934 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:57.462317 24136 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:57.465241 24212 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:05:57.449427 24174 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.468703 21932 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:57.477308 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:57.440699 24134 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:57.482048 24211 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:05:57.482192 24175 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:57.491173 22003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.004s
I20260709 13:05:57.426318 24180 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:57.496407 21975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.002s	sys 0.012s
W20260709 13:05:57.501796 22003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.010s	sys 0.004s
W20260709 13:05:57.504097 21975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.002s	sys 0.012s
W20260709 13:05:57.507429 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.478467 24099 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:57.510560 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.514930 21930 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:57.516454 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:57.488042 24138 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.487968 22584 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:57.499652 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.512367 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.496948 24072 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.507128 24110 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:57.486088 24193 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.535660 21932 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:57.518342 24216 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:57.523895 23941 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.016s
I20260709 13:05:57.509128 24196 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:57.546123 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:57.504045 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 265631us
W20260709 13:05:57.549400 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 328431us
I20260709 13:05:57.549679 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.550446 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:57.550617 23941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.009s	sys 0.016s
W20260709 13:05:57.537308 22584 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 507) took 1672 ms (client timeout 1804 ms). Trace:
I20260709 13:05:57.563385 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:57.566807 22004 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:57.564095 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.551004 24099 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:57.547806 24168 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.558279 24165 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:57.504591 24132 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:57.577801 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 299631us
I20260709 13:05:57.561431 24110 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:57.547967 24062 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.585127 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:57.567122 24138 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:57.569033 23989 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:57.589802 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:57.603292 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.582934 24099 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:57.586004 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.630570 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.618108 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:57.587323 24110 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:57.619448 21935 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:57.621863 24193 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.633991 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:57.635293 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:57.619918 24216 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:57.636195 24216 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.613261 24135 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:57.627434 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 458) took 1410 ms. Trace:
I20260709 13:05:57.638317 24216 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:57.599238 24086 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:57.627805 24064 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.754s	user 0.009s	sys 0.000s
I20260709 13:05:57.598778 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.628396 23989 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.594588 24180 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:57.657603 24222 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:57.657521 23889 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:57.658439 23889 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.658794 23889 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:57.659605 23889 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.660701 23889 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.640929 22006 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:57.663798 23833 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.664402 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:57.665123 22588 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:57.665832 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:57.666563 22006 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:05:57.641387 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:57.668184 24080 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:05:57.668459 22007 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.668798 24080 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.670883 24081 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:57.671500 21933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:05:57.671645 24080 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:57.646616 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.673559 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:57.566403 22584 rpcz_store.cc:269] 0709 13:05:55.865115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.868294 (+  3179us) service_pool.cc:224] Handling call
0709 13:05:56.448438 (+580144us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:57.528341 (+1079903us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61573376}
I20260709 13:05:57.651145 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.681123 21928 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:57.652907 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:57.683388 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:57.654803 24135 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:57.685957 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.686148 24135 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.686390 24180 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:57.686751 24135 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:57.687011 24180 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.687731 24135 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.689107 24180 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:57.656157 24178 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.660938 24072 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:57.667742 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:57.640153 22005 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:57.674294 21932 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:57.638105 22003 rpcz_store.cc:276] 0709 13:05:56.217677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.928369 (+710692us) service_pool.cc:224] Handling call
0709 13:05:56.934130 (+  5761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.962716 (+ 28586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.962742 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.963242 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.504388 (+541146us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:57.504435 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.504510 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.627201 (+122691us) raft_consensus.cc:1685] finished
0709 13:05:57.627219 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.627373 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49081,"raft.queue_time_us":162213,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":30494,"thread_start_us":540790,"threads_started":1,"wal-append.queue_time_us":624071}
W20260709 13:05:57.668815 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.649477 21859 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:57.700976 21932 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:57.678280 22007 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:57.583240 24168 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.682657 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.690325 21856 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:57.615816 24224 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.692404 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.720160 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:57.719107 22005 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:05:57.721071 22006 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:57.729090 22006 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:05:57.731014 22003 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:05:57.736600 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.737423 21855 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 2.
W20260709 13:05:57.741541 24106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.002s	sys 0.014s
I20260709 13:05:57.745682 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.685518 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.746376 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:57.758975 22004 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:05:57.745561 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.760522 21854 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:57.746708 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:57.750610 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 204556us
I20260709 13:05:57.729722 21927 raft_consensus.cc:2368] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
W20260709 13:05:57.756824 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 142046us
I20260709 13:05:57.684749 24136 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.764238 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.691054 23856 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:57.685086 21926 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:57.768858 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.787448 21857 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:05:57.742550 24086 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.756883 24219 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:57.769680 23936 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.810s	user 0.006s	sys 0.005s
I20260709 13:05:57.742420 24231 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.798964 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.747042 24227 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.689455 24099 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.801342 22004 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:05:57.748430 24106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.002s	sys 0.014s
I20260709 13:05:57.788990 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.803229 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.829011 21857 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.804061 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.745249 24178 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:57.746575 24224 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:57.746073 24110 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.765978 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:57.838227 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 140272us
W20260709 13:05:57.836613 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 113595us
W20260709 13:05:57.841411 24205 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.876s	user 0.000s	sys 0.003s
I20260709 13:05:57.845299 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:57.846309 22584 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:05:57.846645 24213 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.687s	user 0.002s	sys 0.000s
I20260709 13:05:57.859583 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:57.826582 24210 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:05:57.805114 24219 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:05:57.873394 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 138473us
I20260709 13:05:57.870819 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.877401 21858 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:57.871877 22004 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:05:57.872020 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:57.870117 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.784742 24196 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:05:57.875042 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:57.879534 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:57.878813 21854 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:05:57.835304 24086 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:57.837530 24146 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.833505 24106 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:57.865199 21853 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:57.837703 24227 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:57.898957 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:57.899106 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.899785 22585 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:57.900586 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.901538 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.889734 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.903151 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:57.904013 21853 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:57.906463 24235 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:57.903153 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:57.916385 21857 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:05:57.884075 24225 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.917171 24242 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:57.917603 24242 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.917923 24242 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.920040 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.879369 24180 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.927479 24173 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.921311 21857 raft_consensus.cc:2410] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:57.939690 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.924881 24224 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.930733 24062 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:57.931628 24214 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
W20260709 13:05:57.934546 24041 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.628s	user 0.006s	sys 0.003s
I20260709 13:05:57.935984 24178 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:57.931908 24189 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.941076 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 283) took 1365 ms. Trace:
I20260709 13:05:57.943388 24168 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:57.944484 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:57.946935 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.947656 21928 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:57.827320 23936 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:57.895821 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:57.916620 22585 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:05:57.922086 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:57.952442 21857 pending_rounds.cc:85] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 5
I20260709 13:05:57.901518 21854 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:57.986893 21857 pending_rounds.cc:107] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:57.944732 24242 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.987849 21854 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:05:57.988082 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:57.989775 21854 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:57.990957 21854 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:57.991185 24242 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:57.992481 21854 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:57.993587 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49172.7us
I20260709 13:05:57.994761 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:05:57.996790 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 76375.7us
I20260709 13:05:57.998126 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:57.999475 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:57.939440 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:57.838780 24231 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:57.987207 21857 raft_consensus.cc:2889] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:57.955188 24244 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:58.004294 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.006045 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:57.938467 23833 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:58.019385 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.020664 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:57.980185 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 213533us
I20260709 13:05:58.007230 24136 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.007718 24081 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:57.871137 24236 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 13:05:57.981006 23786 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.024262 22002 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:58.010489 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:58.013777 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:58.041090 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:57.993409 24238 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:05:58.030157 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 188498us
I20260709 13:05:58.043401 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.017452 24168 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:57.963846 24146 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:58.026000 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 259656us
I20260709 13:05:58.042397 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.008776 24245 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:58.011586 24166 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.012781 24246 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.025666 24081 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.993958 24041 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.032598 24064 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.044492 21929 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:58.055235 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:05:58.046275 24083 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 13:05:58.013028 24189 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:05:58.044656 24136 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.014961 24174 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.044740 23796 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:58.062144 21935 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:05:58.067472 24069 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.007s	sys 0.018s
I20260709 13:05:58.060448 24242 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:58.148263 22590 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:58.062629 24173 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.063120 24246 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.064585 24081 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.063727 24166 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.059710 24236 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.031654 23786 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:05:58.062590 24245 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:58.067051 24064 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.046648 24220 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:05:58.197335 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.226539 24174 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:57.984366 21851 rpcz_store.cc:276] 0709 13:05:56.575318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.578890 (+  3572us) service_pool.cc:224] Handling call
0709 13:05:56.579246 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.737236 (+157990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.738187 (+   951us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.767720 (+ 29533us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.463746 (+696026us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b1800434c20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:05:57.600562 (+136816us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:57.602279 (+  1717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.940563 (+338284us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540018e554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:57.940647 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.940731 (+    84us) raft_consensus.cc:1685] finished
0709 13:05:57.940765 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.940969 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154,"raft.queue_time_us":4887,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3746,"spinlock_wait_cycles":385126016,"thread_start_us":123808,"threads_started":1,"wal-append.queue_time_us":918488}
I20260709 13:05:58.175448 22003 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:58.227151 24064 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.227635 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:58.227810 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:58.228446 21851 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:58.011132 24219 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.229550 24219 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.213210 21933 raft_consensus.cc:606] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:58.223589 23936 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:58.222453 24146 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:58.234579 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.234707 24146 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.235435 24146 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.235754 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.236435 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.214098 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 591) took 1432 ms. Trace:
I20260709 13:05:58.237466 24236 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:58.237713 22588 rpcz_store.cc:276] 0709 13:05:56.781884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.782340 (+   456us) service_pool.cc:224] Handling call
0709 13:05:56.782652 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.665536 (+882884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.213765 (+548229us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b5400609ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:58.214023 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2028936064}
I20260709 13:05:58.238144 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:58.225330 24231 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:58.239962 23786 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:58.241019 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:58.241780 23889 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.558s	user 0.005s	sys 0.005s
I20260709 13:05:58.240628 24166 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.242136 24250 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.167273 24234 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:05:58.242851 24176 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:58.221102 24139 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.047219 24238 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.030642 24086 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.243793 24250 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.012544 24227 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.223260 24235 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:05:58.245985 24250 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:58.246660 21934 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:58.246868 24235 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.224732 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:58.228466 21856 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.071141 24083 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.061343 24168 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.253638 24083 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:58.253891 24041 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:58.255203 24168 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:05:58.255558 22002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1856) took 1715 ms (client timeout 1969 ms). Trace:
I20260709 13:05:58.237998 24236 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:05:58.256004 22002 rpcz_store.cc:269] 0709 13:05:56.540382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.596568 (+ 56186us) service_pool.cc:224] Handling call
0709 13:05:58.255321 (+1658753us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540038a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:58.255493 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969975680}
I20260709 13:05:58.257037 24236 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.218866 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:58.242192 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.243242 24176 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:58.201270 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.247453 24235 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.261373 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:58.261602 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60468.4us
I20260709 13:05:58.236835 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.164913 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1879) took 1373 ms. Trace:
I20260709 13:05:58.202617 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.258320 22005 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:58.263655 21935 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:58.048745 24106 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.262862 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:58.215108 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.273082 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.209379 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.226215 24245 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:58.268761 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 248093us
I20260709 13:05:58.293987 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.312561 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:58.313648 21859 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:58.320181 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:58.321349 21855 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:58.282449 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:58.026360 24244 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.263003 21930 rpcz_store.cc:276] 0709 13:05:56.791536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.962509 (+170973us) service_pool.cc:224] Handling call
0709 13:05:58.164634 (+1202125us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540017e154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:58.164829 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555719040}
I20260709 13:05:58.265945 21928 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:58.284361 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:58.334198 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 289169us
I20260709 13:05:58.334748 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.335527 24226 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:58.336018 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:05:58.284977 24069 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.009s	sys 0.032s
I20260709 13:05:58.260923 24190 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:58.290650 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 81360.6us
I20260709 13:05:58.337606 24226 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:58.338577 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:58.338706 24226 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:58.339358 24226 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.341506 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:58.342835 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.344198 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:58.344645 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.334657 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.345851 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.345973 24226 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:58.346437 21928 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:58.347275 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.347299 24226 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:58.341687 21933 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:58.348637 24226 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:58.350186 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.350502 24226 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.351548 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.352186 24244 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:58.352595 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.353163 24244 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:58.354023 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.355027 21935 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:58.355959 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.348655 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.359155 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.359766 21928 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:58.201010 23796 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.362676 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.260463 24176 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.280448 21856 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:58.369755 24176 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.360775 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:58.363134 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80727.1us
I20260709 13:05:58.237139 21927 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:05:58.363413 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 589) took 1601 ms. Trace:
I20260709 13:05:58.373885 22590 rpcz_store.cc:276] 0709 13:05:56.762436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.773097 (+ 10661us) service_pool.cc:224] Handling call
0709 13:05:56.773475 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.266963 (+1493488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.328392 (+ 61429us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54006b3ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:58.363343 (+ 34951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3161037056}
I20260709 13:05:58.374485 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.375290 21927 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:05:58.245337 24238 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:58.204834 23941 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:58.377113 24190 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:58.377460 23941 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.378126 24190 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:58.373363 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 309346us
I20260709 13:05:58.379698 23941 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:58.376539 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.381389 21927 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:58.382144 24099 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.385840 21927 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:58.390604 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:05:58.392547 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.393498 24099 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:58.398329 24099 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.402515 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.442842 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:58.444006 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:58.400986 21928 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:58.452595 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.287796 24246 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.283634 21853 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:05:58.308161 23889 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:58.354041 24244 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.463418 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:05:58.481895 21934 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:58.482988 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:58.403249 21929 raft_consensus.cc:2368] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
W20260709 13:05:58.469568 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 89316.8us
I20260709 13:05:58.446949 24099 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:58.305179 24136 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:58.403337 23796 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:58.506976 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 465914us
I20260709 13:05:58.474905 24220 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.308239 24138 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.162s
I20260709 13:05:58.305909 24173 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.461786 24242 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:58.467356 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:58.537827 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:05:58.481110 23941 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:58.566896 21854 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:58.508733 24261 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:58.514710 21927 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:05:58.566666 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 99412.9us
I20260709 13:05:58.445420 24166 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.503968 24135 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.576025 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.538704 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:58.580951 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:05:58.573158 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 229032us
I20260709 13:05:58.536779 24083 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.577572 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:58.600863 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:58.533888 24238 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:58.599364 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.567317 23941 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:58.603927 21933 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:58.572978 23936 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:58.570465 24261 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.540174 24220 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:58.607844 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.013s	sys 0.043s
W20260709 13:05:58.628968 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.013s	sys 0.043s
W20260709 13:05:58.629853 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 605542us
I20260709 13:05:58.631554 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:58.569481 24080 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:05:58.637607 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:58.603817 24174 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:58.615899 23941 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:58.604218 24242 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:58.602782 24196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.019s
I20260709 13:05:58.641532 21931 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:58.649264 24219 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:58.649848 24178 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.617381 23936 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.597760 23989 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 13:05:58.656087 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 166923us
I20260709 13:05:58.582594 24064 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.627441 24135 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:58.626444 24261 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.617770 21926 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:05:58.655292 24196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.007s	sys 0.019s
I20260709 13:05:58.651728 24174 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.607012 24083 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.634191 24244 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:05:58.639210 24138 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:05:58.654778 24242 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:58.713802 21852 raft_consensus.cc:2368] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
I20260709 13:05:58.639401 24136 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:05:58.664723 24219 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.626434 24238 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.665690 24235 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:58.668171 23936 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:58.639354 24269 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
W20260709 13:05:58.727259 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.667752 24270 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:58.751956 24219 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:58.711606 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:58.714577 24250 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:58.754310 24276 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:58.698724 24264 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:58.704910 24174 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.754453 24238 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:58.760039 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.781872 23856 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:05:58.754501 24255 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
W20260709 13:05:58.781927 24265 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:05:58.716881 24242 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:58.786813 24270 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:58.757296 24244 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
W20260709 13:05:58.872279 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.786083 24259 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:58.796499 24269 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:58.800588 24135 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:58.930274 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47936.9us
I20260709 13:05:58.927172 21853 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:58.927654 24259 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:58.853638 24173 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:58.966971 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 366151us
I20260709 13:05:58.941985 24274 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:05:58.946063 24271 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:05:58.941422 24246 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.876585 24264 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:05:58.967978 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 387175us
I20260709 13:05:58.954602 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:58.976619 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1851) took 1343 ms. Trace:
I20260709 13:05:58.978518 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:05:58.968143 24081 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.735s	user 0.007s	sys 0.003s
W20260709 13:05:58.984647 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 353158us
I20260709 13:05:58.977039 22000 rpcz_store.cc:276] 0709 13:05:57.633493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.635037 (+  1544us) service_pool.cc:224] Handling call
0709 13:05:57.635390 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.642125 (+  6735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.642154 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.672857 (+ 30703us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.023272 (+350415us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:58.023307 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.327199 (+303892us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b5400426154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:58.327261 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.947749 (+620488us) raft_consensus.cc:1685] finished
0709 13:05:58.947780 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.976554 (+ 28774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14995456,"thread_start_us":280758,"threads_started":1,"wal-append.queue_time_us":776289}
I20260709 13:05:58.985082 21823 leader_election.cc:400] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4.
I20260709 13:05:58.978902 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:58.992969 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:58.994837 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.973317 23833 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:05:58.996811 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:58.997695 22009 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:58.998606 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:58.999205 22002 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:59.001564 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:58.948230 24269 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:59.002635 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:05:59.004112 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.967509 24135 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:59.011880 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:58.969098 24250 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:59.047192 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:58.973748 24166 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:59.037730 21856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.006s	sys 0.014s
I20260709 13:05:58.987617 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:59.052313 21856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.008s	sys 0.014s
W20260709 13:05:59.047005 24223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.019s	sys 0.038s
I20260709 13:05:59.054322 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:59.058946 24264 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:58.976329 24235 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:05:59.061425 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.003804 24273 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:59.060925 21859 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:05:58.978339 23989 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
W20260709 13:05:59.047814 24260 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.443s	user 0.001s	sys 0.000s
I20260709 13:05:59.067677 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 380) took 1390 ms. Trace:
W20260709 13:05:59.083099 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.017s	sys 0.010s
W20260709 13:05:59.083673 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.017s	sys 0.010s
I20260709 13:05:59.081907 21932 rpcz_store.cc:276] 0709 13:05:57.677581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.732356 (+ 54775us) service_pool.cc:224] Handling call
0709 13:05:57.732663 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:58.043230 (+310567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.043299 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:58.047422 (+  4123us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.333943 (+286521us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:58.333978 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.334053 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.067400 (+733347us) raft_consensus.cc:1685] finished
0709 13:05:59.067427 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.067600 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12394,"raft.queue_time_us":73698,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":12519,"thread_start_us":531790,"threads_started":2,"wal-append.queue_time_us":474774}
W20260709 13:05:59.084048 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 596436us
I20260709 13:05:59.015604 24084 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:59.060461 24223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.021s	sys 0.038s
I20260709 13:05:59.070281 22000 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.016984 24190 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:59.050622 24138 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:59.055872 24267 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:59.017218 23786 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.087394 22001 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.073552 24166 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:59.085425 24064 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.091055 24286 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:59.055315 24220 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.002664 24253 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:05:59.010936 24173 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:05:59.115679 24269 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.103372 24166 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:59.123651 24270 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.085988 24274 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:59.134641 24166 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.111874 24292 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:59.071949 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.136155 21853 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:59.135901 24292 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.104228 24139 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.137481 24139 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:59.111505 24176 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.000s
I20260709 13:05:59.064174 23833 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:05:59.134645 24269 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:59.011835 24081 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.142321 24081 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.121342 23786 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.143647 24135 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.145027 24292 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.128774 24219 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.102628 24106 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:59.109280 24280 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:59.146065 24106 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:59.146238 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.146170 24280 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:59.146656 24106 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:59.146924 23796 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.001s
I20260709 13:05:59.080291 24273 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.148659 24273 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:59.150640 24099 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.000s
I20260709 13:05:59.151093 24099 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.092338 24041 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:59.147384 23796 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.153581 24041 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:59.145282 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.155488 23796 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:59.092653 24283 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:59.157148 24176 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:59.157642 24283 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.157697 24176 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.159220 21927 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:05:59.160288 24248 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.973s	user 0.003s	sys 0.019s
I20260709 13:05:59.089740 24138 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:59.147377 24280 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.139086 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.151422 24289 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.145414 23786 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.129102 24083 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.163857 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:59.188637 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:59.165562 22000 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:59.171890 22002 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:59.189429 21928 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:59.156728 24041 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.188659 21932 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:05:59.082401 24285 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:59.190244 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.112749 24238 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:59.160179 24176 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:59.197187 24280 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.197198 24238 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.198024 24238 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.199216 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1599) took 1345 ms. Trace:
I20260709 13:05:59.199658 21929 rpcz_store.cc:276] 0709 13:05:57.853834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.174669 (+320835us) service_pool.cc:224] Handling call
0709 13:05:59.190221 (+1015552us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b540017f554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:59.199138 (+  8917us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92950272}
I20260709 13:05:59.202167 23889 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:59.202632 23889 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:05:59.203037 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.203504 23889 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.154971 24106 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:59.205739 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1653) took 1498 ms (client timeout 1600 ms). Trace:
W20260709 13:05:59.206167 21852 rpcz_store.cc:269] 0709 13:05:57.707639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.707786 (+   147us) service_pool.cc:224] Handling call
0709 13:05:58.224563 (+516777us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:05:59.205552 (+980989us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:59.205682 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079525504}
I20260709 13:05:59.207597 21852 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:59.208177 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1835) took 1265 ms. Trace:
I20260709 13:05:59.208487 21852 rpcz_store.cc:276] 0709 13:05:57.942414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.207112 (+1264698us) service_pool.cc:224] Handling call
0709 13:05:59.207447 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.207964 (+   517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.208127 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:59.211562 21852 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:59.212244 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1663) took 1185 ms. Trace:
I20260709 13:05:59.073182 24259 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.213765 24259 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:59.214887 24263 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.461s	user 0.001s	sys 0.001s
I20260709 13:05:59.216429 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1657) took 1506 ms. Trace:
I20260709 13:05:59.216765 21857 rpcz_store.cc:276] 0709 13:05:57.709577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.950075 (+240498us) service_pool.cc:224] Handling call
0709 13:05:57.952251 (+  2176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:58.013783 (+ 61532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.013823 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:58.014291 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.478480 (+464189us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:58.478518 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.478604 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.214301 (+735697us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:05:59.216155 (+  1854us) raft_consensus.cc:1685] finished
0709 13:05:59.216174 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.216374 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10687872,"thread_start_us":463616,"threads_started":1,"wal-append.queue_time_us":689597}
I20260709 13:05:59.218442 23889 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:59.218550 21857 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.219086 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1823) took 1470 ms. Trace:
I20260709 13:05:59.216831 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.219683 21854 rpcz_store.cc:276] 0709 13:05:57.748840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.996065 (+247225us) service_pool.cc:224] Handling call
0709 13:05:57.996389 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.567384 (+570995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.218819 (+651435us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b5400017a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:59.219018 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311512832}
I20260709 13:05:59.212589 21852 rpcz_store.cc:276] 0709 13:05:58.027320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.211076 (+1183756us) service_pool.cc:224] Handling call
0709 13:05:59.211400 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.211982 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.212170 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:59.221459 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1856) took 1586 ms. Trace:
I20260709 13:05:59.221807 22005 rpcz_store.cc:276] 0709 13:05:57.634717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.721451 (+ 86734us) service_pool.cc:224] Handling call
0709 13:05:58.258123 (+536672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.263720 (+  5597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.221288 (+957568us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b540038a4d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:59.221399 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202044160}
I20260709 13:05:59.100617 24267 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.187747 22585 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:59.182215 23796 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:05:59.160392 24283 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.189220 24289 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.194298 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.194541 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.194924 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.196231 24261 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.205757 24099 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.219192 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1665) took 1191 ms. Trace:
I20260709 13:05:59.113965 24246 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.191800 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:59.224637 21854 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:59.220899 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.171895 24135 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:59.181695 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:05:59.328425 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.363080 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:59.363068 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.412986 22002 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:59.413420 22000 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:59.413888 22590 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:59.413844 21857 rpcz_store.cc:276] 0709 13:05:58.027816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.218118 (+1190302us) service_pool.cc:224] Handling call
0709 13:05:59.218387 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.218974 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.219145 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:59.402156 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1833) took 1515 ms. Trace:
I20260709 13:05:59.414956 22593 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
W20260709 13:05:59.403699 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 401095us
I20260709 13:05:59.364576 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.428017 21856 rpcz_store.cc:276] 0709 13:05:57.886995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.370476 (+483481us) service_pool.cc:224] Handling call
0709 13:05:58.371181 (+   705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.371737 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:58.372576 (+   839us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.401491 (+1028915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:59.401577 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:59.401642 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.401883 (+   241us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.402073 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":370,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":55965,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"thread_start_us":1028174,"threads_started":2}
I20260709 13:05:59.246261 24267 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.440511 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:05:59.444005 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.445080 22002 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:05:59.446357 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.447121 22002 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:05:59.448261 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.243849 24106 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.170544 24178 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.136s
I20260709 13:05:59.242322 24041 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.455684 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.479622 22005 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:59.473528 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.410900 24176 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.485244 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:59.410964 24234 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.480276 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.480948 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:05:59.489457 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 455404us
I20260709 13:05:59.495625 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:59.518898 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.522958 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:59.483155 24131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.013s	user 0.026s	sys 0.043s
W20260709 13:05:59.529398 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 88996us
W20260709 13:05:59.524469 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 137821us
I20260709 13:05:59.531550 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.364887 23796 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.479799 24269 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:05:59.537459 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.014s	sys 0.038s
W20260709 13:05:59.542482 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.014s	sys 0.038s
W20260709 13:05:59.542827 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 902513us
I20260709 13:05:59.537432 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:59.497063 24236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.237s	user 0.010s	sys 0.032s
I20260709 13:05:59.538964 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.482894 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:59.538190 21974 leader_election.cc:336] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.804s (SENT)
I20260709 13:05:59.540454 22590 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:59.528060 24086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.280s	user 0.032s	sys 0.019s
I20260709 13:05:59.295280 24138 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:59.535037 24131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.068s	user 0.026s	sys 0.043s
I20260709 13:05:59.552263 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.552527 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.553733 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:59.495702 24041 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.501180 24176 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.521335 24234 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.679052 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:05:59.709553 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 155904us
I20260709 13:05:59.608850 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 385) took 1262 ms. Trace:
I20260709 13:05:59.559906 22005 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.626497 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:59.711591 21933 rpcz_store.cc:276] 0709 13:05:58.346220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.351279 (+  5059us) service_pool.cc:224] Handling call
0709 13:05:59.608577 (+1257298us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54002a8d54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:05:59.608774 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578327296}
I20260709 13:05:59.712654 22000 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:59.712579 22587 raft_consensus.cc:2376] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6. Re-sending same reply.
W20260709 13:05:59.684186 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.618891 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.536861 21851 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:59.629318 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.734148 22589 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:59.713766 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:05:59.714195 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 170898us
W20260709 13:05:59.542726 24230 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.374s	user 0.030s	sys 0.043s
W20260709 13:05:59.610211 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72876.3us
I20260709 13:05:59.605185 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.719660 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:05:59.743981 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:05:59.720279 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.629352 24290 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:05:59.613124 24246 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:59.549686 24300 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.725711 24301 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.675011 24099 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.539455 24264 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:59.531852 24220 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:59.713171 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.736198 21856 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:05:59.780553 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63543.2us
I20260709 13:05:59.738909 22588 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:05:59.546748 24236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.287s	user 0.010s	sys 0.032s
I20260709 13:05:59.618528 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:59.745342 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:59.749687 21858 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:59.643637 21960 heartbeater.cc:646] Failed to heartbeat to 127.20.66.190:46167 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.20.66.190:46167 timed out after 15.000s (SENT)
W20260709 13:05:59.708143 24086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.032s	sys 0.019s
I20260709 13:05:59.754412 22589 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:59.600557 24173 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.176s
W20260709 13:05:59.614105 24295 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:05:59.560357 23936 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.777s	user 0.007s	sys 0.003s
W20260709 13:05:59.631968 24296 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:05:59.616346 23941 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.950s	user 0.008s	sys 0.000s
I20260709 13:05:59.765730 22005 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:59.797340 21855 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.614382 24261 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.709048 24289 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.640944 24138 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.706461 24181 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.766115 24230 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.597s	user 0.031s	sys 0.045s
I20260709 13:05:59.619297 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:59.621932 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:59.621410 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:59.610987 24292 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.541447 24139 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:59.779199 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.809800 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.807786 24318 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.736971 24299 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:59.781499 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.814044 21856 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
W20260709 13:05:59.801833 24165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.566s	user 0.010s	sys 0.022s
W20260709 13:05:59.784238 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.788379 22588 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:05:59.818050 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.789322 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.789541 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44283.2us
I20260709 13:05:59.789739 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.819072 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:59.551141 24269 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.790757 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:59.626065 24294 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:59.762362 24106 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:59.792203 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44441.8us
I20260709 13:05:59.624575 24238 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.794659 24173 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.769778 24246 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:59.793294 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1827) took 2008 ms. Trace:
I20260709 13:05:59.800117 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:05:59.792346 24131 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:59.806388 21857 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.795264 22584 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:05:59.779357 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.810642 24259 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.001s
I20260709 13:05:59.782089 24099 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.814929 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.809281 24289 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:05:59.866215 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1828) took 2065 ms. Trace:
I20260709 13:05:59.777199 24301 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.820226 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:05:59.764330 24229 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:59.821970 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.832402 24106 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:05:59.831761 24220 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:05:59.825162 21854 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:05:59.812736 24292 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.829419 24319 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.881649 24319 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.883036 24219 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:05:59.883884 24306 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:05:59.830214 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:05:59.875048 24106 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:59.887784 24174 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.009s	user 0.010s	sys 0.000s
W20260709 13:05:59.832082 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.898656 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.808389 24310 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:05:59.821159 24165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.586s	user 0.011s	sys 0.022s
I20260709 13:05:59.813719 24299 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.799144 23936 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.801115 23941 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:59.839114 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 195790us
I20260709 13:05:59.809523 24181 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:05:59.845613 24312 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:05:59.928902 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1836) took 1360 ms. Trace:
I20260709 13:05:59.929282 23936 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.929776 24196 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:05:59.930140 24303 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:05:59.930013 23936 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:59.931463 23936 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:59.931779 21927 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.933415 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:05:59.934613 24272 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.153s	user 0.001s	sys 0.001s
I20260709 13:05:59.937212 22585 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:59.937829 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 97775.5us
I20260709 13:05:59.905575 24174 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.941686 24269 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.942731 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.943135 24174 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:59.943379 22009 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.943699 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:59.944514 21929 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.850659 21857 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:05:59.851881 21856 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:05:59.946167 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:59.946249 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.946848 21857 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:59.947628 24251 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.792s	user 0.017s	sys 0.007s
I20260709 13:05:59.949385 22004 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:05:59.949750 24181 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.836601 21853 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 3: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:05:59.852293 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:59.950559 22590 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.951244 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.952540 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1859) took 2308 ms. Trace:
I20260709 13:05:59.857025 24041 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:05:59.859927 24285 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.565s	user 0.003s	sys 0.006s
I20260709 13:05:59.866671 21859 rpcz_store.cc:276] 0709 13:05:57.800484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.325724 (+525240us) service_pool.cc:224] Handling call
0709 13:05:58.441586 (+115862us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:58.465049 (+ 23463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.465104 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:58.519680 (+ 54576us) log.cc:844] Serialized 590 byte log entry
0709 13:05:59.060759 (+541079us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:59.182065 (+121306us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.182813 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.182926 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.865972 (+683046us) raft_consensus.cc:1685] finished
0709 13:05:59.865995 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.866151 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":216,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":32228,"thread_start_us":458638,"threads_started":1,"wal-append.queue_time_us":900548}
I20260709 13:05:59.953809 24285 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:59.954243 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:59.955070 21859 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:05:59.866919 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:05:59.870083 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49951.4us
I20260709 13:05:59.930207 24196 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:05:59.822649 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:05:59.874605 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:59.957295 24196 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.958138 21933 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:59.958638 24196 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.959075 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:59.904932 24323 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:05:59.959679 21933 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:05:59.902863 24176 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:59.903007 24234 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.960472 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:59.781158 24300 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.838351 24131 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:59.902452 23889 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.683s	user 0.007s	sys 0.003s
I20260709 13:05:59.961939 24131 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:59.835225 21851 rpcz_store.cc:276] 0709 13:05:57.784458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.419039 (+1634581us) service_pool.cc:224] Handling call
0709 13:05:59.491649 (+ 72610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:59.751110 (+259461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.791863 (+ 40753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:59.962190 23941 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.962574 24300 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.962957 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.929312 24301 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.963639 21851 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:05:59.929301 21852 rpcz_store.cc:276] 0709 13:05:58.568652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.221350 (+652698us) service_pool.cc:224] Handling call
0709 13:05:59.221630 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:59.221821 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.221844 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:59.227297 (+  5453us) log.cc:844] Serialized 26 byte log entry
0709 13:05:59.540286 (+312989us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:59.540338 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.540507 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.928690 (+388183us) raft_consensus.cc:1685] finished
0709 13:05:59.928709 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.928844 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":101316,"threads_started":1,"wal-append.queue_time_us":246769}
I20260709 13:05:59.864465 24267 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.847266 23796 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:05:59.840049 24080 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:05:59.879765 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:05:59.993782 21934 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:59.994107 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 127456us
W20260709 13:05:59.881080 24278 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.022s	user 0.000s	sys 0.010s
I20260709 13:05:59.875008 24264 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:59.871565 24139 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:05:59.884656 24319 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.888495 24220 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:05:59.838546 24173 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.839684 24322 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:05:59.891742 24289 consensus_queue.cc:1243] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:59.893126 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:05:59.829366 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.895494 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.902097 24242 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.953s	user 0.009s	sys 0.001s
I20260709 13:05:59.903535 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1822) took 2155 ms. Trace:
I20260709 13:05:59.859936 24099 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.904223 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:00.028733 22592 raft_consensus.cc:2368] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5.
I20260709 13:06:00.029196 21928 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:05:59.813150 24318 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.031687 24289 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:59.845997 24246 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:00.034149 24318 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.893543 24229 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:05:59.923921 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:00.036652 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:59.926201 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:00.037606 21928 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:06:00.037971 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 111852us
I20260709 13:05:59.924885 24280 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.039187 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:00.039386 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:05:59.922379 24299 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.947472 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:05:59.737574 22000 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:00.044471 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1873) took 1037 ms. Trace:
I20260709 13:06:00.044795 22000 rpcz_store.cc:276] 0709 13:05:59.006967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.722109 (+715142us) service_pool.cc:224] Handling call
0709 13:05:59.722421 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.044179 (+321758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.044418 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.045176 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:00.046105 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:05:59.827554 24086 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:59.952767 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.050556 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:05:59.953147 22006 rpcz_store.cc:276] 0709 13:05:57.643873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.746534 (+102661us) service_pool.cc:224] Handling call
0709 13:05:57.757665 (+ 11131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.790320 (+ 32655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.790349 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.790821 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.869792 (+ 78971us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:57.870255 (+   463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.870382 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.946977 (+2076595us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:05:59.952182 (+  5205us) raft_consensus.cc:1685] finished
0709 13:05:59.952212 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.952466 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32187,"raft.queue_time_us":32301,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":33663,"spinlock_wait_cycles":104544640,"thread_start_us":46365,"threads_started":1,"wal-append.queue_time_us":239943}
I20260709 13:05:59.955821 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:05:59.969897 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:00.052307 21933 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:05:59.977983 24238 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.979301 24110 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:59.961424 24106 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:05:59.936899 24292 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:05:59.917512 24310 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:05:59.986742 24273 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.801s	user 0.003s	sys 0.007s
I20260709 13:05:59.986770 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:00.062364 21856 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:59.987650 21857 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:59.987802 24234 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.956015 24166 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.056664 21934 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:05:59.995337 22557 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:00.015417 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.029531 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.997829 24264 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.997498 24041 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:00.027810 21858 rpcz_store.cc:276] 0709 13:05:57.748518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.993281 (+244763us) service_pool.cc:224] Handling call
0709 13:05:58.017205 (+ 23924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.786756 (+1769551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.903294 (+116538us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54003d2a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:05:59.903467 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3983659904}
W20260709 13:06:00.036746 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 112901us
I20260709 13:05:59.927048 24321 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.927433 24249 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.704s	user 0.008s	sys 0.010s
I20260709 13:06:00.040540 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:00.040614 21928 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:06:00.041395 24069 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:00.042092 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1608) took 2152 ms. Trace:
I20260709 13:06:00.041761 24229 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.051182 22004 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:00.055020 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:00.064229 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:00.053766 24086 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:05:59.988564 22009 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:06:00.066248 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70993.5us
I20260709 13:06:00.087181 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:00.114863 21935 rpcz_store.cc:276] 0709 13:05:57.890177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.409849 (+519672us) service_pool.cc:224] Handling call
0709 13:05:58.410309 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.569108 (+158799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.569152 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:58.569207 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:58.569251 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.041783 (+1472532us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:00.041847 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.042018 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25001,"raft.queue_time_us":25171,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":9677,"spinlock_wait_cycles":294722816}
W20260709 13:06:00.107204 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66964.3us
I20260709 13:06:00.154130 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:00.154408 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:00.042812 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:00.155758 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:00.086902 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:00.049459 24189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.900s	user 0.036s	sys 0.028s
I20260709 13:06:00.055075 24165 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:00.055455 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:00.159965 21856 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:00.160414 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.161259 22589 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:00.161399 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:00.161545 22000 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:00.129136 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:00.061573 24301 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 13:06:00.162312 23889 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:00.162722 23889 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:00.053769 23856 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:00.163100 23889 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:00.163309 23856 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.163713 23856 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.163483 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:00.132879 24302 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.520s	user 0.004s	sys 0.000s
I20260709 13:06:00.165140 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:00.136454 24314 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.166255 24314 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:00.166326 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1863) took 1172 ms. Trace:
I20260709 13:06:00.167188 24314 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.166850 22008 rpcz_store.cc:276] 0709 13:05:58.994072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.994218 (+   146us) service_pool.cc:224] Handling call
0709 13:05:58.994540 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:58.994780 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.994806 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:59.001326 (+  6520us) log.cc:844] Serialized 26 byte log entry
0709 13:05:59.449760 (+448434us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:59.449814 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.449903 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.132261 (+682358us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:00.165922 (+ 33661us) raft_consensus.cc:1685] finished
0709 13:06:00.165951 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.166213 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76179712,"thread_start_us":383898,"threads_started":1,"wal-append.queue_time_us":551945}
I20260709 13:06:00.138116 24226 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.123101 24321 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.170390 21927 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:00.163097 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:00.171917 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 84828.4us
W20260709 13:06:00.139360 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1606) took 2271 ms (client timeout 2024 ms). Trace:
I20260709 13:06:00.172948 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:00.173197 21931 rpcz_store.cc:269] 0709 13:05:57.867480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.241948 (+374468us) service_pool.cc:224] Handling call
0709 13:06:00.139101 (+1897153us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:00.139280 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877277440}
I20260709 13:06:00.174754 21931 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:06:00.175803 22004 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:00.176019 23889 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.180124 24321 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:00.141812 22586 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:00.181296 24062 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:00.142168 21929 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:00.181757 23889 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:00.181885 24062 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.182717 24062 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.185125 24283 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:00.054055 24139 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:06:00.186599 24139 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:00.187103 24139 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:00.188010 24139 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.144137 24106 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:00.035413 24242 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.189563 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.191334 24242 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:00.145202 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.191761 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.146111 24131 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.147323 24310 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:00.066540 24227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.819s	user 0.013s	sys 0.033s
W20260709 13:06:00.192884 24227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.945s	user 0.020s	sys 0.040s
I20260709 13:06:00.110574 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:00.194044 24131 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:00.068787 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:00.144881 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.195742 22003 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:00.156556 21933 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:00.157177 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:00.188385 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:00.158872 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.198935 22592 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:00.200489 24242 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:00.201546 24242 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:00.202803 24242 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.207150 24139 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:00.207779 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.208516 22588 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:00.210705 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.210368 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:00.213934 21928 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:00.214498 24132 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:00.213383 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.217698 21931 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:00.217759 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:00.225486 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:00.227481 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.228577 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.229787 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:00.230804 22586 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:00.228176 22004 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
W20260709 13:06:00.158876 24189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.080s	user 0.040s	sys 0.028s
I20260709 13:06:00.237721 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.238552 22009 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:00.053948 24280 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.245982 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.064690 24231 raft_consensus.cc:2764] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:00.239614 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.241166 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:00.254395 22006 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:06:00.241510 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.150091 24138 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:00.243382 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.088169 24258 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.067873 24264 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.150496 24238 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:00.241748 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 87936.1us
I20260709 13:06:00.241727 21858 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:00.156883 24086 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:00.248085 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:00.263774 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.276221 22006 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:00.277237 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:00.278769 21856 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:00.274853 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:00.272109 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:00.287602 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.088644 24234 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.289623 22006 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:00.257014 24131 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:00.067997 24273 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.289062 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:00.289716 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.309170 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.319703 22006 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:06:00.319643 22009 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:00.320638 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:00.321573 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.321843 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1854) took 2687 ms. Trace:
I20260709 13:06:00.317736 24267 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:00.322189 22007 rpcz_store.cc:276] 0709 13:05:57.634234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.707768 (+ 73534us) service_pool.cc:224] Handling call
0709 13:05:57.708206 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.733223 (+ 25017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.733257 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.743163 (+  9906us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.956965 (+213802us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:57.957058 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.957162 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.926797 (+1969635us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:00.321565 (+394768us) raft_consensus.cc:1685] finished
0709 13:06:00.321587 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.321788 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4071,"raft.queue_time_us":103,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":17313,"spinlock_wait_cycles":136869248,"thread_start_us":152431,"threads_started":1,"wal-append.queue_time_us":311754}
W20260709 13:06:00.326764 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.293721 24322 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:00.290203 24189 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:00.339756 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:00.289915 24106 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:00.354171 22009 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:00.355074 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1877) took 1346 ms. Trace:
I20260709 13:06:00.268261 24138 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.307721 24131 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:00.355944 22009 rpcz_store.cc:276] 0709 13:05:59.008212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.330753 (+1322541us) service_pool.cc:224] Handling call
0709 13:06:00.353824 (+ 23071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.354755 (+   931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.355003 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:00.378831 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.886s (SENT) [suppressed 30 similar messages]
I20260709 13:06:00.358599 22006 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:00.371349 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1879) took 1362 ms. Trace:
W20260709 13:06:00.388614 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.388161 22007 rpcz_store.cc:276] 0709 13:05:59.008848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.334944 (+1326096us) service_pool.cc:224] Handling call
0709 13:06:00.341672 (+  6728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.370817 (+ 29145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.370870 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:00.370928 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:00.371002 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.371112 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.371273 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28690,"raft.queue_time_us":39555}
I20260709 13:06:00.382249 24131 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.417518 22007 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:00.418316 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 475) took 1320 ms. Trace:
I20260709 13:06:00.418985 22008 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:00.418648 22007 rpcz_store.cc:276] 0709 13:05:59.097767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.403080 (+1305313us) service_pool.cc:224] Handling call
0709 13:06:00.417220 (+ 14140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.418021 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.418234 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.419945 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:00.423583 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.419638 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 478) took 1319 ms. Trace:
I20260709 13:06:00.428804 21928 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.428478 22008 rpcz_store.cc:276] 0709 13:05:59.099902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.418443 (+1318541us) service_pool.cc:224] Handling call
0709 13:06:00.418749 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.419420 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.419589 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.430866 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 483) took 1289 ms. Trace:
I20260709 13:06:00.431349 22008 rpcz_store.cc:276] 0709 13:05:59.141283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.429628 (+1288345us) service_pool.cc:224] Handling call
0709 13:06:00.430043 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.430416 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.430457 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:00.430515 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:00.430559 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.430634 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.430781 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.432689 22006 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:00.435195 22007 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:00.435314 22006 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:00.436574 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 482) took 1295 ms. Trace:
I20260709 13:06:00.360217 21851 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.437206 22007 rpcz_store.cc:276] 0709 13:05:59.141046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.425674 (+1284628us) service_pool.cc:224] Handling call
0709 13:06:00.434893 (+  9219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.435968 (+  1075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.436224 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.387073 24318 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.354393 24189 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.446561 22009 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:00.447465 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 474) took 1350 ms. Trace:
I20260709 13:06:00.448027 22009 rpcz_store.cc:276] 0709 13:05:59.097057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.400333 (+1303276us) service_pool.cc:224] Handling call
0709 13:06:00.446153 (+ 45820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.447094 (+   941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.447371 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.436518 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 484) took 1295 ms. Trace:
I20260709 13:06:00.450696 21934 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.451012 22006 rpcz_store.cc:276] 0709 13:05:59.141529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.434798 (+1293269us) service_pool.cc:224] Handling call
0709 13:06:00.435063 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.436270 (+  1207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.436461 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:00.345510 24322 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:00.454778 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 462) took 3962 ms. Trace:
I20260709 13:06:00.398383 24328 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.331077 24227 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:06:00.460927 22001 rpcz_store.cc:276] 0709 13:05:56.492305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.237594 (+745289us) service_pool.cc:224] Handling call
0709 13:05:57.237892 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.102268 (+1864376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.454493 (+1352225us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b5400425ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:00.454698 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4252702592}
I20260709 13:06:00.479069 22001 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:06:00.498168 22000 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:00.507723 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1868) took 1502 ms. Trace:
I20260709 13:06:00.508313 22005 rpcz_store.cc:276] 0709 13:05:59.005472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.482780 (+477308us) service_pool.cc:224] Handling call
0709 13:06:00.507451 (+1024671us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b540040d154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:00.507638 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88588032}
I20260709 13:06:00.511224 22005 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:00.512629 22005 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:00.517077 24146 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.517664 24146 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:00.518712 24146 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.520437 24146 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.523186 21927 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:00.526368 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1832) took 2677 ms. Trace:
I20260709 13:06:00.526891 21855 rpcz_store.cc:276] 0709 13:05:57.848663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.332958 (+484295us) service_pool.cc:224] Handling call
0709 13:05:58.333541 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.797804 (+1464263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.526049 (+728245us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400059bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:00.526282 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3365859584}
I20260709 13:06:00.528831 22001 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:00.446174 24189 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.429047 24301 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:00.546828 22001 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:00.420413 24310 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:06:00.549252 22000 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:00.549656 21934 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:06:00.479125 24329 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.092s	user 0.000s	sys 0.002s
I20260709 13:06:00.556080 21928 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
W20260709 13:06:00.561852 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.037s	sys 0.042s
W20260709 13:06:00.562343 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.037s	sys 0.042s
W20260709 13:06:00.562932 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 732719us
I20260709 13:06:00.563876 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:00.475126 24322 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.471302 24173 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:00.439585 24318 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:00.588975 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 257425us
I20260709 13:06:00.590871 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:00.584697 21825 leader_election.cc:400] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5.
W20260709 13:06:00.547022 24256 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.034s	user 0.045s	sys 0.049s
I20260709 13:06:00.592073 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:00.592427 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:00.593209 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:00.593806 21855 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:00.593711 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:00.574064 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 195276us
W20260709 13:06:00.604077 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.475688 24110 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
W20260709 13:06:00.615250 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.615382 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:00.610411 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:00.597496 24256 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.380s	user 0.045s	sys 0.050s
I20260709 13:06:00.575627 24292 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:00.629387 22008 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:06:00.485519 24328 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:00.643774 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80003.4us
I20260709 13:06:00.645195 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:00.652905 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:00.653648 21856 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.622128 24250 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:00.566326 24301 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:00.689158 24338 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:06:00.690869 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 733632us
I20260709 13:06:00.635931 24247 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:00.659186 24256 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:00.668301 24132 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:00.617441 24072 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:06:00.616240 24080 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:06:00.672252 24250 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:00.692438 24316 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.697698 23889 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.658564 24173 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:00.737560 24166 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.003s
I20260709 13:06:00.739061 24301 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:00.739573 24138 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:00.745703 24236 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:00.746270 24318 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:00.748543 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:00.714428 24256 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:00.749735 23856 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.000s
W20260709 13:06:00.750535 24069 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.000s
I20260709 13:06:00.751462 24328 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:00.755793 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:00.756291 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:00.758912 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.759090 21860 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:06:00.714269 24226 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.545s	user 0.016s	sys 0.002s
I20260709 13:06:00.671921 24106 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 13:06:00.742473 24062 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.000s
I20260709 13:06:00.760008 24132 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:00.764688 24345 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:00.767486 21856 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:00.796010 21933 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:00.802037 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47918.3us
I20260709 13:06:00.760648 24316 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:00.759675 24258 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:00.786983 24238 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
I20260709 13:06:00.788486 23889 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.838564 24106 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.788252 24173 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:06:00.839085 24106 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:00.768859 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:00.808182 24081 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.822204 24345 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:00.768304 24273 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:00.820715 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:00.772420 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:00.838783 24238 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:00.803184 24041 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.862520 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:00.813233 24110 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:00.867859 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.816201 21927 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.815630 24069 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:00.802848 24331 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:00.889642 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.785826 24131 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.821990 24320 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.806811 23856 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.946635 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:01.018603 21856 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:00.960469 21933 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:00.862334 24345 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.840132 24062 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:00.865837 24138 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.944339 21928 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:01.013532 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:01.022145 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:00.868465 24226 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.781718 24250 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.032822 21928 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:00.933213 24166 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.956419 24299 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.000s
I20260709 13:06:00.876735 24041 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:00.933665 24110 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:01.037549 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 388361us
I20260709 13:06:00.961336 24289 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.865213 24072 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 13:06:00.865433 24081 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:00.868144 24267 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.156s
I20260709 13:06:01.081552 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:00.963692 24331 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:01.084749 21859 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:01.017644 24264 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.000s
I20260709 13:06:00.963274 24165 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.963014 24321 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.090921 22592 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:06:01.125984 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.146155 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.219410 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:01.257905 22008 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:01.037539 24328 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:01.039134 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:01.192332 21856 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.250118 22003 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:06:01.252597 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.022s	sys 0.071s
W20260709 13:06:01.264721 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.023s	sys 0.071s
I20260709 13:06:01.009372 24301 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:01.039140 24146 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.510s	user 0.016s	sys 0.000s
W20260709 13:06:01.265429 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 555860us
I20260709 13:06:01.086325 24110 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.267252 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:00.962462 24347 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.313s
I20260709 13:06:01.020561 24283 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.232s
W20260709 13:06:01.190312 24300 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.023s	sys 0.074s
I20260709 13:06:01.324254 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:01.041772 24069 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.230s
I20260709 13:06:01.036365 24138 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.322724 21856 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:01.258657 24250 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:01.311160 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.017s	sys 0.033s
W20260709 13:06:01.331522 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.018s	sys 0.033s
I20260709 13:06:01.331137 24231 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:06:01.333089 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 681402us
I20260709 13:06:01.334059 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:01.260061 24328 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:01.338570 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:01.042515 24226 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.263298 24139 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.246766 24331 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:01.329504 24220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.045s	sys 0.097s
W20260709 13:06:01.348644 24220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.046s	sys 0.098s
I20260709 13:06:01.255653 24132 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:01.355819 21931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.021s	sys 0.015s
W20260709 13:06:01.356346 21931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.021s	sys 0.015s
W20260709 13:06:01.358585 21974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.017s	sys 0.026s
I20260709 13:06:01.343747 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:01.088652 24289 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.328125 24273 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:01.324432 24245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.572s	user 0.025s	sys 0.053s
I20260709 13:06:01.346220 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:01.359112 21974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.018s	sys 0.027s
I20260709 13:06:01.259632 24321 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:01.324337 24300 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.023s	sys 0.075s
I20260709 13:06:01.350716 24220 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:01.345068 24313 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.262858 24264 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.266204 24166 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.082823 24299 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:01.364461 24247 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.018s	sys 0.048s
I20260709 13:06:00.876444 24322 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.269637 24165 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.030194 24320 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.345064 24342 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:06:01.040920 24316 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:01.385224 22590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.010s	sys 0.057s
W20260709 13:06:01.368278 24229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.011s	sys 0.033s
W20260709 13:06:01.366412 24245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.614s	user 0.025s	sys 0.053s
I20260709 13:06:00.866544 24256 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.357218 24139 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.388274 24220 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:01.342187 24041 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.392670 23889 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.312294 24301 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.408497 22007 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:01.398955 22590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.010s	sys 0.057s
I20260709 13:06:01.382249 24313 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:01.393708 24099 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.007s	sys 0.028s
W20260709 13:06:01.404745 24229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.012s	sys 0.034s
W20260709 13:06:01.394443 24247 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.018s	sys 0.048s
W20260709 13:06:01.418118 24189 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.841s	user 0.010s	sys 0.001s
I20260709 13:06:01.413285 24316 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.419245 24256 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:01.421346 24325 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.014s	sys 0.041s
I20260709 13:06:01.440348 24247 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:01.445338 24321 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.446256 24139 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.418782 24220 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:01.393177 24328 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.466579 24345 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.467130 24234 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.468927 24146 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:01.469511 24146 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:01.469995 24345 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.470603 24146 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:01.473147 24351 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:06:01.475317 24280 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.477449 24173 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:01.451436 24325 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.014s	sys 0.042s
I20260709 13:06:01.456028 24353 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:01.481768 24348 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 13:06:01.491194 24328 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.489354 24353 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:01.499570 24354 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:01.501467 24289 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.512810 24299 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:01.517251 24322 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.542167 24353 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.508196 24354 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.551882 24353 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:01.566947 24360 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:06:01.571583 24314 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.581748 24313 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.596217 24354 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:01.597187 24318 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.000s	sys 0.006s
W20260709 13:06:01.602833 24318 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.001s	sys 0.006s
W20260709 13:06:01.601858 24341 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.130s	user 0.002s	sys 0.008s
I20260709 13:06:01.611347 24331 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.614389 24331 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:01.615092 24355 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.213s	user 0.007s	sys 0.000s
W20260709 13:06:01.667917 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 321598us
W20260709 13:06:01.679534 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.682231 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.683578 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1909) took 1766 ms. Trace:
I20260709 13:06:01.687796 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:01.691359 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 398) took 1633 ms. Trace:
W20260709 13:06:01.691754 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.670877 21851 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.208s	user 0.010s	sys 0.001s
I20260709 13:06:01.684185 21930 rpcz_store.cc:276] 0709 13:05:59.917199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.919117 (+  1918us) service_pool.cc:224] Handling call
0709 13:05:59.924706 (+  5589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.802665 (+877959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.802694 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:00.802744 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:00.802777 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.682817 (+880040us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54002e7cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:01.682939 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.683303 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32319,"raft.queue_time_us":4573,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":10916,"spinlock_wait_cycles":2008610688}
I20260709 13:06:01.695044 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1917) took 1729 ms. Trace:
I20260709 13:06:01.692065 21931 rpcz_store.cc:276] 0709 13:06:00.057675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.268086 (+210411us) service_pool.cc:224] Handling call
0709 13:06:00.303692 (+ 35606us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:00.304133 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.304167 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:00.325389 (+ 21222us) log.cc:844] Serialized 52 byte log entry
0709 13:06:01.372353 (+1046964us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:01.688977 (+316624us) log.cc:844] Serialized 14 byte log entry
0709 13:06:01.690412 (+  1435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.690610 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.690695 (+    85us) raft_consensus.cc:1685] finished
0709 13:06:01.690716 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.691113 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5946,"raft.queue_time_us":4747,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"thread_start_us":974718,"threads_started":1,"wal-append.queue_time_us":1150252}
W20260709 13:06:01.672732 21929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.274s	user 0.022s	sys 0.046s
W20260709 13:06:01.728413 21929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.342s	user 0.023s	sys 0.046s
W20260709 13:06:01.671348 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.730180 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 396) took 1673 ms. Trace:
I20260709 13:06:01.730682 21929 rpcz_store.cc:276] 0709 13:06:00.057103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.205568 (+148465us) service_pool.cc:224] Handling call
0709 13:06:00.206053 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.206322 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:00.359671 (+153349us) log.cc:844] Serialized 14 byte log entry
0709 13:06:01.729408 (+1369737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:01.729533 (+   125us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:01.729695 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.729820 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.730117 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":548116,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"thread_start_us":1451585,"threads_started":2,"wal-append.queue_time_us":1184120}
I20260709 13:06:01.735392 21929 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:01.410986 21935 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:06:01.740159 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 404) took 1612 ms. Trace:
I20260709 13:06:01.740638 21935 rpcz_store.cc:276] 0709 13:06:00.127956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.154160 (+ 26204us) service_pool.cc:224] Handling call
0709 13:06:01.739768 (+1585608us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400229054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.739973 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2885192960}
I20260709 13:06:01.742846 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1928) took 1135 ms. Trace:
I20260709 13:06:01.745098 21929 rpcz_store.cc:276] 0709 13:06:00.606938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.734052 (+1127114us) service_pool.cc:224] Handling call
0709 13:06:01.734676 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.742460 (+  7784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.742748 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:01.692736 21851 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
W20260709 13:06:01.719493 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1648) took 4067 ms (client timeout 1726 ms). Trace:
W20260709 13:06:01.748234 21860 rpcz_store.cc:269] 0709 13:05:57.651922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.652617 (+   695us) service_pool.cc:224] Handling call
0709 13:06:01.718902 (+4066285us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.719154 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6359009152}
W20260709 13:06:01.703202 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.695473 22002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 472) took 2600 ms (client timeout 2966 ms). Trace:
I20260709 13:06:01.674516 21857 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:01.673250 21854 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:01.427137 24041 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.705089 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 309) took 2084 ms. Trace:
I20260709 13:06:01.719072 21930 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:01.717523 21934 rpcz_store.cc:276] 0709 13:05:59.965453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.163241 (+197788us) service_pool.cc:224] Handling call
0709 13:06:01.694470 (+1531229us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.694705 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643462528}
W20260709 13:06:01.774804 22002 rpcz_store.cc:269] 0709 13:05:59.095366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.536459 (+441093us) service_pool.cc:224] Handling call
0709 13:06:01.694819 (+2158360us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54001982d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.695152 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4149325184}
W20260709 13:06:01.838167 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.435923 24099 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.007s	sys 0.029s
I20260709 13:06:01.782589 24267 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:01.783288 24264 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:01.791568 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.009s
W20260709 13:06:01.795171 21928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.007s
I20260709 13:06:01.786724 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:01.832901 24310 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.833678 24138 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:01.693585 22004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.003s	sys 0.008s
W20260709 13:06:01.408819 24080 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.005s	sys 0.064s
I20260709 13:06:01.424891 23856 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.785516 24318 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:01.835363 21859 rpcz_store.cc:276] 0709 13:05:59.620408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.063844 (+443436us) service_pool.cc:224] Handling call
0709 13:06:00.102233 (+ 38389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.247446 (+1145213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.704343 (+456897us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b540010ca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:01.704798 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258918272}
W20260709 13:06:01.790778 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1679) took 2127 ms (client timeout 2826 ms). Trace:
I20260709 13:06:01.843206 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1927) took 1232 ms. Trace:
W20260709 13:06:01.849627 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.002s	sys 0.009s
W20260709 13:06:01.852768 21928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.007s	sys 0.007s
I20260709 13:06:01.866417 21930 rpcz_store.cc:276] 0709 13:06:00.606547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.717658 (+1111111us) service_pool.cc:224] Handling call
0709 13:06:01.718367 (+   709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.835903 (+117536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.839210 (+  3307us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:01.857391 24381 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:06:01.857257 24289 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:01.858470 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.858613 22006 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:01.859212 24345 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:01.431092 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:01.861140 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 535) took 1642 ms. Trace:
W20260709 13:06:01.874081 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.013s	sys 0.016s
W20260709 13:06:01.874581 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.013s	sys 0.016s
I20260709 13:06:01.874930 24357 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:01.875604 24357 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:01.880053 24220 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.881386 24364 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.882372 24364 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.504665 24247 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.885493 24289 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.862718 22009 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 13:06:01.858515 22588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.012s	sys 0.013s
I20260709 13:06:01.853652 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 614) took 1164 ms. Trace:
I20260709 13:06:01.444814 24189 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.861162 24283 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:01.450718 24131 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.846724 24267 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:01.875787 24139 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:01.900410 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 175753us
I20260709 13:06:01.900832 24139 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.902072 24379 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:01.902886 24379 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.903640 24139 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.891166 24226 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.907312 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:01.908329 22003 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:01.909793 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:01.910286 22008 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:01.910207 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:01.910808 22002 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:01.911850 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.011s	sys 0.028s
W20260709 13:06:01.912281 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.011s	sys 0.029s
I20260709 13:06:01.915156 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:01.915658 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.915982 22000 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:01.908813 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:01.916980 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:01.918036 24226 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:01.918330 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 490) took 2066 ms. Trace:
I20260709 13:06:01.918833 24226 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.919586 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:01.921373 22000 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:01.922614 24301 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.922487 24345 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.918835 22007 rpcz_store.cc:276] 0709 13:05:59.851590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.502424 (+650834us) service_pool.cc:224] Handling call
0709 13:06:00.502795 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.421769 (+918974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.917786 (+496017us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b540033ead4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:01.918131 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082341760}
I20260709 13:06:01.924263 24320 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.925895 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:01.927147 24356 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:01.928066 24356 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:01.929836 24256 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:01.930505 24354 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.930675 24256 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.931286 24354 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:01.935828 21856 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:01.937960 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1844) took 1339 ms. Trace:
I20260709 13:06:01.938464 21856 rpcz_store.cc:276] 0709 13:06:00.598697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.343416 (+744719us) service_pool.cc:224] Handling call
0709 13:06:01.937565 (+594149us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.937830 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323093888}
I20260709 13:06:01.896332 24234 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:01.897117 21926 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:01.940436 24310 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.941205 24353 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:01.941562 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1929) took 1334 ms. Trace:
I20260709 13:06:01.942173 21926 rpcz_store.cc:276] 0709 13:06:00.607524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.772046 (+164522us) service_pool.cc:224] Handling call
0709 13:06:01.941247 (+1169201us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54003066d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.941458 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098457984}
I20260709 13:06:01.912236 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:01.943742 22005 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:01.945847 24166 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.946111 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 403) took 1740 ms. Trace:
I20260709 13:06:01.946511 21933 rpcz_store.cc:276] 0709 13:06:00.114765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.026090 (+911325us) service_pool.cc:224] Handling call
0709 13:06:01.087377 (+ 61287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.087847 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.087889 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.257655 (+169766us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.822038 (+564383us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:01.822119 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.822261 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.830188 (+  7927us) raft_consensus.cc:1685] finished
0709 13:06:01.830222 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.855443 (+ 25221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5403,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":55901,"thread_start_us":480195,"threads_started":1,"wal-append.queue_time_us":495470}
I20260709 13:06:01.947638 24234 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.948966 21860 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:01.948475 24234 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:01.949685 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 316) took 1358 ms. Trace:
I20260709 13:06:01.950057 21860 rpcz_store.cc:276] 0709 13:06:00.591651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.921389 (+1329738us) service_pool.cc:224] Handling call
0709 13:06:01.922130 (+   741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.949401 (+ 27271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.949608 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:01.951222 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:01.951761 24234 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.952435 24234 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:01.954110 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:01.955799 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:01.956497 21859 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:01.956760 24354 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:01.958921 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:01.960433 24165 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:01.961707 24318 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:01.896006 22588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.012s	sys 0.013s
I20260709 13:06:01.941692 24353 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:01.901388 24138 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:01.938349 24267 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:01.889714 22009 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 3
I20260709 13:06:01.873847 22592 rpcz_store.cc:276] 0709 13:06:00.218101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.222161 (+  4060us) service_pool.cc:224] Handling call
0709 13:06:00.225088 (+  2927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.255911 (+1030823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.860442 (+604531us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b5400684f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:01.860857 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990367616}
I20260709 13:06:01.990191 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 308) took 2370 ms. Trace:
I20260709 13:06:01.982591 24345 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:01.961143 24080 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.008s	sys 0.064s
W20260709 13:06:01.968719 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.925945 22585 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 540) took 1683 ms (client timeout 1618 ms). Trace:
I20260709 13:06:01.948174 24310 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:01.874950 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 1.00475e+06us
I20260709 13:06:01.889988 22590 rpcz_store.cc:276] 0709 13:06:00.669639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.669846 (+   207us) service_pool.cc:224] Handling call
0709 13:06:00.670256 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.670551 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.670587 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:00.671119 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.423333 (+752214us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:01.423374 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.423438 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.833609 (+410171us) raft_consensus.cc:1685] finished
0709 13:06:01.833663 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.834020 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":635886,"threads_started":1,"wal-append.queue_time_us":729562}
I20260709 13:06:01.981846 22000 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:01.977944 24189 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.052217 21855 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:01.952737 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:01.868861 22004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.010s	sys 0.015s
W20260709 13:06:02.048543 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.026742 21857 rpcz_store.cc:276] 0709 13:05:59.619960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.063400 (+443440us) service_pool.cc:224] Handling call
0709 13:06:00.063682 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.775185 (+1711503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.989792 (+214607us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54000591d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:01.990093 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3702687104}
I20260709 13:06:02.027153 22009 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.4
W20260709 13:06:01.967839 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 928705us
I20260709 13:06:01.961794 24379 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.067610 22009 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:02.071868 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.073897 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.074018 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.075030 22003 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:02.075917 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:01.972878 24357 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.078699 24236 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:02.079651 24236 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.081322 24080 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.082135 24080 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.084463 24258 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.064968 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.094460 22005 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:06:02.070266 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1675) took 2412 ms (client timeout 1694 ms). Trace:
W20260709 13:06:01.992596 21927 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.963s	user 0.007s	sys 0.010s
I20260709 13:06:02.087162 21856 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:01.891592 24289 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:02.101774 21854 rpcz_store.cc:269] 0709 13:05:59.658152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.878583 (+220431us) service_pool.cc:224] Handling call
0709 13:06:01.991045 (+2112462us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540025a154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:02.070087 (+ 79042us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3780809088}
I20260709 13:06:01.964442 24364 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:01.966975 24226 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.132438 22000 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:01.866255 21851 rpcz_store.cc:269] 0709 13:05:59.659417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.996775 (+337358us) service_pool.cc:224] Handling call
0709 13:06:01.748919 (+1752144us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54000fcb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:01.786434 (+ 37515us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448635264}
I20260709 13:06:02.053722 24138 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:02.109679 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:02.028044 22585 rpcz_store.cc:269] 0709 13:06:00.242488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.246794 (+  4306us) service_pool.cc:224] Handling call
0709 13:06:01.425066 (+1178272us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaeee 0000000000375939 00007f3b7571e1e0 00007f3b757e70c4 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:01.925624 (+500558us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10254720}
I20260709 13:06:02.067747 24165 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.130333 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 536) took 1911 ms. Trace:
W20260709 13:06:02.146853 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.145699 22006 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:02.154670 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.159667 24086 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.838s	user 0.012s	sys 0.008s
I20260709 13:06:02.156836 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.163666 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 408) took 1973 ms. Trace:
I20260709 13:06:02.164593 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.162282 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.166357 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.176376 24081 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:02.177055 22586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.031s
W20260709 13:06:02.177529 22586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.000s	sys 0.032s
I20260709 13:06:02.177727 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.178772 22584 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:02.179656 24280 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:02.180500 24280 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.180711 24106 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.181741 24072 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.182370 24106 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:02.182420 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.182946 24280 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:02.183610 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.159684 24132 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.185806 24132 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.186491 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.187759 21859 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:02.183503 24245 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:02.189246 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.188472 24132 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.190135 24357 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:02.192655 24236 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.193477 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.193240 24165 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.194096 22592 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:02.195129 24080 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.195211 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.195537 24165 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.196089 21859 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:02.199160 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:02.202133 24353 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.202910 24326 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:02.203411 24326 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:02.203603 24353 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.207194 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1841) took 1608 ms. Trace:
I20260709 13:06:02.209764 24138 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.211427 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.213371 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.214426 22591 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:02.214471 24236 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:02.215698 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 477) took 3116 ms. Trace:
I20260709 13:06:02.216357 24236 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:02.162715 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.012s	sys 0.020s
I20260709 13:06:02.221166 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.222039 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.222038 21859 rpcz_store.cc:276] 0709 13:06:00.598030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.196863 (+1598833us) service_pool.cc:224] Handling call
0709 13:06:02.201912 (+  5049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.203735 (+  1823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.204089 (+   354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.204394 (+   305us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.204810 (+   416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.205430 (+   620us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.206495 (+  1065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.220736 24236 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:02.140370 24226 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.143940 21927 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:02.146492 22588 rpcz_store.cc:276] 0709 13:06:00.218414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.218810 (+   396us) service_pool.cc:224] Handling call
0709 13:06:00.219132 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.220120 (+   988us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:00.285028 (+ 64908us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.079688 (+1794660us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:02.080631 (+   943us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.080890 (+   259us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:02.080949 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.081121 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.085760 (+  4639us) raft_consensus.cc:1685] finished
0709 13:06:02.085781 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.130268 (+ 44487us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":247721,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":36795,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":168,"thread_start_us":1735784,"threads_started":3,"wal-append.queue_time_us":690469}
W20260709 13:06:02.166242 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.017s	sys 0.008s
W20260709 13:06:02.221400 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.418s	user 0.012s	sys 0.020s
I20260709 13:06:02.225909 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.212771 24267 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:02.240031 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 84511.3us
I20260709 13:06:02.159739 24069 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:06:02.243609 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.021s	sys 0.008s
W20260709 13:06:02.243949 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 78294.4us
I20260709 13:06:02.175364 24256 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.257316 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.259498 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.259610 24220 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.260363 22591 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:02.260444 24220 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.261634 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 611) took 2023 ms. Trace:
I20260709 13:06:02.263597 24220 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.262131 22586 rpcz_store.cc:276] 0709 13:06:00.238773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.250781 (+ 12008us) service_pool.cc:224] Handling call
0709 13:06:00.308370 (+ 57589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.384272 (+ 75902us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:00.956568 (+572296us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.178024 (+1221456us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:02.178759 (+   735us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.179002 (+   243us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.179041 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.179130 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.261322 (+ 82192us) raft_consensus.cc:1685] finished
0709 13:06:02.261344 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.261566 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1185,"prepare.queue_time_us":72299,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":11840,"raft.queue_time_us":6416,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":18704,"thread_start_us":1335194,"threads_started":2,"wal-append.queue_time_us":1216524}
I20260709 13:06:02.164098 21928 rpcz_store.cc:276] 0709 13:06:00.190723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.039832 (+849109us) service_pool.cc:224] Handling call
0709 13:06:01.040201 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.040453 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:01.190740 (+150287us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.069114 (+878374us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.103632 (+ 34518us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.129927 (+ 26295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.140982 (+ 11055us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.163602 (+ 22620us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13776,"prepare.queue_time_us":204406,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":12791,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":35734,"thread_start_us":675986,"threads_started":1,"wal-append.queue_time_us":602258}
I20260709 13:06:02.265324 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:02.266207 24110 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.938s	user 0.004s	sys 0.011s
I20260709 13:06:02.266664 24110 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:02.267540 24250 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.925s	user 0.011s	sys 0.005s
W20260709 13:06:02.269119 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 925572us
I20260709 13:06:02.273103 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.273787 22009 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:02.273931 21860 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:02.274263 24318 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.275311 22000 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:02.269523 21858 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:02.266127 22003 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:02.216147 22004 rpcz_store.cc:276] 0709 13:05:59.099184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.417714 (+1318530us) service_pool.cc:224] Handling call
0709 13:06:00.417983 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:00.418182 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:00.798758 (+380576us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.152589 (+1353831us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.174235 (+ 21646us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:02.215356 (+ 41121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.215481 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.215646 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12912,"prepare.queue_time_us":621220,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":966,"raft.queue_time_us":63920,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":11505,"thread_start_us":1621166,"threads_started":2,"wal-append.queue_time_us":756179}
I20260709 13:06:02.274520 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1842) took 1676 ms. Trace:
I20260709 13:06:02.276829 21860 rpcz_store.cc:276] 0709 13:06:00.598247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.251645 (+1653398us) service_pool.cc:224] Handling call
0709 13:06:02.273522 (+ 21877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.274257 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.274445 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:02.280923 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.258551 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.46731e+06us
I20260709 13:06:02.289366 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:02.225895 24086 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.287235 24080 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.204713 24326 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.174311 24234 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:02.283244 24242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.014s	sys 0.023s
I20260709 13:06:02.278599 24220 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.300889 22002 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:02.306435 24086 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.307749 21932 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [3.5-4.6]
W20260709 13:06:02.309476 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.309661 24072 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:02.312608 21853 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:02.313413 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.314404 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.315764 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.315279 24326 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:02.281549 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1930) took 1650 ms (client timeout 1665 ms). Trace:
W20260709 13:06:02.317413 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 170633us
I20260709 13:06:02.324893 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 315) took 1733 ms. Trace:
I20260709 13:06:02.330669 22005 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:02.326112 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.324805 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.255599 24226 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.320188 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.328928 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.321504 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:02.343771 21856 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:02.330935 21855 rpcz_store.cc:276] 0709 13:06:00.591444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.594510 (+  3066us) service_pool.cc:224] Handling call
0709 13:06:02.051831 (+1457321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.067461 (+ 15630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.322232 (+254771us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540025a150. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:02.324830 (+  2598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3341308800}
W20260709 13:06:02.345182 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.345835 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:02.310364 24242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.682s	user 0.014s	sys 0.024s
I20260709 13:06:02.342828 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.350159 22004 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:02.324687 24072 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.352038 21857 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:02.320983 24283 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:02.357707 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1845) took 1753 ms. Trace:
I20260709 13:06:02.358125 21857 rpcz_store.cc:276] 0709 13:06:00.599028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.284807 (+1685779us) service_pool.cc:224] Handling call
0709 13:06:02.325266 (+ 40459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.352514 (+ 27248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.352706 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.354161 22003 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:06:02.361338 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.362763 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.362936 22592 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:02.363169 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:02.355252 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:02.366897 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87502us
I20260709 13:06:02.367745 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:02.363873 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.363937 21933 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:02.321695 24250 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:06:02.377174 22588 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
W20260709 13:06:02.377475 24385 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:06:02.378223 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.383370 22005 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:06:02.378765 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:02.330678 21927 rpcz_store.cc:269] 0709 13:06:00.608057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.755461 (+147404us) service_pool.cc:224] Handling call
0709 13:06:02.252087 (+1496626us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:02.389050 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 152456us
I20260709 13:06:02.377918 24165 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:02.392194 24384 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.393287 24384 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:02.393698 24384 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:02.394788 24384 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.396144 24384 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.397521 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:02.398563 21926 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:02.399346 22008 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:02.400327 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.400961 22008 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:06:02.402160 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.402280 22008 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:02.403033 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.403559 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.404435 24367 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:02.404687 24229 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.404841 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:02.405355 24229 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.405067 24367 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.406402 24229 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.407145 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.407822 24367 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.409492 24234 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.410745 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.411919 22591 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:02.413627 24081 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.414100 24247 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.414436 24081 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.414731 24247 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.417198 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.417402 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.418088 22590 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:02.416700 24081 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.418248 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.417344 22592 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:02.380342 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.354477 24256 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:02.386533 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 65110us
W20260709 13:06:02.386850 24358 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:06:02.387187 24226 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.387755 21855 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:02.388446 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.413574 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:02.419440 22005 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:02.439217 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:02.370296 24138 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:02.441636 22003 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:02.423202 22002 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:02.359200 24242 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.384415 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:02.445158 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.445130 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:02.413831 24110 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.449512 24280 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:02.450042 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.450947 21855 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:06:02.453306 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46256.4us
I20260709 13:06:02.453442 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:02.454044 24041 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:02.454401 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 402) took 2354 ms. Trace:
I20260709 13:06:02.454934 24041 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.455076 21932 rpcz_store.cc:276] 0709 13:06:00.100343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.815717 (+715374us) service_pool.cc:224] Handling call
0709 13:06:02.283930 (+1468213us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.332423 (+ 48493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.454200 (+121777us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540017e650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:02.454340 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3269446400}
I20260709 13:06:02.454306 21926 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:02.452749 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1848) took 1853 ms. Trace:
W20260709 13:06:02.459602 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 81851.5us
I20260709 13:06:02.459558 21855 rpcz_store.cc:276] 0709 13:06:00.599725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.451600 (+1851875us) service_pool.cc:224] Handling call
0709 13:06:02.451970 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.452371 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.452408 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.452471 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:02.452500 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.452559 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.452676 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4179}
I20260709 13:06:02.454308 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.416714 24247 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.413671 24383 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.490824 24383 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.496161 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.500274 24062 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.502497 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:02.503755 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:02.462868 22005 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 13:06:02.369226 24220 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.463985 24041 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:02.465029 24283 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.466969 24146 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.468494 24138 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.469664 24242 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.471714 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1850) took 1871 ms. Trace:
I20260709 13:06:02.455102 24086 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:02.533597 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.534142 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.437588 22585 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:02.456682 22009 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:02.473462 24226 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:02.473923 24369 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:02.474809 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.476222 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1847) took 1876 ms. Trace:
I20260709 13:06:02.477420 21851 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:06:02.543635 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.539850 21858 rpcz_store.cc:276] 0709 13:06:00.599499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.021818 (+422319us) service_pool.cc:224] Handling call
0709 13:06:02.475928 (+1454110us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002fbcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:02.476162 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866085760}
I20260709 13:06:02.477870 21857 raft_consensus.cc:2376] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4. Re-sending same reply.
I20260709 13:06:02.479645 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:02.481896 24386 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:06:02.483888 24165 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:02.484194 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1676) took 2825 ms (client timeout 2117 ms). Trace:
I20260709 13:06:02.485052 21852 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:02.485247 24289 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:02.485982 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.561066 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.489748 24234 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:02.560586 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.539279 24226 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.504896 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:02.509487 22587 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:02.523952 21856 rpcz_store.cc:276] 0709 13:06:00.600281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.470696 (+1870415us) service_pool.cc:224] Handling call
0709 13:06:02.471091 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.471402 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.471428 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.471486 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.471515 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.471566 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.471662 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.550354 21855 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:02.553146 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:02.553586 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 105702us
I20260709 13:06:02.571540 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1849) took 1971 ms. Trace:
I20260709 13:06:02.459462 24256 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.559801 24166 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.489194 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75734.6us
I20260709 13:06:02.538043 24383 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.572109 21855 rpcz_store.cc:276] 0709 13:06:00.600053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.506074 (+1906021us) service_pool.cc:224] Handling call
0709 13:06:02.546391 (+ 40317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.571213 (+ 24822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.571469 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:02.554752 24387 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:06:02.567934 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:02.610478 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:02.585531 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.552207 24086 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:02.612388 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.613394 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.613540 24086 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.615531 24364 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:06:02.556614 21853 rpcz_store.cc:269] 0709 13:05:59.658549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.950718 (+292169us) service_pool.cc:224] Handling call
0709 13:06:02.483930 (+2533212us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b540008b1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:02.484119 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5420617600}
I20260709 13:06:02.617697 24234 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:02.617921 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.618690 22589 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:02.618817 24234 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.620754 24369 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.621253 24369 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.621605 24369 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.622458 24072 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:02.625782 24393 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:06:02.588835 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:02.590045 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:02.590583 24242 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.590708 24384 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:02.597823 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:02.598493 21758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 286) took 18828 ms (client timeout 15000 ms). Trace:
I20260709 13:06:02.598912 24138 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.599407 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:02.605607 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 119870us
I20260709 13:06:02.605361 24229 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.606202 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:02.607374 24220 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:02.608004 24256 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.567622 24283 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.561718 24310 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.628551 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 74269.1us
W20260709 13:06:02.598560 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.550016 22584 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:02.649796 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:02.651055 24245 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.653108 24331 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:02.608533 24166 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:02.654466 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.655709 24242 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:02.697376 24242 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.662170 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:06:02.644510 21758 rpcz_store.cc:269] 0709 13:05:43.770366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.781412 (+ 11046us) service_pool.cc:224] Handling call
0709 13:05:44.308205 (+526793us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:02.598173 (+18289968us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:57.742124 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:57.917691 (+175567us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:57.917735 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:05:57.917769 (+    34us) tablet.cc:662] Decoding operations
0709 13:05:57.918444 (+   675us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:57.918513 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:57.918549 (+    36us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:05:57.984915 (+ 66366us) tablet.cc:701] Row locks acquired
0709 13:05:57.992922 (+  8007us) write_op.cc:260] PREPARE: finished
0709 13:05:58.012104 (+ 19182us) write_op.cc:270] Start()
0709 13:05:58.012231 (+   127us) write_op.cc:276] Timestamp: P: 1783602358012042 usec, L: 0
0709 13:05:58.012256 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:05:58.047125 (+ 34869us) log.cc:844] Serialized 1307 byte log entry
0709 13:06:00.719678 (+2672553us) op_driver.cc:464] REPLICATION: finished
0709 13:06:01.548351 (+828673us) write_op.cc:301] APPLY: starting
0709 13:06:01.551296 (+  2945us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:01.556469 (+  5173us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:01.856027 (+299558us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:01.878174 (+ 22147us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:01.878621 (+   447us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:06:01.878722 (+   101us) write_op.cc:312] APPLY: finished
0709 13:06:01.879294 (+   572us) log.cc:844] Serialized 35 byte log entry
0709 13:06:02.174721 (+295427us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:02.205278 (+ 30557us) write_op.cc:454] Released schema lock
0709 13:06:02.205384 (+   106us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:02.206142 (+   758us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:02.206174 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":182737,"reported_tablets":12,"spinlock_wait_cycles":93897984,"child_traces":[["op",{"apply.queue_time_us":819088,"apply.run_cpu_time_us":28397,"apply.run_wall_time_us":660739,"num_ops":3,"prepare.queue_time_us":462306,"prepare.run_cpu_time_us":65156,"prepare.run_wall_time_us":1416550,"raft.queue_time_us":792397,"raft.run_cpu_time_us":59689,"raft.run_wall_time_us":874290,"replication_time_us":2656345,"thread_start_us":2436324,"threads_started":5,"wal-append.queue_time_us":1723495,"wal-append.run_cpu_time_us":50153,"wal-append.run_wall_time_us":1595697}]]}
I20260709 13:06:02.700302 24242 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:02.645171 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72700.1us
I20260709 13:06:02.575866 24226 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:02.663945 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51678.2us
I20260709 13:06:02.673494 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:02.676476 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:02.658879 24072 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:02.671152 24364 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:06:02.694119 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 105399us
I20260709 13:06:02.707530 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:02.683635 24383 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:06:02.695933 24310 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.697939 24166 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.651062 24165 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:02.718355 24165 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:02.718483 24229 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.718931 24165 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:02.720582 24283 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.705515 22009 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:02.706439 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:02.712679 24173 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.692685 24245 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.716732 24220 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:02.724295 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.714830 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:02.691607 24331 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:02.734162 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 271057us
I20260709 13:06:02.702267 24110 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:02.733253 24345 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:06:02.741659 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:02.745864 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.737465 24220 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:02.749104 24220 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:02.749208 24262 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.755789 24280 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:02.756585 24081 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:02.759466 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.767232 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.768203 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:02.747650 21926 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:02.775918 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 225108us
I20260709 13:06:02.747062 24041 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.740816 24364 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:02.756994 24081 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.784543 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:02.791020 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 58600us
I20260709 13:06:02.789310 22009 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
W20260709 13:06:02.780129 21972 connection.cc:256] RPC call timeout handler was delayed by -1.7527315560000716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 13:06:02.792274 24081 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:02.798017 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.799419 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.746858 24072 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:06:02.776806 24245 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.778141 24166 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.740679 24173 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.828918 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.843899 22593 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:06:02.845095 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:02.808225 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.829154 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:02.801716 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:02.811455 22590 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:02.783114 24041 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:02.803345 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1525668630000609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
I20260709 13:06:02.782153 24289 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:02.821350 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.821806 24080 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:02.822503 22589 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:06:02.825529 24364 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.827549 24081 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:02.786451 24110 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:06:02.807512 24354 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:02.865967 22587 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:02.857611 24041 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.868330 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:02.867467 22589 pending_rounds.cc:85] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 3
I20260709 13:06:02.838546 24072 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:02.849352 24229 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:02.866248 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 177066us
I20260709 13:06:02.837142 24138 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:06:02.866359 24220 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.840400 24331 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.882293 24310 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:02.885685 24353 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.003s
I20260709 13:06:02.883279 24110 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:06:02.883337 24398 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:06:02.898290 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:02.887996 24354 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:02.884279 22589 pending_rounds.cc:107] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 13:06:02.908476 24081 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:02.884017 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 75753.2us
I20260709 13:06:02.885211 24166 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:02.889154 24229 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.924288 24229 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:02.922442 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:02.930312 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.930511 24402 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.931006 24402 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.931361 24402 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.931609 24310 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.933930 24310 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.936126 24310 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:02.920245 22589 raft_consensus.cc:2917] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:06:02.937654 22589 raft_consensus.cc:2889] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:02.938341 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:02.940271 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:02.940856 24138 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.940920 24220 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:02.886766 24072 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:02.946157 21854 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:02.903242 24379 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.905730 24283 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:02.902698 24234 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:02.912657 24256 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.889755 24080 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:06:02.961434 21900 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:02.955792 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:02.917692 24110 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:02.901796 24165 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.971891 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:02.967434 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 69207.5us
I20260709 13:06:02.965348 24081 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.975961 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:02.906216 24384 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:06:02.948422 24138 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:02.956919 24166 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:02.957482 24310 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:02.988001 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:02.957674 24283 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:03.000051 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:03.029660 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 152655us
I20260709 13:06:02.987133 24138 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.995450 24166 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:02.998484 24146 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:03.049947 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 262052us
I20260709 13:06:03.053267 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:03.054170 21860 raft_consensus.cc:2376] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5. Re-sending same reply.
I20260709 13:06:03.031386 24367 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:02.969290 24267 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:06:02.985818 24369 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:02.982429 24379 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.047850 24081 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:03.051045 24166 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:02.999704 24256 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:02.987792 24234 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:03.075224 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 90688.5us
I20260709 13:06:03.059369 24367 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:03.083427 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:03.057472 24220 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:03.061743 24146 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:03.087301 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 147125us
I20260709 13:06:03.075312 24081 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:03.091549 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 108875us
I20260709 13:06:03.105463 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.106256 21853 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:06:03.107679 24403 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:03.108180 24403 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.108517 24403 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.111223 24238 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:06:03.114552 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1853) took 2294 ms. Trace:
I20260709 13:06:03.114975 21858 rpcz_store.cc:276] 0709 13:06:00.819941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.556793 (+1736852us) service_pool.cc:224] Handling call
0709 13:06:02.650667 (+ 93874us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.683651 (+ 32984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.683685 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:02.695664 (+ 11979us) log.cc:844] Serialized 52 byte log entry
0709 13:06:02.826954 (+131290us) spinlock_profiling.cc:243] Waited 7.94 ms on lock 0x7b1800490540. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:06:03.014549 (+187595us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:03.044315 (+ 29766us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.044535 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.044606 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.114234 (+ 69628us) raft_consensus.cc:1685] finished
0709 13:06:03.114326 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.114474 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15870,"prepare.queue_time_us":33490,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":25883,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":12981,"spinlock_wait_cycles":18264320,"thread_start_us":187321,"threads_started":1,"wal-append.queue_time_us":403198}
I20260709 13:06:03.118273 24379 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:03.118932 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.119868 21855 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:03.096836 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:03.051654 24289 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.092110 24220 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:03.101147 24405 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.160s	user 0.000s	sys 0.005s
W20260709 13:06:03.135650 24408 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:06:03.143812 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:03.100014 24081 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.095046 24283 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.061841 24110 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.150463 21855 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:03.147811 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89217.1us
I20260709 13:06:03.170087 22591 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:03.124184 24080 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:03.190063 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49230.9us
I20260709 13:06:03.197666 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.079413 24354 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.146597 24234 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:03.195127 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.198107 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.207690 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:03.217383 22009 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:03.217655 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.213791 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:03.217391 22004 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.216790 22001 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:03.219655 22007 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:03.222219 22005 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:06:03.226567 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 143277us
I20260709 13:06:03.229176 22008 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:03.167138 24379 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:03.148525 24041 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.238130 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:03.241683 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57324.3us
I20260709 13:06:03.218629 24310 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:03.192318 24226 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.251359 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:03.221385 24165 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.256096 24229 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.233690 24110 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.262636 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:03.256006 24310 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.281352 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:03.260460 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.282182 24415 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.282599 21857 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:03.283061 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:03.283565 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.199491 24080 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.286629 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:03.287798 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.289129 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:03.289726 22005 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.291608 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:06:03.292548 21928 raft_consensus.cc:606] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:03.267953 24226 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.292932 21928 raft_consensus.cc:614] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:03.293416 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.294155 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.268146 24229 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.212303 24367 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.268597 24238 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.273437 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.275049 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:03.277141 24424 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.278856 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:03.284895 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:03.306937 22005 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:03.307806 22008 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.280270 24428 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:03.284886 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.299558 21855 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:03.249799 24283 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.305666 21858 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:03.313730 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 116562us
W20260709 13:06:03.321717 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 73928.4us
I20260709 13:06:03.326370 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:03.286731 24080 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:03.329304 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.310586 24238 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.296192 24146 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.331686 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.326352 24234 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:03.347230 21823 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.992s (SENT)
W20260709 13:06:03.353940 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.354595 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.358021 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.363776 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:03.310753 24428 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.370599 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:03.367374 24256 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:03.374457 24256 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:03.374979 24256 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.375468 24256 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.379459 21928 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:03.363848 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:03.383852 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 102623us
I20260709 13:06:03.384238 21935 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.384832 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:03.386104 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:03.372525 21856 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:03.325551 24165 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:06:03.355633 24369 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.313164 24220 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.371893 21853 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:06:03.404532 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 74238.8us
I20260709 13:06:03.371055 24234 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.401871 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:03.406308 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 97319.7us
I20260709 13:06:03.378393 24326 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:03.352973 24229 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:06:03.391610 24245 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.000s
I20260709 13:06:03.371997 24428 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.374730 24238 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.410467 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:03.366896 24418 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:03.416013 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:03.420374 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.421579 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.422950 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.332630 24424 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.426117 21933 raft_consensus.cc:2368] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
W20260709 13:06:03.431345 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 99653.5us
I20260709 13:06:03.415691 24245 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.435099 24245 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:03.436178 24081 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.413259 24234 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:03.412957 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 317) took 2813 ms. Trace:
I20260709 13:06:03.451877 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:03.454077 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.447312 21935 raft_consensus.cc:2410] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:03.417501 24110 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:06:03.461114 21931 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
W20260709 13:06:03.462821 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.462616 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:03.440109 24415 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:03.464217 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.466827 21933 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.469905 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.441886 24080 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.470911 21935 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.463466 24439 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:03.450883 24238 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:03.473428 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:03.493503 24414 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:06:03.460142 24041 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.470021 24418 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:03.540876 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:03.517552 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.441022 24423 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.459637 24220 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.594866 22008 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:03.596031 21928 raft_consensus.cc:2410] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:03.463399 24342 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.162s
I20260709 13:06:03.455530 24326 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:03.541363 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1861) took 1463 ms (client timeout 1915 ms). Trace:
W20260709 13:06:03.596930 21854 rpcz_store.cc:269] 0709 13:06:02.078477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.149784 (+ 71307us) service_pool.cc:224] Handling call
0709 13:06:03.522775 (+1372991us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b5400017f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:03.541291 (+ 18516us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610379648}
I20260709 13:06:03.513988 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:03.597884 22004 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:03.495685 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.598129 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:03.598851 22001 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:03.450506 24369 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:03.505551 21931 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:03.575654 21933 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:03.581771 22590 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:03.583241 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:03.584823 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.573513 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:03.586678 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:03.517405 24416 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:06:03.598619 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 141912us
I20260709 13:06:03.583465 24430 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:03.588330 24081 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:03.602193 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:03.606563 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109532us
I20260709 13:06:03.600675 21860 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:03.630010 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.588497 24238 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:03.632709 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.514191 24427 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:03.637661 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:03.512454 24429 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:03.512259 24421 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.527943 24433 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.638445 22593 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.608127 24132 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.640581 24445 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.640769 24041 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:03.641111 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.658128 21931 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:03.641589 24431 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:06:03.642158 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.640650 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:03.643350 24430 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:03.611302 24439 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:03.609447 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:03.449251 21851 rpcz_store.cc:276] 0709 13:06:00.596814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.190614 (+1593800us) service_pool.cc:224] Handling call
0709 13:06:02.477146 (+286532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.547195 (+ 70049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.395915 (+848720us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002fbcd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:03.409649 (+ 13734us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657679104}
I20260709 13:06:03.644079 21857 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:03.571870 24418 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.639731 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:03.607939 24369 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:03.670614 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 68659.8us
I20260709 13:06:03.590019 24425 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.617859 21935 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:03.658908 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.672642 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.505805 24080 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.683656 21860 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:06:03.666819 24138 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.004s
I20260709 13:06:03.671936 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:03.648190 24429 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.647125 24404 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:03.681740 21859 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:03.669497 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60166.5us
W20260709 13:06:03.684268 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44696.6us
I20260709 13:06:03.691466 21926 raft_consensus.cc:2368] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
W20260709 13:06:03.668237 24262 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.011s	sys 0.006s
I20260709 13:06:03.696969 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.667268 24439 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.696542 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:03.670641 24132 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.664340 24445 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.706187 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1692) took 1073 ms. Trace:
I20260709 13:06:03.678011 24426 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:03.678764 24418 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.750545 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:03.754149 21858 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.759630 21856 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:03.706697 21859 rpcz_store.cc:276] 0709 13:06:02.628108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.661110 (+1033002us) service_pool.cc:224] Handling call
0709 13:06:03.681532 (+ 20422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.701274 (+ 19742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.701506 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.707868 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:03.688860 24138 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.699906 21857 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:03.711090 24328 raft_consensus.cc:2764] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:03.712064 24132 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.713019 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:03.713361 24041 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.713954 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:03.701224 24262 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.011s	sys 0.006s
W20260709 13:06:03.715178 24403 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.002s
I20260709 13:06:03.695909 24429 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:03.716992 24402 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.785s	user 0.013s	sys 0.002s
I20260709 13:06:03.719643 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.721377 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.810662 22008 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:03.722200 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.811398 22004 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:03.721396 24430 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.720578 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.807132 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 663) took 1220 ms. Trace:
I20260709 13:06:03.838752 22009 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:03.727443 24353 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:03.738909 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.740401 22593 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:03.743364 24400 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.906s	user 0.003s	sys 0.000s
W20260709 13:06:03.745690 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:03.752826 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.678471 24369 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.761325 24310 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:03.764463 24445 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:03.772791 24256 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:03.769627 24422 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.773451 24086 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:03.772622 24437 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:06:03.774147 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:03.778443 24404 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:03.671243 24444 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:03.760686 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 310) took 3170 ms. Trace:
I20260709 13:06:03.787793 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:03.793330 21858 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:03.798414 21926 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:03.702134 24439 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:03.798180 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:03.710094 24426 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:03.838785 22591 rpcz_store.cc:276] 0709 13:06:02.586664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.616602 (+ 29938us) service_pool.cc:224] Handling call
0709 13:06:03.151854 (+535252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.173106 (+ 21252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.806855 (+633749us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400530cd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:03.807053 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1230207232}
I20260709 13:06:03.785439 24173 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:03.848724 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.765133 24326 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:03.694432 24425 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:03.850051 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 137143us
W20260709 13:06:03.972815 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.153159 21935 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:04.154023 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.154974 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:04.155249 22008 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:04.157225 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:04.157239 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.160457 22004 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:03.808687 24165 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.177651 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.198565 21858 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:03.849767 24415 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:04.201429 24415 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:04.203214 24415 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:04.201069 21852 rpcz_store.cc:276] 0709 13:06:00.590114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.590272 (+   158us) service_pool.cc:224] Handling call
0709 13:06:02.484738 (+1894466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.585092 (+100354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.760349 (+1175257us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b540008b1d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:03.760582 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4345311616}
W20260709 13:06:03.898672 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.904068 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:03.861649 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 500) took 1556 ms. Trace:
W20260709 13:06:03.857772 24166 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.765s	user 0.011s	sys 0.006s
W20260709 13:06:04.365312 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.365559 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.368222 21933 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:04.368156 22008 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.365789 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:04.174702 24422 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:04.367378 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.024s	sys 0.096s
W20260709 13:06:04.102495 24283 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.002s
W20260709 13:06:04.380594 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.024s	sys 0.096s
W20260709 13:06:03.911931 24398 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:06:04.385067 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 713147us
I20260709 13:06:04.388619 22004 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:04.391388 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:04.381279 22001 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:03.804381 24041 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.392400 21858 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:06:04.392789 21858 pending_rounds.cc:85] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 1
I20260709 13:06:04.393054 21858 pending_rounds.cc:107] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:06:04.393290 21858 raft_consensus.cc:2889] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:04.394580 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:04.394759 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.174746 24267 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.404321 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:04.397289 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:04.405486 22004 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:04.352095 24404 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:04.409299 21854 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:04.407296 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:04.193310 24424 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.368222 22003 rpcz_store.cc:276] 0709 13:06:02.309448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.446770 (+137322us) service_pool.cc:224] Handling call
0709 13:06:02.482941 (+ 36171us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.483423 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.483451 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:02.484127 (+   676us) log.cc:844] Serialized 52 byte log entry
0709 13:06:02.670617 (+186490us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.670659 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.670761 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.742470 (+1071709us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:03.861293 (+118823us) raft_consensus.cc:1685] finished
0709 13:06:03.861322 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.861584 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2616,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"spinlock_wait_cycles":69353856,"thread_start_us":186027,"threads_started":1,"wal-append.queue_time_us":305974}
I20260709 13:06:03.838291 24403 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.357188 24426 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:04.417088 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.418812 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:04.422374 22004 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:03.850736 24234 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.381850 24250 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:03.843971 24402 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.425263 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.423880 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.432621 22007 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:04.430928 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:04.420393 24283 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.434068 22001 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.431946 22009 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.435068 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:04.434551 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:04.437585 21931 raft_consensus.cc:2376] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5. Re-sending same reply.
W20260709 13:06:04.417758 24262 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.040s	sys 0.097s
I20260709 13:06:04.150136 24072 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.448091 22007 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:06:03.859872 24418 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.415000 24444 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.173621 24086 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.410943 24267 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.910730 24310 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.509280 24086 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.509758 24310 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.449123 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.431974 24245 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.425855 24364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.545s	user 0.012s	sys 0.030s
I20260709 13:06:04.505759 22007 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:04.513285 21932 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:04.382586 24166 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:04.453480 24262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.040s	sys 0.097s
I20260709 13:06:04.522053 22003 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:04.524319 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.536736 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.537221 21931 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:04.436058 24080 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.447517 24283 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:04.525281 24364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.645s	user 0.013s	sys 0.031s
I20260709 13:06:04.537391 22007 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:06:04.546846 22004 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.548381 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.539811 21935 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:04.512351 24267 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:04.524940 24456 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:06:04.565433 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.566197 21931 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:04.567106 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:04.567564 21931 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:04.525098 24138 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.549916 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:04.450201 24353 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:06:04.497886 24250 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:04.574285 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 734118us
I20260709 13:06:04.543306 24457 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.544727 24283 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.583809 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:04.584851 21935 raft_consensus.cc:2368] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5.
I20260709 13:06:04.539333 24245 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.597810 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.599112 21935 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:04.600492 21931 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:04.586189 24457 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:04.605363 24448 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.410s	user 0.001s	sys 0.000s
W20260709 13:06:04.606980 24452 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:06:04.611061 24369 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.760s	user 0.008s	sys 0.000s
I20260709 13:06:04.611572 24369 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.613138 21928 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:04.578868 24310 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.614934 24310 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:04.627929 24433 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.958s	user 0.007s	sys 0.001s
I20260709 13:06:04.628393 24433 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:04.629791 24250 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.630170 24422 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.629812 21859 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:06:04.630975 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1874) took 1359 ms. Trace:
I20260709 13:06:04.631376 21859 rpcz_store.cc:276] 0709 13:06:03.271860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.790086 (+518226us) service_pool.cc:224] Handling call
0709 13:06:04.629380 (+839294us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:06:04.629572 (+   192us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:04.630709 (+  1137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.630920 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46082816}
I20260709 13:06:04.583524 24138 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:04.637274 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:04.630961 24428 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.174s	user 0.011s	sys 0.000s
I20260709 13:06:04.652511 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.635735 24425 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.662865 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.668731 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:04.672645 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.651894 24234 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.674346 21931 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:04.675117 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:04.657608 24256 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.677814 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:04.662446 24331 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:04.660888 24310 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.680431 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:04.682960 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:04.668058 24041 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.690876 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:04.696911 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 544345us
W20260709 13:06:04.692797 24459 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:06:04.693740 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1924) took 1120 ms. Trace:
I20260709 13:06:04.694590 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1921) took 1231 ms. Trace:
I20260709 13:06:04.676060 24425 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:04.691732 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:04.657845 24138 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.697041 24310 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.653916 24428 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.698964 24403 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.712821 22006 rpcz_store.cc:276] 0709 13:06:03.462765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.469094 (+  6329us) service_pool.cc:224] Handling call
0709 13:06:03.538892 (+ 69798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.539070 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:03.801422 (+262352us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.694242 (+892820us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.694296 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:04.694344 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.694426 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.694532 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":892374,"prepare.queue_time_us":989,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":895019,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":261991,"threads_started":1}
I20260709 13:06:04.718535 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:04.718741 24138 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.719077 24247 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:04.719108 22006 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:04.719656 24247 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:04.720117 24247 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:04.697423 24440 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:06:04.719322 21935 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 2.
I20260709 13:06:04.699936 24354 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.700029 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.700070 24245 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.706086 24418 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.695245 24086 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:06:04.709569 24369 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.695565 24234 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.709954 24166 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.711706 24267 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:04.714394 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.713168 22002 rpcz_store.cc:276] 0709 13:06:03.573294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.573438 (+   144us) service_pool.cc:224] Handling call
0709 13:06:03.573752 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.606425 (+ 32673us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:03.606470 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.607050 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.143590 (+536540us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b180032ffe0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:06:04.419944 (+276354us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:04.419998 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.420076 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.693387 (+273311us) raft_consensus.cc:1685] finished
0709 13:06:04.693419 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.693672 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":702,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":21896,"spinlock_wait_cycles":2423552,"thread_start_us":276117,"threads_started":1,"wal-append.queue_time_us":1031116}
I20260709 13:06:04.715116 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:04.686241 24383 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:04.682788 24364 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:04.698535 24173 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.739531 24041 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.720675 21853 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:04.723286 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:04.728359 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.730310 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.716182 24256 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 13:06:04.715622 24165 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.515s	user 0.002s	sys 0.008s
I20260709 13:06:04.740142 22007 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [4.5-5.6]
W20260709 13:06:04.780097 24429 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.928s	user 0.005s	sys 0.005s
I20260709 13:06:04.748533 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:04.767263 24331 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.716032 24310 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.716742 21931 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.746446 24354 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.688895 24402 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.749128 24234 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:04.757517 24423 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.145s	user 0.010s	sys 0.002s
I20260709 13:06:04.781927 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:04.785367 22002 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:04.750372 24425 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:04.741878 22001 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:04.823907 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1909) took 1091 ms. Trace:
I20260709 13:06:04.795202 21855 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:04.825258 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1878) took 1527 ms (client timeout 1782 ms). Trace:
W20260709 13:06:04.825657 21855 rpcz_store.cc:269] 0709 13:06:03.297832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.327802 (+ 29970us) service_pool.cc:224] Handling call
0709 13:06:04.748385 (+1420583us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:04.825197 (+ 76812us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11540736}
I20260709 13:06:04.711997 24426 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.747565 24353 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:06:04.778465 24449 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.751698 24280 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:06:04.783291 24173 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.800472 24331 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:04.801410 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 103770us
I20260709 13:06:04.819707 24423 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:04.827597 24106 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.782688 24383 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:04.757309 24418 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:06:04.789906 24080 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:04.805802 24165 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.839614 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:04.840637 22007 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:04.824540 22001 rpcz_store.cc:276] 0709 13:06:03.732210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.404110 (+671900us) service_pool.cc:224] Handling call
0709 13:06:04.823836 (+419726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.817974 24234 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.810057 24318 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:04.841280 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.810370 24310 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.883661 21934 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:04.851114 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:04.838160 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 77580.7us
I20260709 13:06:04.736243 24404 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.791220 21927 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:04.859704 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.788765 24041 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:04.861678 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.864769 24220 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.258s	user 0.005s	sys 0.009s
I20260709 13:06:04.865490 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:04.853585 24364 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:04.866930 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:04.880211 24455 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.877486 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:04.803755 24429 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.916859 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.882740 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:04.889981 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 67930.5us
W20260709 13:06:04.887233 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49485us
I20260709 13:06:04.829144 24439 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.869148 24106 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:04.901176 22005 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:04.867298 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:04.951220 22587 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:04.903784 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.953069 21931 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.954108 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:04.954516 24439 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.955235 21926 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:04.907023 22591 raft_consensus.cc:2368] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6.
I20260709 13:06:04.952652 24318 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.956732 24367 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:04.909868 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:04.911985 22008 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 13:06:04.895159 24292 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.934s	user 0.009s	sys 0.015s
I20260709 13:06:04.918700 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:04.963320 24292 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.132s	user 0.010s	sys 0.016s
W20260709 13:06:04.963594 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44985.6us
I20260709 13:06:04.961585 22008 pending_rounds.cc:85] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 3
I20260709 13:06:04.964522 22008 pending_rounds.cc:107] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:06:04.922674 24173 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.923631 24364 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.966362 24474 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:04.968008 24474 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:04.926508 24404 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:04.929514 24466 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:06:04.972361 24474 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.974627 24383 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:04.914407 24471 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.977901 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 115298us
I20260709 13:06:04.930294 24331 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.903100 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:04.931898 24429 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.897356 24086 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:06:04.935232 22009 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.916081 24455 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.904085 24423 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.940763 24267 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.941267 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1930) took 1165 ms. Trace:
I20260709 13:06:04.942402 24428 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:04.945704 22588 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:04.964766 22008 raft_consensus.cc:2889] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:04.954721 21931 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:04.877787 24425 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.855082 24449 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.977486 24471 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:04.983337 24041 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:04.888072 24080 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:04.960858 24106 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.057303 21972 leader_election.cc:400] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
I20260709 13:06:05.057634 21972 leader_election.cc:403] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:06:05.058364 24475 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
I20260709 13:06:05.000516 24444 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:05.059908 24444 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:05.060985 24444 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.062917 24471 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.009390 24423 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.006225 22004 rpcz_store.cc:276] 0709 13:06:03.775519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.548550 (+773031us) service_pool.cc:224] Handling call
0709 13:06:04.548904 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.700540 (+151636us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:04.813143 (+112603us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.890814 (+ 77671us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.891434 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.891620 (+   186us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:04.891668 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.891762 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.937800 (+ 46038us) raft_consensus.cc:1685] finished
0709 13:06:04.937822 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.941203 (+  3381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181287,"prepare.queue_time_us":67790,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":190432,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":28337,"thread_start_us":109306,"threads_started":2,"wal-append.queue_time_us":77205}
I20260709 13:06:05.069906 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:04.987039 24415 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.590s	user 0.003s	sys 0.009s
I20260709 13:06:05.070688 21851 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.070950 24041 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:05.071645 24041 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.073604 24262 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:05.074375 24262 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:05.070969 24415 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.054594 24234 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.068128 24423 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.988119 24426 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:05.045974 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.055294 24455 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:05.127032 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.131062 21931 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:05.045871 24354 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.132658 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1943) took 1389 ms. Trace:
I20260709 13:06:05.133198 21931 rpcz_store.cc:276] 0709 13:06:03.743480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.053512 (+1310032us) service_pool.cc:224] Handling call
0709 13:06:05.132593 (+ 79081us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:05.117660 24421 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.444s	user 0.011s	sys 0.003s
I20260709 13:06:05.108906 24267 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:05.135401 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1928) took 1360 ms. Trace:
W20260709 13:06:05.141191 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162193us
I20260709 13:06:05.127853 24328 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.148736 24234 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.136772 22003 rpcz_store.cc:276] 0709 13:06:03.774952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.542403 (+767451us) service_pool.cc:224] Handling call
0709 13:06:04.561329 (+ 18926us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:04.717923 (+156594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.717955 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:04.718570 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 13:06:04.844857 (+126287us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:04.865511 (+ 20654us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.990076 (+124565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.990161 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.990204 (+    43us) raft_consensus.cc:1685] finished
0709 13:06:04.990217 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.135322 (+145105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154204,"prepare.queue_time_us":10493,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":193962,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":7186,"thread_start_us":239951,"threads_started":2,"wal-append.queue_time_us":129254}
I20260709 13:06:05.124437 24310 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.141752 24421 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:05.139714 24449 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.140004 24444 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:05.152931 21851 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:05.162884 24310 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.163717 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1873) took 1892 ms. Trace:
I20260709 13:06:05.166311 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:05.143195 24426 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.173892 24426 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.176008 24426 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.168848 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.154773 24415 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.171053 21854 rpcz_store.cc:276] 0709 13:06:03.271632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.678305 (+406673us) service_pool.cc:224] Handling call
0709 13:06:05.163548 (+1485243us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b540010cf54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.163657 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636512512}
I20260709 13:06:05.157474 24041 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.142905 24267 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:05.171280 24283 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 13:06:05.209868 21931 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:05.181015 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1877) took 1883 ms. Trace:
I20260709 13:06:05.213323 21851 rpcz_store.cc:276] 0709 13:06:03.297615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.704703 (+407088us) service_pool.cc:224] Handling call
0709 13:06:05.069766 (+1365063us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaeee 0000000000375939 00007f3b7571e1e0 00007f3b757e70c4 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:05.180928 (+111162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37329792}
W20260709 13:06:05.214584 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.213505 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:05.216495 24310 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:05.209143 24447 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.103s	user 0.001s	sys 0.004s
I20260709 13:06:05.158737 24328 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.158284 24262 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.329830 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:05.320536 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.330101 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1870) took 2059 ms (client timeout 1851 ms). Trace:
W20260709 13:06:05.282773 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.339998 21853 rpcz_store.cc:269] 0709 13:06:03.270867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.517258 (+246391us) service_pool.cc:224] Handling call
0709 13:06:05.329864 (+1812606us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540025ab54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.330016 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638388480}
W20260709 13:06:05.330703 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1949) took 1585 ms (client timeout 1994 ms). Trace:
W20260709 13:06:05.328815 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 413) took 2383 ms (client timeout 2891 ms). Trace:
I20260709 13:06:05.321241 21852 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:05.341308 21932 rpcz_store.cc:269] 0709 13:06:02.945430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.955470 (+ 10040us) service_pool.cc:224] Handling call
0709 13:06:05.328500 (+2373030us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.328706 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3534591744}
W20260709 13:06:05.340898 21926 rpcz_store.cc:269] 0709 13:06:03.745431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.524134 (+778703us) service_pool.cc:224] Handling call
0709 13:06:05.330467 (+806333us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54003075d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.330625 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933901056}
I20260709 13:06:05.342841 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:05.344241 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.344864 21926 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
W20260709 13:06:05.318328 24458 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.763s	user 0.002s	sys 0.000s
I20260709 13:06:05.348153 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.350813 24267 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.351122 21926 raft_consensus.cc:2410] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:06:05.349365 24426 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.351639 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1641) took 1841 ms. Trace:
I20260709 13:06:05.352025 21926 rpcz_store.cc:276] 0709 13:06:03.510452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.347906 (+1837454us) service_pool.cc:224] Handling call
0709 13:06:05.351580 (+  3674us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.351944 24426 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.354372 24426 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.323798 24080 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.326288 24166 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:05.356252 24166 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:05.356307 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 195943us
I20260709 13:06:05.357278 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.263760 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.362286 24262 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:05.363680 24072 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.365190 21931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:06:05.366070 21931 raft_consensus.cc:606] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.366734 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48000 (request call id 19) took 1694 ms. Trace:
I20260709 13:06:05.366828 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 420) took 1983 ms. Trace:
I20260709 13:06:05.367187 21931 rpcz_store.cc:276] 0709 13:06:03.672233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.361164 (+1688931us) service_pool.cc:224] Handling call
0709 13:06:05.366632 (+  5468us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.367358 21934 rpcz_store.cc:276] 0709 13:06:03.384532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.474392 (+ 89860us) service_pool.cc:224] Handling call
0709 13:06:03.592118 (+117726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.884300 (+1292182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.366542 (+482242us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400207954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:05.366758 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969512832}
I20260709 13:06:05.216661 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:05.371078 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:05.371304 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1640) took 1900 ms (client timeout 2060 ms). Trace:
W20260709 13:06:05.371810 21928 rpcz_store.cc:269] 0709 13:06:03.470848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.629679 (+158831us) service_pool.cc:224] Handling call
0709 13:06:05.371101 (+1741422us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54004295d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.371241 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2256470784}
W20260709 13:06:05.373283 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1642) took 1836 ms (client timeout 2269 ms). Trace:
W20260709 13:06:05.373921 21933 rpcz_store.cc:269] 0709 13:06:03.536823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.641892 (+105069us) service_pool.cc:224] Handling call
0709 13:06:05.373018 (+1731126us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b5400429fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.373205 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625513856}
I20260709 13:06:05.327668 24430 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:06:05.370442 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 153841us
W20260709 13:06:05.378259 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.378260 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:05.342518 24310 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.379738 21932 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:05.380450 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1947) took 1635 ms. Trace:
I20260709 13:06:05.382534 21926 rpcz_store.cc:276] 0709 13:06:03.744758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.352900 (+1608142us) service_pool.cc:224] Handling call
0709 13:06:05.380350 (+ 27450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.383507 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1953) took 1605 ms. Trace:
I20260709 13:06:05.215041 24318 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:06:05.371150 24247 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.003s
I20260709 13:06:05.384835 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.385408 21932 rpcz_store.cc:276] 0709 13:06:03.778469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.361284 (+1582815us) service_pool.cc:224] Handling call
0709 13:06:05.383445 (+ 22161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.375633 24426 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:05.396970 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:05.375850 24477 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:05.264205 24106 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:05.400918 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 263120us
I20260709 13:06:05.401113 21928 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:05.401834 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 425) took 1861 ms. Trace:
I20260709 13:06:05.402281 21928 rpcz_store.cc:276] 0709 13:06:03.540212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.383541 (+1843329us) service_pool.cc:224] Handling call
0709 13:06:05.400891 (+ 17350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.401585 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.401777 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.398377 22585 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:06:05.379141 24430 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:05.398285 24449 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.384158 24310 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.210831 24283 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.384156 24041 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:05.383692 24165 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:06:05.399781 24247 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.357475 24423 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.402773 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:05.416216 24328 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:05.417899 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1955) took 1638 ms. Trace:
I20260709 13:06:05.417795 24430 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.418330 21935 rpcz_store.cc:276] 0709 13:06:03.779032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.600015 (+820983us) service_pool.cc:224] Handling call
0709 13:06:05.417711 (+817696us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b5400429ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.417837 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152039680}
I20260709 13:06:05.419032 24430 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:05.411849 24440 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.421880 24440 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.422816 24423 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.423833 21931 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:05.408965 24415 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:05.406922 24426 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:05.411113 21928 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:05.406565 22585 pending_rounds.cc:85] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 5
I20260709 13:06:05.413363 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 418) took 2102 ms. Trace:
I20260709 13:06:05.430233 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.407279 24455 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:05.438529 24455 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.438964 24455 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:05.439283 24283 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:05.407545 24234 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:05.423398 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.891s (SENT) [suppressed 11 similar messages]
I20260709 13:06:05.425419 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.430301 21930 rpcz_store.cc:276] 0709 13:06:03.306824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.307042 (+   218us) service_pool.cc:224] Handling call
0709 13:06:03.337902 (+ 30860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.338191 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.338221 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.338778 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.645081 (+306303us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:03.645138 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.645246 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.178592 (+1533346us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:05.398582 (+219990us) raft_consensus.cc:1685] finished
0709 13:06:05.398605 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.409381 (+ 10776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25376000,"thread_start_us":268211,"threads_started":1,"wal-append.queue_time_us":509757}
W20260709 13:06:05.447471 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 62622.4us
I20260709 13:06:05.431778 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1934) took 1703 ms. Trace:
I20260709 13:06:05.448482 21928 rpcz_store.cc:276] 0709 13:06:03.728343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.403288 (+1674945us) service_pool.cc:224] Handling call
0709 13:06:05.410909 (+  7621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.431546 (+ 20637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.431723 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:05.430383 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.434537 24367 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:05.392356 24166 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.452162 21856 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:05.436445 24426 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.437446 24424 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:06:05.465444 22588 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:05.465521 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:05.406329 24318 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:05.437767 21929 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:05.409219 21932 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:05.449752 22585 pending_rounds.cc:107] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:06:05.452895 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:05.432783 24440 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.451092 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:05.482461 21928 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:05.485306 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 426) took 1882 ms. Trace:
I20260709 13:06:05.494655 21932 rpcz_store.cc:276] 0709 13:06:03.602674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.400006 (+1797332us) service_pool.cc:224] Handling call
0709 13:06:05.409013 (+  9007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.485016 (+ 76003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.485252 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.453223 24367 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.424562 24041 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.424815 24449 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.501832 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1939) took 1752 ms. Trace:
I20260709 13:06:05.505096 21928 rpcz_store.cc:276] 0709 13:06:03.742171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.449305 (+1707134us) service_pool.cc:224] Handling call
0709 13:06:05.482272 (+ 32967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.494230 (+ 11958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.494488 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.464663 24283 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:05.502240 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:05.504197 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55998us
I20260709 13:06:05.510608 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.506649 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.512393 24072 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.513119 24440 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:05.513764 24474 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:05.514312 24474 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:05.507839 21930 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:05.515198 24474 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.515709 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1948) took 1770 ms. Trace:
I20260709 13:06:05.516049 21930 rpcz_store.cc:276] 0709 13:06:03.745108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.477788 (+1732680us) service_pool.cc:224] Handling call
0709 13:06:05.507614 (+ 29826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.515471 (+  7857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.515655 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.517387 21930 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:05.517937 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1954) took 1739 ms. Trace:
I20260709 13:06:05.512907 24072 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.518265 21930 rpcz_store.cc:276] 0709 13:06:03.778754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.516758 (+1738004us) service_pool.cc:224] Handling call
0709 13:06:05.517088 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.517716 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.517883 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.518700 24106 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.519990 24072 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:05.520987 24486 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:06:05.480765 24445 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.522465 21930 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:05.485554 22585 raft_consensus.cc:2889] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:05.452703 24234 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.526670 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:05.529392 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1958) took 1743 ms. Trace:
I20260709 13:06:05.501245 24444 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:05.531961 21932 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:05.504789 24367 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:05.538676 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1951) took 1792 ms. Trace:
I20260709 13:06:05.539268 21932 rpcz_store.cc:276] 0709 13:06:03.777838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.511561 (+1733723us) service_pool.cc:224] Handling call
0709 13:06:05.531674 (+ 20113us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.538297 (+  6623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.538601 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.544961 21825 leader_election.cc:400] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6.
I20260709 13:06:05.548379 21825 leader_election.cc:403] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:05.536672 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:05.553918 21972 leader_election.cc:400] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:06:05.530273 24364 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.556025 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.556226 24364 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.556934 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.521901 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1938) took 1766 ms. Trace:
I20260709 13:06:05.509099 24166 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.533460 21930 rpcz_store.cc:276] 0709 13:06:03.779809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.519060 (+1739251us) service_pool.cc:224] Handling call
0709 13:06:05.519318 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.523096 (+  3778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.523325 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.517953 24247 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.523880 24328 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:05.557973 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:05.520082 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 414) took 2574 ms (client timeout 2132 ms). Trace:
I20260709 13:06:05.536880 24440 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:05.564584 24364 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:05.535704 24445 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.560501 24106 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:05.530858 24283 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:05.568183 24430 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.568323 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.568841 24457 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.569376 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.638553 22591 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEARNER]: Advancing to term 5
I20260709 13:06:05.569891 24433 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:05.570453 24455 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.535218 24444 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.559499 21929 rpcz_store.cc:276] 0709 13:06:03.741848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.437224 (+1695376us) service_pool.cc:224] Handling call
0709 13:06:05.437587 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.480483 (+ 42896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.508634 (+ 28151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.586715 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:05.593294 24318 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.598609 24041 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:05.600390 24449 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.530223 24415 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:05.608757 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:06:05.610409 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.605908 24328 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:05.695645 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:05.696765 22592 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:05.614526 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.620307 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 75679.3us
I20260709 13:06:05.563337 24220 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:05.637359 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 81471.8us
I20260709 13:06:05.645639 22587 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:05.611640 24383 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.558723 24453 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.696379 24328 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.563174 21930 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:05.704133 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:05.704713 24454 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.509s	user 0.005s	sys 0.000s
I20260709 13:06:05.705693 24220 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:05.678830 24444 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:05.707695 21932 raft_consensus.cc:606] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:05.724784 21932 raft_consensus.cc:614] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:05.718493 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:05.720741 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1925) took 2130 ms. Trace:
I20260709 13:06:05.721099 24220 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:05.722162 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1960) took 1863 ms. Trace:
W20260709 13:06:05.606235 21927 rpcz_store.cc:269] 0709 13:06:02.945759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.964292 (+ 18533us) service_pool.cc:224] Handling call
0709 13:06:03.738794 (+774502us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaeee 0000000000375939 00007f3b7571e1e0 00007f3b757e70c4 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:05.519834 (+1781040us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b540003cd54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.520006 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2090807424}
I20260709 13:06:05.745224 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 421) took 2334 ms. Trace:
I20260709 13:06:05.744073 21930 rpcz_store.cc:276] 0709 13:06:03.854864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.562558 (+1707694us) service_pool.cc:224] Handling call
0709 13:06:05.562913 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.707447 (+144534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.717204 (+  9757us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.745412 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:05.745651 21931 rpcz_store.cc:276] 0709 13:06:03.411216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.368448 (+1957232us) service_pool.cc:224] Handling call
0709 13:06:05.368824 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.428889 (+ 60065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.744957 (+316068us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b54003061d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:05.745166 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126049536}
I20260709 13:06:05.746659 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:05.747579 22593 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:05.747953 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 422) took 2308 ms. Trace:
I20260709 13:06:05.748452 21934 rpcz_store.cc:276] 0709 13:06:03.439754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.368936 (+1929182us) service_pool.cc:224] Handling call
0709 13:06:05.424002 (+ 55066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.424165 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.424187 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.424224 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:05.424273 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.424333 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.747719 (+323386us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b5400429ad0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:05.747880 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123471104}
I20260709 13:06:05.729187 22000 rpcz_store.cc:276] 0709 13:06:03.590730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.590866 (+   136us) service_pool.cc:224] Handling call
0709 13:06:03.633134 (+ 42268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.641204 (+  8070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.641231 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.641714 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.693290 (+ 51576us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:03.693338 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.693420 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.704133 (+2010713us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:05.720418 (+ 16285us) raft_consensus.cc:1685] finished
0709 13:06:05.720439 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.720656 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6372,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":3528,"spinlock_wait_cycles":6374784,"thread_start_us":51222,"threads_started":1,"wal-append.queue_time_us":213431}
I20260709 13:06:05.640166 24430 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.750615 22592 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:05.763336 24455 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.773041 24283 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.775712 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 328) took 2497 ms. Trace:
I20260709 13:06:05.776224 21856 rpcz_store.cc:276] 0709 13:06:03.278476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.790834 (+512358us) service_pool.cc:224] Handling call
0709 13:06:05.360544 (+1569710us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:06:05.360732 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.463067 (+102335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.775421 (+312354us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b540018f1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:05.775637 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216897024}
I20260709 13:06:05.777516 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:05.778301 21851 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:05.779625 22589 raft_consensus.cc:2376] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6. Re-sending same reply.
W20260709 13:06:05.779800 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 328789us
I20260709 13:06:05.780596 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.780735 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1937) took 2039 ms. Trace:
I20260709 13:06:05.781308 21935 rpcz_store.cc:276] 0709 13:06:03.741534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.419147 (+1677613us) service_pool.cc:224] Handling call
0709 13:06:05.419515 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.419757 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.419779 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.420281 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.536078 (+115797us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:05.536126 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.536187 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.780455 (+244268us) raft_consensus.cc:1685] finished
0709 13:06:05.780482 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.780662 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":115448,"threads_started":1,"wal-append.queue_time_us":260983}
I20260709 13:06:05.783325 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1933) took 2055 ms. Trace:
I20260709 13:06:05.783759 21926 rpcz_store.cc:276] 0709 13:06:03.727973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.390709 (+1662736us) service_pool.cc:224] Handling call
0709 13:06:05.391179 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.408341 (+ 17162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.408370 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.432094 (+ 23724us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.637258 (+205164us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:05.637294 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.637357 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.783125 (+145768us) raft_consensus.cc:1685] finished
0709 13:06:05.783143 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.783272 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6441,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":173,"spinlock_wait_cycles":5376,"thread_start_us":131803,"threads_started":1,"wal-append.queue_time_us":230091}
I20260709 13:06:05.796718 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.798651 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:05.802976 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87669.8us
I20260709 13:06:05.804220 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.807207 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.795893 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:05.814358 21926 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:06:05.818001 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.819433 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.821194 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:05.821300 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.822126 21934 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:05.823084 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.812834 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:05.724889 24453 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:05.825178 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 401768us
I20260709 13:06:05.825704 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.809792 24430 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:05.847224 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:05.860616 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.907035 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.896963 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:05.907917 21934 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.901350 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:05.813149 24433 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:05.906311 22587 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:05.823648 24434 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:05.825260 24166 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:05.825031 24415 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:06:05.913825 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 109798us
I20260709 13:06:05.923841 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.915768 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.916411 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:05.925196 21930 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:05.829257 24165 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:05.916922 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.920907 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.926374 21935 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:05.932044 21931 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:05.932386 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:05.909238 24428 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:05.935495 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.910290 24433 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:05.900725 24310 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.942305 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:05.943020 21933 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
W20260709 13:06:05.943475 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.927441 24165 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.941846 21927 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:05.962358 24165 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:05.941461 24430 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:05.972669 22591 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260709 13:06:05.964517 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 116097us
I20260709 13:06:05.964984 24267 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:05.934391 24429 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:05.969077 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:05.975315 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:05.955015 24220 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:05.942771 24433 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:05.974649 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.933261 24426 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:05.980120 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.950182 24455 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:06:05.988791 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.016551 22002 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:05.975147 21929 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:05.990985 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:05.993151 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:05.993435 21934 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:05.993641 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:05.973624 24430 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.994803 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:05.995358 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:05.995975 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:05.996570 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.033159 22005 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.033490 22007 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.997206 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.034337 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:06.034648 22000 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:05.999993 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:06.035494 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
W20260709 13:06:06.035740 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44842.5us
I20260709 13:06:06.000838 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1950) took 2255 ms. Trace:
I20260709 13:06:06.036599 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.036405 21928 rpcz_store.cc:276] 0709 13:06:03.745743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.505975 (+1760232us) service_pool.cc:224] Handling call
0709 13:06:05.527960 (+ 21985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.541055 (+ 13095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.541098 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.603201 (+ 62103us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.854078 (+250877us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:05.854138 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.854247 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.000548 (+146301us) raft_consensus.cc:1685] finished
0709 13:06:06.000577 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.000762 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":215856,"threads_started":1,"wal-append.queue_time_us":321346}
I20260709 13:06:05.963510 24477 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.039515 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1921) took 1048 ms. Trace:
I20260709 13:06:06.040076 22000 rpcz_store.cc:276] 0709 13:06:04.990820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.790685 (+799865us) service_pool.cc:224] Handling call
0709 13:06:06.039437 (+248752us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.979836 24434 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.005126 22593 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.005159 24457 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:06.008035 24367 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.037392 24328 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.059012 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:06.060640 24367 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.009945 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:05.961612 24493 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:06:06.010926 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:06.017387 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.064292 22587 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.025328 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1920) took 1034 ms. Trace:
I20260709 13:06:06.064528 24493 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:06.026980 22004 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.951573 24428 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:05.994199 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:05.980691 24262 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.006s
I20260709 13:06:06.025398 24166 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:06.043479 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1923) took 1052 ms. Trace:
I20260709 13:06:06.003993 24234 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.046239 22006 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:06.012072 24415 raft_consensus.cc:2764] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 13:06:06.018541 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.076282 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1905) took 2345 ms. Trace:
I20260709 13:06:06.081849 22003 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:06.064770 22002 rpcz_store.cc:276] 0709 13:06:04.990592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.778106 (+787514us) service_pool.cc:224] Handling call
0709 13:06:06.025257 (+247151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.059082 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.090137 22008 rpcz_store.cc:276] 0709 13:06:03.731006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.430735 (+699729us) service_pool.cc:224] Handling call
0709 13:06:04.458836 (+ 28101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.126535 (+667699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.076021 (+949486us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54000ca8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:06.076214 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029153536}
I20260709 13:06:06.094796 22001 rpcz_store.cc:276] 0709 13:06:04.991511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.324948 (+333437us) service_pool.cc:224] Handling call
0709 13:06:06.043393 (+718445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.095005 22009 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:06.095928 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1916) took 1106 ms. Trace:
I20260709 13:06:06.096292 22009 rpcz_store.cc:276] 0709 13:06:04.989582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.116890 (+127308us) service_pool.cc:224] Handling call
0709 13:06:06.095869 (+978979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.101745 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1919) took 1099 ms. Trace:
I20260709 13:06:06.102334 22006 rpcz_store.cc:276] 0709 13:06:04.990341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.010560 (+ 20219us) service_pool.cc:224] Handling call
0709 13:06:06.089733 (+1079173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.037518 24430 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.115840 22004 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:06.103893 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.105432 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.116396 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1925) took 1073 ms. Trace:
I20260709 13:06:06.108170 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:06.116729 22000 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:06.117236 21934 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:06.116921 22002 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:06.119601 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1917) took 1129 ms. Trace:
I20260709 13:06:06.119890 22002 rpcz_store.cc:276] 0709 13:06:04.989863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.105119 (+1115256us) service_pool.cc:224] Handling call
0709 13:06:06.119530 (+ 14411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.120738 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.121416 22002 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:06.121767 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1927) took 1077 ms. Trace:
I20260709 13:06:06.122048 22002 rpcz_store.cc:276] 0709 13:06:05.044615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.120517 (+1075902us) service_pool.cc:224] Handling call
0709 13:06:06.121719 (+  1202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.122856 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.123407 22002 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:06.123561 22000 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
W20260709 13:06:06.123595 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 113730us
I20260709 13:06:06.123903 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1922) took 1132 ms. Trace:
I20260709 13:06:06.124128 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1928) took 1079 ms. Trace:
I20260709 13:06:06.109803 22592 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:06.124681 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 107382us
I20260709 13:06:06.126076 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:06.091306 24106 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:06.111657 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.136035 22006 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.137173 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.138257 21928 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:06.056481 24457 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.142294 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.143332 21933 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:06.081537 24477 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:06.068964 24423 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:06.171442 22006 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:06.172061 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1918) took 1182 ms. Trace:
I20260709 13:06:06.172442 22006 rpcz_store.cc:276] 0709 13:06:04.990099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.111407 (+1121308us) service_pool.cc:224] Handling call
0709 13:06:06.172000 (+ 60593us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.199121 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:06.199853 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.124454 22000 rpcz_store.cc:276] 0709 13:06:05.044964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.050467 (+1005503us) service_pool.cc:224] Handling call
0709 13:06:06.124077 (+ 73610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.116765 22004 rpcz_store.cc:276] 0709 13:06:05.043471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.070011 (+ 26540us) service_pool.cc:224] Handling call
0709 13:06:06.116337 (+1046326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.069830 24328 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.132825 21934 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
W20260709 13:06:06.145123 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86192us
I20260709 13:06:06.207726 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.126883 24426 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.148881 24173 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:06.141765 24457 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.155691 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.221463 22001 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:06.222311 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.222869 22001 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:06.221655 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:06.163094 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1941) took 1037 ms. Trace:
I20260709 13:06:06.088958 24493 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:06.224041 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.224062 22003 rpcz_store.cc:276] 0709 13:06:05.125855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.170292 (+ 44437us) service_pool.cc:224] Handling call
0709 13:06:06.162850 (+992558us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b5400169254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:06.163012 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112783744}
I20260709 13:06:06.160766 24166 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:06:06.084702 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 673) took 2751 ms. Trace:
I20260709 13:06:06.228246 24493 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.228467 22585 rpcz_store.cc:276] 0709 13:06:03.333364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.352878 (+ 19514us) service_pool.cc:224] Handling call
0709 13:06:03.409299 (+ 56421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.536174 (+2126875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.084377 (+548203us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54005019d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:06.084601 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4567025408}
I20260709 13:06:06.229387 24493 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.085526 24445 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.229820 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.178503 24165 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:06.230618 22003 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:06.065868 22586 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 628) took 2526 ms (client timeout 2790 ms). Trace:
I20260709 13:06:06.231503 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.231701 22007 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:06.186830 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1880) took 1991 ms. Trace:
I20260709 13:06:06.232268 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1926) took 1188 ms. Trace:
I20260709 13:06:06.187458 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:06.191259 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 485784us
W20260709 13:06:06.192611 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.193178 24383 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6.
I20260709 13:06:06.194842 24428 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.197989 24423 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:06.196095 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 415540us
I20260709 13:06:06.196165 24310 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.196743 24474 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:06.144058 24434 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:06.201172 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.249085 22006 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:06.110672 22005 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:06.204308 22587 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:06.184121 21753 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 3, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:06.147276 21932 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:06.250595 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.250890 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1942) took 1100 ms. Trace:
I20260709 13:06:06.212462 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.124325 22002 rpcz_store.cc:276] 0709 13:06:04.991225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.122598 (+1131373us) service_pool.cc:224] Handling call
0709 13:06:06.123854 (+  1256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.217563 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:06.140803 24511 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.251549 22005 rpcz_store.cc:276] 0709 13:06:05.150704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.386184 (+235480us) service_pool.cc:224] Handling call
0709 13:06:06.250830 (+864646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.261061 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.263419 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.264012 21854 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:06.264791 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.265545 22001 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:06.226778 22000 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:06:06.232810 22007 rpcz_store.cc:276] 0709 13:06:05.044235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.421556 (+377321us) service_pool.cc:224] Handling call
0709 13:06:06.232214 (+810658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.225922 24426 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:06.195091 24425 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:06.239395 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:06.232390 21860 rpcz_store.cc:276] 0709 13:06:04.195033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.195199 (+   166us) service_pool.cc:224] Handling call
0709 13:06:04.195545 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.195753 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.195776 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.428519 (+1232743us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b2c000e8e68. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:06:05.432262 (+  3743us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.614672 (+182410us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:05.640604 (+ 25932us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.186304 (+545700us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b1800477580. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:06:06.186480 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.186580 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.186621 (+    41us) raft_consensus.cc:1685] finished
0709 13:06:06.186635 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.186767 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7829,"prepare.queue_time_us":29428,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":546499,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":77550208,"thread_start_us":182036,"threads_started":1,"wal-append.queue_time_us":329528}
I20260709 13:06:06.247673 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.252802 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:06.254285 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:06.255851 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.276146 21859 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:06.265710 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.277287 22003 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:06.233377 22008 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.266685 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.279119 22001 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:06.266744 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.279835 22007 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:06.245747 24445 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.261988 22005 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:06:06.268784 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.208268 24477 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:06.262842 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.271688 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.288497 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.246008 24328 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:06.256651 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.282321 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:06.283046 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 1940) took 1317 ms. Trace:
I20260709 13:06:06.233032 24220 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:06:06.247499 24086 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.292470 21753 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 4, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.306003 22007 pending_rounds.cc:85] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 2
I20260709 13:06:06.306510 22007 pending_rounds.cc:107] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:06.306859 22007 raft_consensus.cc:2889] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.307855 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.315768 21932 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:06.315487 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.315857 22008 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:06.308790 21859 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:06.286623 22005 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.289408 21853 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 2.
I20260709 13:06:06.289441 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:06.325610 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1939) took 1360 ms. Trace:
I20260709 13:06:06.325908 22002 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
W20260709 13:06:06.226094 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 92728.5us
I20260709 13:06:06.327984 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:06.328285 21930 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:06.235707 24506 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:06:06.291153 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1943) took 1125 ms. Trace:
I20260709 13:06:06.294121 22003 rpcz_store.cc:276] 0709 13:06:04.966118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.265505 (+1299387us) service_pool.cc:224] Handling call
0709 13:06:06.282991 (+ 17486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.294436 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:06.329541 22006 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:06.329258 22001 rpcz_store.cc:276] 0709 13:06:05.165879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.290431 (+1124552us) service_pool.cc:224] Handling call
0709 13:06:06.290676 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.290847 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.290871 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.290923 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:06.290959 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.291010 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.291103 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.331696 22001 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:06.332921 22001 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:06.325992 22005 rpcz_store.cc:276] 0709 13:06:04.965416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.286158 (+1320742us) service_pool.cc:224] Handling call
0709 13:06:06.286453 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.325387 (+ 38934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.325560 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.334154 22001 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:06.337513 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:06.340123 22002 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:06.234165 22586 rpcz_store.cc:269] 0709 13:06:03.539754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.582933 (+ 43179us) service_pool.cc:224] Handling call
0709 13:06:06.065538 (+2482605us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54005000d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:06.065764 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4488639360}
I20260709 13:06:06.341750 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:06.293685 24383 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.345983 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:06.350147 22002 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.297751 21927 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.293556 24165 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.273236 24433 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:06.386651 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 169150us
I20260709 13:06:06.389361 22002 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:06:06.389602 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 107344us
I20260709 13:06:06.389819 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.390508 21823 leader_election.cc:400] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5.
I20260709 13:06:06.390837 21823 leader_election.cc:403] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:06:06.393442 22009 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:06:06.393635 22002 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:06.393903 22009 pending_rounds.cc:85] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:06:06.394131 22009 pending_rounds.cc:107] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:06:06.395351 22009 raft_consensus.cc:2889] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.397069 22009 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:06.398583 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.290400 24445 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.344012 24292 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:06.345486 24250 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.426260 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:06.318214 24430 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.326896 24086 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:06.417184 24471 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5.
I20260709 13:06:06.346729 24434 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.385557 24512 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.386350 24262 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.416178 21753 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:06.315481 24328 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:06.298061 24423 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:06.438300 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 185625us
I20260709 13:06:06.425670 24292 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.438402 21857 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:06.386154 24310 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:06:06.408351 24318 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:06.437302 24521 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.419723 24367 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.419962 24165 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:06.450331 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.345809 24511 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:06:06.457653 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.472702 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:06.440656 24425 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.475910 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:06.454478 24514 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:06:06.484951 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 160879us
I20260709 13:06:06.485147 21858 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:06:06.485704 21753 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
W20260709 13:06:06.487980 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 142092us
I20260709 13:06:06.488602 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.488786 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:06.455794 24292 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.491804 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:06.494582 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:06.495590 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:06.496529 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.458163 24449 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.474298 24383 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.471817 24165 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.455289 24495 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.507016 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.508005 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.478667 24262 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:06.520112 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:06.523242 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:06.526022 24318 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:06.527500 24318 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:06.532130 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.504956 21753 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 3 to 4, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.539431 21855 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:06.540422 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.504231 24512 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:06.545444 24512 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.502133 24086 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.497347 24521 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:06.586597 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.481542 24367 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.504402 24434 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:06.514782 24506 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:06.512511 24165 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:06.451006 24426 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.611104 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.613775 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.614537 21855 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:06.625598 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.628108 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.629686 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.630663 21934 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:06.629648 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.634404 21932 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:06.634753 21935 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:06:06.635179 21935 pending_rounds.cc:85] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 2
I20260709 13:06:06.635489 21935 pending_rounds.cc:107] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:06:06.635798 21935 raft_consensus.cc:2889] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.637247 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.642719 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:06.644589 21935 pending_rounds.cc:107] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:06:06.644929 21935 raft_consensus.cc:2889] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.645262 21930 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:06:06.645679 21930 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 3
I20260709 13:06:06.645850 21934 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:06.645943 21930 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 13:06:06.646395 21930 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:06:06.628736 21926 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:06.644860 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.515861 24383 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.600723 24512 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:06.512696 24449 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.798009 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:06.745605 21753 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
W20260709 13:06:06.802263 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.802134 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.803174 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:06.648083 22592 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:06.600288 24529 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:06.803622 21930 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.806098 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.807675 22589 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:06.843222 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.535369 24495 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:06.810717 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 178477us
I20260709 13:06:06.805356 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:06.804066 24509 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.420s	user 0.000s	sys 0.001s
I20260709 13:06:06.810590 24506 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:06.811525 24326 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.646407 21935 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:06.643821 24434 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:06.744374 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:06.865809 21852 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:06.868716 21860 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:06.840271 24433 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.606112 24525 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:06.875589 22589 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:06:06.860571 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:06.862785 24430 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:06.833935 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 444154us
W20260709 13:06:06.837880 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 365228us
I20260709 13:06:06.814193 24110 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:06.894711 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:06.847388 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44301.9us
I20260709 13:06:06.860123 21933 raft_consensus.cc:2376] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4. Re-sending same reply.
W20260709 13:06:06.862439 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 225247us
I20260709 13:06:06.869304 24445 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:06.866963 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.818329 24477 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:06.819476 24310 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:06.832834 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 336294us
I20260709 13:06:06.858574 24529 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.867759 24086 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:06.919350 24474 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.656s	user 0.005s	sys 0.006s
I20260709 13:06:06.925309 24529 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.925384 24086 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.925617 24474 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.926049 24086 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.926343 21859 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:06.890173 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.855662 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.930652 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:06.932121 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.876001 22589 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 3
I20260709 13:06:06.933055 22589 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 13:06:06.933139 21859 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:06.933727 22589 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:06:06.934378 22589 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.868237 24328 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:06.901575 24526 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:06:06.939487 22002 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:06.903573 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1951) took 1373 ms. Trace:
I20260709 13:06:06.904016 24367 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:06.905189 21929 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:06.909920 24449 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.911370 24383 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.913631 24495 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.896011 24523 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.916745 24165 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.969599 24523 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.971118 24165 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.971781 24523 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.622298 24521 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.938953 24267 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:06.973148 24267 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:06.973029 24292 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:06.974169 24267 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.975232 24267 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:06.965691 24506 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:06.916738 24173 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
I20260709 13:06:06.977270 24173 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:06.896955 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.978785 24523 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:06.979010 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 82164.8us
I20260709 13:06:06.980048 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:06.980314 24523 raft_consensus.cc:993] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:06:06.920603 21855 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:06.981141 24537 raft_consensus.cc:1081] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Signalling peer 8f3b35a35257416db3eaaf364cf6b52b to start an election
I20260709 13:06:06.899520 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:06.932979 24086 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.984143 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:06.862932 24262 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:06.938637 22001 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:06.984824 22584 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:06.985236 24086 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:06.940317 22005 rpcz_store.cc:276] 0709 13:06:05.530623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.334856 (+804233us) service_pool.cc:224] Handling call
0709 13:06:06.335078 (+   222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.392297 (+ 57219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.394156 (+  1859us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:06.420941 (+ 26785us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.595456 (+174515us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:06.595512 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.595612 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.903352 (+307740us) raft_consensus.cc:1685] finished
0709 13:06:06.903374 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.903514 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26022,"raft.queue_time_us":154137,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":11660,"thread_start_us":198796,"threads_started":2,"wal-append.queue_time_us":374350}
I20260709 13:06:06.844137 21753 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 3 to 4, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.869112 24326 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.986891 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:06.987670 24326 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.980854 24523 raft_consensus.cc:2764] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:06.988874 21753 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:06.990494 21753 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 3, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:06:06.992336 21753 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 3, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:06.914189 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 285572us
W20260709 13:06:06.982208 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 82778.8us
I20260709 13:06:06.983242 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.005319 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:07.006626 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:07.007321 22000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1924) took 1993 ms (client timeout 2644 ms). Trace:
I20260709 13:06:06.984282 24495 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:06.994937 22585 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:07.012346 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:06.975188 24292 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:06.935110 24536 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.003114 21823 connection.cc:256] RPC call timeout handler was delayed by -1.3452157539999234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 13:06:07.018587 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:07.018692 21823 connection.cc:256] RPC call timeout handler was delayed by -1.3357482829999299s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3315s for a response
W20260709 13:06:06.966262 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:06.964859 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43486.4us
I20260709 13:06:07.005419 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:07.007416 22008 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:07.007790 22000 rpcz_store.cc:269] 0709 13:06:05.014424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.207496 (+1193072us) service_pool.cc:224] Handling call
0709 13:06:07.007117 (+799621us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b5400166554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.007261 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41592064}
I20260709 13:06:06.918430 24110 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:06.900568 24434 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.014122 21753 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:07.014773 24537 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:07.003124 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:07.030781 21973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.023s
W20260709 13:06:07.031749 21973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.006s	sys 0.023s
I20260709 13:06:07.009553 24310 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:07.011734 24495 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.012212 24523 raft_consensus.cc:2764] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:06:07.009316 24086 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:06:07.044833 21855 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:06.944319 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:07.033409 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.034430 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.020682 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.045696 21926 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
W20260709 13:06:07.046056 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 101839us
I20260709 13:06:07.046269 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:07.046629 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:07.036597 21854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.021s	sys 0.018s
I20260709 13:06:07.037796 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:07.040167 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.029211 21753 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 3, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.039007 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:07.048506 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:07.051115 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.026428 24538 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:07.041289 24433 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:07.044965 21972 connection.cc:256] RPC call timeout handler was delayed by -2.7104109489999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22690000000000002s for a response
I20260709 13:06:07.047209 21926 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:07.048322 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:07.051196 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:07.043249 24495 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.052343 21928 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:06:07.050523 24444 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.293s	user 0.010s	sys 0.003s
I20260709 13:06:07.053737 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:07.069571 21930 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:07.072575 22005 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:07.054055 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.027706 24173 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:06:07.051811 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:07.050428 21753 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:07.036053 24537 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:07.063643 22000 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:07.043565 21853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.009s	sys 0.009s
I20260709 13:06:07.069483 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:07.049341 21854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.022s	sys 0.018s
I20260709 13:06:07.048388 24292 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:07.085414 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:07.050606 24383 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:07.068150 24540 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.084780 24534 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:06:07.089082 24495 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:07.089702 24527 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:06:07.059199 24533 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:07.098282 21928 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:07.093467 22005 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:07.290191 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:07.291074 22005 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:06:07.092296 21853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.009s	sys 0.009s
I20260709 13:06:07.025913 24536 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:07.066710 24433 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.107718 22001 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:07.120337 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.108701 21753 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:07.122006 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.128078 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58702.6us
I20260709 13:06:07.085901 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.299779 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.131207 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.300050 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:07.300539 21934 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:07.300662 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:07.074524 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.243778 24429 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:07.162978 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 108274us
I20260709 13:06:07.262014 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 557) took 1623 ms. Trace:
W20260709 13:06:07.214588 24493 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.985s	user 0.012s	sys 0.001s
I20260709 13:06:07.064255 24449 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:06:07.191108 24086 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.135277 24533 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:07.190183 24453 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:07.092679 24444 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.300137 24433 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.127801 24262 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:06:07.124596 24474 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:06:07.097775 24537 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.201858 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:07.205860 24457 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.987s	user 0.008s	sys 0.003s
I20260709 13:06:07.130810 24525 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.091141 24531 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260709 13:06:07.178416 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 104331us
I20260709 13:06:07.126919 24540 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:07.266505 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.269630 24513 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:06:07.216557 24515 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.725s	user 0.000s	sys 0.001s
I20260709 13:06:07.202539 24495 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:07.221979 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1938) took 2333 ms. Trace:
W20260709 13:06:07.275589 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 266775us
I20260709 13:06:07.278599 24529 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:07.167362 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 117159us
I20260709 13:06:07.127590 24383 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:07.302505 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.400084 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.401324 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.306793 22593 rpcz_store.cc:276] 0709 13:06:05.638273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.748205 (+109932us) service_pool.cc:224] Handling call
0709 13:06:06.004818 (+256613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.049051 (+ 44233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.261709 (+1212658us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54005325d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.261917 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518117248}
I20260709 13:06:07.312026 22002 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:07.393347 22007 rpcz_store.cc:276] 0709 13:06:04.888795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.285645 (+1396850us) service_pool.cc:224] Handling call
0709 13:06:06.305721 (+ 20076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.406040 (+100319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.406074 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:06.406651 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.486274 (+ 79623us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:06.486307 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.486370 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.221682 (+735312us) raft_consensus.cc:1685] finished
0709 13:06:07.221718 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.221900 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":48563,"threads_started":1,"wal-append.queue_time_us":441034}
I20260709 13:06:07.312826 22557 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:07.299328 24471 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:07.189242 24110 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:07.333137 21935 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:07.450001 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 141536us
I20260709 13:06:07.333351 21934 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:07.305783 24429 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:07.311354 24421 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.338286 21854 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:07.388414 21931 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:07.389385 24513 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:07.392478 22590 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:07.308527 24173 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:07.313645 24267 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:07.395151 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:07.474679 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79603.2us
I20260709 13:06:07.333986 24517 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.367664 24477 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:07.390548 24444 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:07.414523 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.392933 21929 raft_consensus.cc:2368] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:06:07.331526 24533 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.451099 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:07.469436 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 648) took 1540 ms. Trace:
I20260709 13:06:07.498476 22591 rpcz_store.cc:276] 0709 13:06:05.928880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.061407 (+132527us) service_pool.cc:224] Handling call
0709 13:06:06.095611 (+ 34204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.095983 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.096013 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:06.111747 (+ 15734us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.297979 (+186232us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:06.298038 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.298133 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.215828 (+917695us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:07.391417 (+175589us) raft_consensus.cc:1685] finished
0709 13:06:07.391438 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.469326 (+ 77888us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":25890,"spinlock_wait_cycles":42117504,"thread_start_us":97041,"threads_started":1,"wal-append.queue_time_us":241251}
I20260709 13:06:07.475957 21932 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:07.480422 21934 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:07.333503 24292 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:06:07.393177 24426 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.481209 24262 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:07.481469 24477 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:07.502678 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.473660 24173 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:07.504627 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 105866us
I20260709 13:06:07.474277 24445 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:07.481743 24549 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
W20260709 13:06:07.478556 24524 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.616s	user 0.000s	sys 0.006s
I20260709 13:06:07.336230 24512 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:07.331286 24546 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.174s
I20260709 13:06:07.391321 24457 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.331632 24086 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:07.518743 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67768.4us
I20260709 13:06:07.496476 24511 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.477357 24110 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:06:07.540553 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.503504 24267 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:07.543725 21854 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:07.515990 24262 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:07.523527 24173 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:07.567957 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 241929us
W20260709 13:06:07.568591 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.523861 24444 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:07.522658 24445 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:07.575428 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.488268 24517 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.522017 24449 consensus_queue.cc:1243] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:06:07.583751 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.593021 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.593665 21854 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.593675 24477 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.595274 24477 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:07.598914 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 330) took 1168 ms. Trace:
W20260709 13:06:07.600039 22588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 638) took 2095 ms (client timeout 2147 ms). Trace:
I20260709 13:06:07.599315 21856 rpcz_store.cc:276] 0709 13:06:06.430208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.430336 (+   128us) service_pool.cc:224] Handling call
0709 13:06:06.443686 (+ 13350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.473591 (+ 29905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.473631 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:06.500620 (+ 26989us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.870646 (+370026us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:06.870700 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.598525 (+727825us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b54002b9b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:07.598573 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.598617 (+    44us) raft_consensus.cc:1685] finished
0709 13:06:07.598630 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.598855 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8877,"raft.queue_time_us":3121,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7728,"spinlock_wait_cycles":11208960,"thread_start_us":366263,"threads_started":1,"wal-append.queue_time_us":464931}
W20260709 13:06:07.600422 22588 rpcz_store.cc:269] 0709 13:06:05.504603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.506340 (+  1737us) service_pool.cc:224] Handling call
0709 13:06:07.599857 (+2093517us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54006b3ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.599985 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605607808}
I20260709 13:06:07.601788 24536 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:07.602169 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1896) took 1204 ms. Trace:
I20260709 13:06:07.602538 21859 rpcz_store.cc:276] 0709 13:06:06.398112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.398330 (+   218us) service_pool.cc:224] Handling call
0709 13:06:07.602019 (+1203689us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b540010c054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.602116 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212292480}
I20260709 13:06:07.603672 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.604555 24086 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:07.605597 24551 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:06:07.583806 24429 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.609987 24429 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.544420 24383 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:07.612916 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.617956 21854 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:07.541766 24426 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:07.630177 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51137.4us
I20260709 13:06:07.581516 24445 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:07.569068 24453 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:06:07.546074 24421 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:06:07.616528 24423 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.633150 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.647557 21854 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:07.648892 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.650241 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.652086 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:07.652269 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 107457us
I20260709 13:06:07.653898 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.578343 24493 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:06:07.543995 24513 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.592269 24546 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:06:07.638863 24557 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:07.651454 24449 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:06:07.661190 24513 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:07.655130 24474 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.663729 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.651851 24536 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.665472 24553 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:06:07.665308 24536 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:07.665810 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 337) took 1172 ms. Trace:
I20260709 13:06:07.666242 21860 rpcz_store.cc:276] 0709 13:06:06.493916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.507684 (+ 13768us) service_pool.cc:224] Handling call
0709 13:06:07.665629 (+1157945us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b540028c654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.665740 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815337216}
I20260709 13:06:07.667129 24559 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:07.667917 24561 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:07.668627 24561 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:07.669093 24540 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:07.664884 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.672214 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.672870 24531 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:07.673380 24550 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:06:07.641089 24262 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.674437 24531 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.674811 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:07.675128 24512 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:07.675158 24086 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:07.675735 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.667340 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.677181 21860 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:07.678035 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.678224 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.678707 21860 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:07.678861 21928 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:07.679440 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.679965 21860 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:07.680042 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.680945 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:07.680145 21934 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:07.673261 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.681602 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:07.665155 24267 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.682153 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.658056 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.683679 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:07.684463 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.692495 24540 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:07.692426 24559 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:07.693209 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:07.693392 24540 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.694037 24540 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.694096 24561 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:07.694525 24540 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.694693 24520 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:07.694979 24540 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.695113 24520 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.695493 24520 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.697291 24506 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.692528 24415 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.655766 24558 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:07.709575 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 136902us
I20260709 13:06:07.712734 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:07.684510 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.716463 21854 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:07.717401 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 344) took 1102 ms. Trace:
I20260709 13:06:07.717851 21854 rpcz_store.cc:276] 0709 13:06:06.615203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.715727 (+1100524us) service_pool.cc:224] Handling call
0709 13:06:07.716088 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.716991 (+   903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.717334 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:07.719226 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.719756 21854 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:07.720498 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 345) took 1098 ms. Trace:
I20260709 13:06:07.720868 21854 rpcz_store.cc:276] 0709 13:06:06.621943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.719327 (+1097384us) service_pool.cc:224] Handling call
0709 13:06:07.719588 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.720218 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.720442 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:07.660678 24557 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:07.732606 21854 raft_consensus.cc:1217] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-4.4]   Dedup: 3.3->[4.4-4.4]
I20260709 13:06:07.699093 24383 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.685211 24428 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.685467 24523 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.738675 21854 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260709 13:06:07.698390 24415 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.700846 24512 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.711791 24558 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:07.714320 24529 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:07.785403 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.771338 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 198175us
I20260709 13:06:07.776726 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.786191 21854 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:07.686817 24531 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:06:07.745472 24453 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.787111 21934 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:07.684629 24560 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.115s
I20260709 13:06:07.790889 21928 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:07.792735 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:07.846309 21935 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.799695 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 446) took 1691 ms. Trace:
I20260709 13:06:07.800448 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1890) took 1550 ms. Trace:
I20260709 13:06:07.801616 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1980) took 1169 ms. Trace:
I20260709 13:06:07.850068 21857 rpcz_store.cc:276] 0709 13:06:06.249631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.249762 (+   131us) service_pool.cc:224] Handling call
0709 13:06:07.800284 (+1550522us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54003d3e54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.800389 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351300864}
I20260709 13:06:07.851182 21932 rpcz_store.cc:276] 0709 13:06:06.632520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.644544 (+ 12024us) service_pool.cc:224] Handling call
0709 13:06:07.801419 (+1156875us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b5400098dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.801558 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904816640}
I20260709 13:06:07.802605 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.805132 24546 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.806545 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.808301 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1946) took 2399 ms. Trace:
I20260709 13:06:07.809840 24086 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:07.745795 24173 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:07.859933 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.810091 24558 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:07.776165 24523 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.773396 24415 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.732398 24565 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.151s
I20260709 13:06:07.753772 24383 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:07.845458 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57951.1us
W20260709 13:06:07.844558 22556 connection.cc:256] RPC call timeout handler was delayed by -1.2999852529999316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:06:07.774186 24444 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:07.856766 22003 rpcz_store.cc:276] 0709 13:06:05.408559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.338432 (+929873us) service_pool.cc:224] Handling call
0709 13:06:06.338716 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.480812 (+142096us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:06.481722 (+   910us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.289527 (+807805us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b18003c5560. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:06:07.316193 (+ 26666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:07.316280 (+    87us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:07.316341 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.808049 (+491708us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54002710d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:07.808102 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.808238 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":4923,"raft.queue_time_us":1245,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4076,"spinlock_wait_cycles":335220480,"thread_start_us":26280,"threads_started":1,"wal-append.queue_time_us":930502}
I20260709 13:06:07.883033 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:07.837818 24564 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:06:07.883718 21929 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.884238 24383 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.838876 24453 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:07.885324 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1249268100000336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
W20260709 13:06:07.885705 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0985422049999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
W20260709 13:06:07.886018 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.886281 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0031031439999651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260709 13:06:07.886942 24453 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:07.887109 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:07.887480 24453 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.887929 24453 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.888223 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:07.889214 24571 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:07.890298 24571 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.890673 24571 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.891088 24571 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:07.891723 22590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 637) took 2410 ms (client timeout 1550 ms). Trace:
W20260709 13:06:07.892144 22590 rpcz_store.cc:269] 0709 13:06:05.481326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.481513 (+   187us) service_pool.cc:224] Handling call
0709 13:06:07.891525 (+2410012us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54006676d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.891652 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4328268032}
I20260709 13:06:07.876504 24559 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:06:07.849754 21927 rpcz_store.cc:276] 0709 13:06:06.108211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.111800 (+  3589us) service_pool.cc:224] Handling call
0709 13:06:06.157394 (+ 45594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.389415 (+232021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.799397 (+1409982us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54002a7454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:07.799621 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284407680}
I20260709 13:06:07.894050 24569 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:07.894500 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.843850 24428 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.850407 24530 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.869248 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:07.870342 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.903878 21934 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:07.904857 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.871726 21854 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:07.894816 24523 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:07.843458 24493 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.902561 24564 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:07.844063 24457 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:07.881237 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:07.882949 24444 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.894596 24569 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:07.867445 24173 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:07.859090 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 132782us
W20260709 13:06:07.917629 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.917582 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.898694 21935 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:07.901088 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:07.902482 24267 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:07.838760 24421 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:07.901144 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:07.906929 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:07.880077 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1947) took 2400 ms. Trace:
I20260709 13:06:07.925199 21932 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:07.925055 22006 rpcz_store.cc:276] 0709 13:06:05.479838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.330269 (+850431us) service_pool.cc:224] Handling call
0709 13:06:06.330546 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.338187 (+  7641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.338218 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.338255 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:06.338288 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.879842 (+1541554us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b540038b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:07.879886 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.880012 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17117952}
W20260709 13:06:07.880715 24543 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.685s	user 0.003s	sys 0.000s
I20260709 13:06:07.880594 24558 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.910586 24493 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:07.934294 24493 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.936008 24493 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.937748 24569 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.941395 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:07.942581 21932 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:07.855229 24557 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:07.905711 24428 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.951187 24428 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:07.929360 24560 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:06:07.914652 24517 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:07.908982 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 336) took 1417 ms (client timeout 1750 ms). Trace:
W20260709 13:06:07.881659 24478 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.041s	sys 0.063s
W20260709 13:06:07.905800 24433 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.001s
I20260709 13:06:07.918835 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:07.972141 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71169.7us
W20260709 13:06:07.972582 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53819.6us
W20260709 13:06:07.922012 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 262388us
I20260709 13:06:07.922478 21857 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:07.973817 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:07.922591 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.923717 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:07.977239 21935 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:07.903070 24453 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:07.912256 24530 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.930823 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:07.967801 21852 rpcz_store.cc:269] 0709 13:06:06.491977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.597710 (+105733us) service_pool.cc:224] Handling call
0709 13:06:07.908833 (+1311123us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54000fcb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:07.908929 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583560576}
W20260709 13:06:07.969544 24478 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.041s	sys 0.065s
I20260709 13:06:07.978329 21927 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:07.978888 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.924664 24421 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:07.979499 21927 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:07.979748 21852 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:07.980269 24421 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:07.981267 24572 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:07.981739 24572 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.982143 24572 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:07.983196 21857 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:07.908756 24570 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:06:07.986064 24262 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.987442 24426 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:07.969770 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 62920.8us
I20260709 13:06:07.990630 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:07.991375 21854 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.994880 21858 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:06:07.996404 21858 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:07.924137 24523 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.988636 24530 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:07.999159 21853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:41266
I20260709 13:06:07.999863 21853 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:08.000200 21853 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.004519 24433 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.005870 21934 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:08.006914 24433 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:08.007121 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:07.985268 21851 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:08.027778 21854 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:07.923077 24267 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:08.033159 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:08.029199 21852 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
W20260709 13:06:08.033726 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.031596 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:08.035375 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:08.035969 21858 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.036197 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:08.036748 21852 raft_consensus.cc:2376] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6. Re-sending same reply.
I20260709 13:06:08.037328 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:08.038002 22006 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.038448 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1944) took 2859 ms. Trace:
I20260709 13:06:08.028586 21853 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.040410 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:08.041304 22004 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.038800 22008 rpcz_store.cc:276] 0709 13:06:05.178804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.317401 (+1138597us) service_pool.cc:224] Handling call
0709 13:06:07.007240 (+689839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.024248 (+ 17008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.038275 (+1014027us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b5400166550. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:08.038395 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1585768832}
I20260709 13:06:08.042299 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.045961 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:08.046036 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:08.045765 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:08.065941 22002 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.017499 24428 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:08.029495 24429 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:08.093259 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57248.8us
I20260709 13:06:08.033366 24517 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.034931 24453 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:08.072127 21858 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:08.092824 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46922.1us
I20260709 13:06:08.106947 22006 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
W20260709 13:06:08.108592 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:08.110136 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.037062 24474 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.056859 24106 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:08.144250 21856 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:08.108404 24529 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:06:08.045670 24576 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:08.056661 24530 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:08.094151 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:08.091995 24444 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:07.992874 24571 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.038326 24570 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:06:08.153823 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:08.144906 24560 consensus_queue.cc:1243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:06:08.108906 24533 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:06:08.044879 24563 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:08.156883 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62848us
I20260709 13:06:08.152366 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:08.122917 22004 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:08.161432 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:08.067404 24166 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.145671 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:08.067816 24477 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:08.108242 24523 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:08.215159 24523 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:08.216214 24523 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:08.217769 24579 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:08.219552 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:08.220338 22586 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:08.221679 24540 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 13:06:08.222276 24540 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.224721 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:08.225533 22591 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:08.085623 24110 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:08.097561 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 216336us
I20260709 13:06:08.167706 22002 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:08.143362 24517 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.154894 24572 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.170025 24471 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:08.154522 24530 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.171233 21853 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:08.144476 24106 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.176645 24423 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:08.145998 24328 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:06:08.162938 24533 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.177619 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:08.263389 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 85868.3us
I20260709 13:06:08.179090 24506 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.267758 24423 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:08.268618 24506 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:08.269160 24506 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:08.269975 24506 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.270576 24530 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.139387 24581 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:08.272509 24106 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.180641 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:08.272914 24581 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.273321 24581 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.258451 24471 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:08.142429 24449 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.277362 24449 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.278045 24571 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:08.269170 24423 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.256075 24535 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:06:08.279167 21859 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:08.279448 24449 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:08.142297 24220 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.279821 24535 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:08.280328 24535 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:08.280117 24220 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.281270 24535 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.282241 24220 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.182847 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:08.254139 24540 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.184306 24563 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.286255 24517 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.287523 24563 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:08.183167 21858 raft_consensus.cc:2376] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6. Re-sending same reply.
I20260709 13:06:08.145668 24576 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.146414 24428 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:08.290047 24576 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.191247 21929 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:08.159754 24444 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:08.292140 24533 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:08.293202 21860 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:08.292917 24444 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.109163 24445 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.295308 24421 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.144657 24564 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:06:08.201619 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:08.202611 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:08.206806 24537 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.738s	user 0.007s	sys 0.003s
I20260709 13:06:08.056560 24565 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 13:06:08.161057 24426 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:08.213039 24578 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:06:08.213099 24529 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:08.227741 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 141694us
I20260709 13:06:08.256379 24110 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.251137 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:08.344270 22006 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:08.251689 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.345867 22002 raft_consensus.cc:2410] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:08.322211 24383 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.351280 24383 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:08.353677 21857 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:08.251976 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.252836 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.253391 22005 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:08.160216 24453 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:08.180779 24415 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.261662 22003 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:08.259426 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:08.371007 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.371322 22008 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:08.282639 24535 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.093366 24577 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
W20260709 13:06:08.259944 24520 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.563s	user 0.001s	sys 0.010s
I20260709 13:06:08.298977 21935 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:08.377477 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:08.378090 22587 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.299886 21926 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:08.301115 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:08.302145 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:08.290812 24428 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.318498 21929 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:08.248944 24477 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.331248 21933 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:06:08.284886 24572 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:08.338392 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:08.390211 22587 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
W20260709 13:06:08.390668 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52399.9us
W20260709 13:06:08.247874 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.373152 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:08.393594 22590 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:06:08.395069 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.400220 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:08.373142 22009 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:08.402324 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:08.342583 24426 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:08.380142 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 79383.4us
I20260709 13:06:08.383668 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:08.411010 22591 raft_consensus.cc:2410] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:06:08.384600 24453 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.385232 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:08.402983 21926 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:08.416191 21926 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:08.380102 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 79098.6us
I20260709 13:06:08.375464 24564 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:06:08.405134 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.379709 24529 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:08.381119 24538 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:08.406785 22004 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:08.411584 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:08.426339 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:08.422556 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:08.456346 21935 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:08.405831 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:08.455912 22590 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
W20260709 13:06:08.404029 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 429465us
I20260709 13:06:08.330070 24537 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:08.423177 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.456970 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:08.389174 24310 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:08.457692 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51952.1us
I20260709 13:06:08.461762 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:08.462177 22001 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:06:08.462432 21935 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.463456 22009 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:08.463600 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:08.463620 22007 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:08.466279 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:08.433284 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.417331 21926 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.474345 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:08.408987 24570 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.416669 24571 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:08.486126 21935 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:08.415705 24445 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:08.454115 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:08.457890 24512 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.494149 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:08.512220 21933 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:08.381563 24520 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.504453 22587 raft_consensus.cc:1217] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Deduplicated request from leader. Original: 1.4->[3.5-5.6]   Dedup: 3.5->[5.6-5.6]
I20260709 13:06:08.523846 22587 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:06:08.457065 24165 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.507784 21931 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
W20260709 13:06:08.494448 24555 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:06:08.467228 24572 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:08.416361 24533 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:06:08.489800 24558 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.494069 24429 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:08.523351 24520 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.546355 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.552932 22000 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:08.551633 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.549909 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.551960 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:08.576071 22006 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.550977 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.577812 22587 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 6. Ops: []
I20260709 13:06:08.554809 22009 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:08.577733 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.575030 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:08.579181 22001 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:08.579613 22000 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
I20260709 13:06:08.580987 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:08.581861 22000 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.525017 22584 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.489499 24262 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:06:08.589186 22001 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:08.591626 22001 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:06:08.580441 22007 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:06:08.597116 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.457410 24564 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.494621 24364 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:06:08.532954 24423 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:08.585759 24550 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:06:08.541646 24429 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:08.499732 24569 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:06:08.508919 24588 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:06:08.528687 24310 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:06:08.616139 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:08.529633 24512 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.635483 22000 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:08.597307 24572 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.646054 22006 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:08.537704 24165 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:08.645599 22009 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:08.649868 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:08.638377 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.618098 24364 consensus_queue.cc:1243] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:06:08.609547 24585 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:08.632468 24429 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.610371 24558 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:08.618166 24538 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:08.666347 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 192089us
I20260709 13:06:08.670032 22587 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:06:08.671516 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.673079 24477 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:08.662923 24585 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:08.674144 24110 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:08.656680 24572 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:08.676303 24594 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:06:08.681191 24415 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:08.683779 24571 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:08.611922 24529 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:08.685719 24595 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:06:08.688090 24512 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:08.672880 24575 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.551628 24426 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.614878 24423 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:08.730515 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:09.122464 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.047493 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 342307us
I20260709 13:06:09.158883 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:08.758843 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.047955 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:09.164196 22007 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:08.731441 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:09.168004 22556 connection.cc:256] RPC call timeout handler was delayed by -1.3519302319999724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
W20260709 13:06:08.900460 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 436288us
I20260709 13:06:08.664557 22586 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:09.047791 24471 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.773s	user 0.008s	sys 0.003s
I20260709 13:06:09.066082 24556 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:06:08.747166 24493 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.008s
W20260709 13:06:08.999794 24603 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:06:09.210672 24477 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.210997 24588 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:09.307297 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 349) took 1515 ms. Trace:
I20260709 13:06:09.225561 24445 raft_consensus.cc:2764] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:09.343937 21856 rpcz_store.cc:276] 0709 13:06:07.791482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.806243 (+ 14761us) service_pool.cc:224] Handling call
0709 13:06:09.307045 (+1500802us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54000596d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:09.307213 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661229312}
I20260709 13:06:09.267634 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 664) took 1078 ms. Trace:
I20260709 13:06:08.703244 24428 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:09.357579 24512 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.211087 24575 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.357661 24423 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:09.371402 22002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.011s	sys 0.024s
W20260709 13:06:09.410449 22002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.011s	sys 0.024s
I20260709 13:06:08.685863 24558 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.278563 24415 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.292361 24556 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:08.907683 24537 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:09.339254 24608 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.338593 24588 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:06:09.443032 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1648) took 2411 ms (client timeout 2809 ms). Trace:
I20260709 13:06:09.385213 24428 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:09.420776 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 339) took 2904 ms. Trace:
I20260709 13:06:09.448196 21855 rpcz_store.cc:276] 0709 13:06:06.516779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.987738 (+470959us) service_pool.cc:224] Handling call
0709 13:06:06.988018 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.045255 (+ 57237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.420438 (+2375183us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b540008b1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:09.420685 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130241408}
I20260709 13:06:09.351382 24533 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.283692 24471 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.350973 24383 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:09.443500 21930 rpcz_store.cc:269] 0709 13:06:07.031222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.033174 (+  1952us) service_pool.cc:224] Handling call
0709 13:06:09.442775 (+2409601us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54003075d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:09.442951 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54513920}
W20260709 13:06:09.424242 24364 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.012s
I20260709 13:06:09.213948 24538 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.411557 24512 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.382782 22586 rpcz_store.cc:276] 0709 13:06:08.188606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.189380 (+   774us) service_pool.cc:224] Handling call
0709 13:06:09.226327 (+1036947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:09.460326 21855 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:09.464299 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.446166 24588 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.783402 24581 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.495409 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1995) took 1196 ms. Trace:
I20260709 13:06:09.500360 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1916) took 1580 ms. Trace:
I20260709 13:06:09.351605 24477 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.501924 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:09.269912 24585 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.504101 21852 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:09.467708 24576 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.177s	user 0.013s	sys 0.000s
I20260709 13:06:09.510131 21855 rpcz_store.cc:276] 0709 13:06:07.919965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.459806 (+1539841us) service_pool.cc:224] Handling call
0709 13:06:09.460160 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.500096 (+ 39936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.500302 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:09.492730 24364 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.006s	sys 0.012s
I20260709 13:06:09.508431 21932 rpcz_store.cc:276] 0709 13:06:08.298771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.298900 (+   129us) service_pool.cc:224] Handling call
0709 13:06:08.311263 (+ 12363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.319170 (+  7907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.319220 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.336494 (+ 17274us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.532972 (+196478us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:08.533043 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.533120 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.495098 (+961978us) raft_consensus.cc:1685] finished
0709 13:06:09.495128 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.495330 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1503,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":14755,"thread_start_us":129342,"threads_started":1,"wal-append.queue_time_us":301334}
I20260709 13:06:09.468679 24165 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:09.536141 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:09.502658 24423 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:09.532423 22000 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:06:09.424122 24415 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.532976 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:09.541558 22009 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:06:09.535398 22001 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:09.537015 22006 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:09.507738 24611 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.710s	user 0.000s	sys 0.001s
I20260709 13:06:09.549918 24575 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:09.551221 22593 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:09.469573 24533 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:09.571815 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.017s	sys 0.048s
W20260709 13:06:09.572944 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.017s	sys 0.048s
I20260709 13:06:09.529271 24523 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 13:06:09.578727 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.534232 24537 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:06:09.557330 24585 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:09.501125 24383 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:09.595259 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.599366 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.457690 24608 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.534744 24576 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.629549 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 342) took 3024 ms. Trace:
W20260709 13:06:09.626665 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.000s	sys 0.014s
I20260709 13:06:09.633291 21860 rpcz_store.cc:276] 0709 13:06:06.605670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.680699 (+1075029us) service_pool.cc:224] Handling call
0709 13:06:07.680921 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.293609 (+612688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.629233 (+1335624us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:09.629475 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407643904}
I20260709 13:06:09.512449 24556 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.641906 24556 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:09.645443 24429 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.942s	user 0.010s	sys 0.002s
I20260709 13:06:09.647320 24429 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.631505 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:09.651219 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2002) took 1222 ms. Trace:
I20260709 13:06:09.661643 21858 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:09.658201 21853 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:09.636525 24512 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:09.661341 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1710) took 2592 ms (client timeout 2181 ms). Trace:
I20260709 13:06:09.660980 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1970) took 1107 ms. Trace:
I20260709 13:06:09.552626 24423 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:09.661819 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 452) took 1893 ms (client timeout 2187 ms). Trace:
W20260709 13:06:09.635185 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.000s	sys 0.015s
W20260709 13:06:09.666692 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.666464 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:09.671056 21934 rpcz_store.cc:269] 0709 13:06:07.768043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.904603 (+136560us) service_pool.cc:224] Handling call
0709 13:06:09.661646 (+1757043us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400207954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:09.661765 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230637952}
I20260709 13:06:09.597036 24572 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:09.668082 22002 rpcz_store.cc:276] 0709 13:06:08.554684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.583786 (+ 29102us) service_pool.cc:224] Handling call
0709 13:06:08.584163 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.584580 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.584617 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.600883 (+ 16266us) log.cc:844] Serialized 26 byte log entry
0709 13:06:09.411026 (+810143us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:09.411086 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.411185 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.660769 (+249584us) raft_consensus.cc:1685] finished
0709 13:06:09.660789 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.660924 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1779,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":21681,"thread_start_us":763312,"threads_started":1,"wal-append.queue_time_us":1041567}
I20260709 13:06:09.666220 21929 rpcz_store.cc:276] 0709 13:06:08.428856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.508088 (+ 79232us) service_pool.cc:224] Handling call
0709 13:06:08.535990 (+ 27902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.536452 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.536492 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.538810 (+  2318us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.792279 (+253469us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:08.792336 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.792427 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.650917 (+858490us) raft_consensus.cc:1685] finished
0709 13:06:09.650948 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.651134 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":784,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":29710,"thread_start_us":181092,"threads_started":1,"wal-append.queue_time_us":1042297}
W20260709 13:06:09.668006 21859 rpcz_store.cc:269] 0709 13:06:07.069113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.603423 (+534310us) service_pool.cc:224] Handling call
0709 13:06:09.661163 (+2057740us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54000591d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:09.661275 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536059264}
I20260709 13:06:09.676532 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:09.681784 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:09.674187 22000 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:09.628899 24474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.000s	sys 0.014s
W20260709 13:06:09.653367 24613 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 13:06:09.648901 24535 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.267s	user 0.008s	sys 0.001s
I20260709 13:06:09.691509 24623 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:09.695341 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:09.557498 24428 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.693863 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:09.656167 24328 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.660269 24424 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:09.671556 24421 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.358s	user 0.012s	sys 0.002s
I20260709 13:06:09.699940 22008 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:06:09.698966 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:09.657912 24576 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:09.654799 24608 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:09.720077 21859 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:06:09.720204 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 283847us
I20260709 13:06:09.717998 22008 pending_rounds.cc:85] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:06:09.684382 24559 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:09.720935 22008 pending_rounds.cc:107] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 5.7
W20260709 13:06:09.616858 22584 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.999s	user 0.006s	sys 0.002s
I20260709 13:06:09.721372 22008 raft_consensus.cc:2889] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:09.672350 24625 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:09.723609 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:09.719430 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 187834us
W20260709 13:06:09.725644 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 217418us
I20260709 13:06:09.707058 24429 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:09.735038 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.717780 24615 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.722987 24559 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.719471 24614 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:09.707129 24623 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:09.723172 24428 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:09.783649 24559 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:09.722987 24610 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.804809 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:09.722239 24328 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:09.746740 24535 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.806034 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:09.752388 24421 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.749796 24471 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:09.723397 22584 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:09.748631 24630 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:09.815322 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1980) took 1049 ms. Trace:
I20260709 13:06:09.781184 24625 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:10.031407 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 307829us
I20260709 13:06:09.814162 24608 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:09.810467 24477 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:09.854491 24512 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:10.034925 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:09.814316 24576 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:10.052048 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.058334 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.045112 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1914) took 2126 ms. Trace:
I20260709 13:06:10.011577 24630 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.814828 24086 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:10.012049 24569 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.026391 22584 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 696) took 1640 ms (client timeout 1743 ms). Trace:
I20260709 13:06:09.783063 24615 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:10.066139 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.032601 24415 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.011335 24310 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.202s
I20260709 13:06:10.074146 21856 rpcz_store.cc:276] 0709 13:06:07.919447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.348212 (+1428765us) service_pool.cc:224] Handling call
0709 13:06:09.442957 (+ 94745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.443172 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:09.707546 (+264374us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.044367 (+336821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:10.044443 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:10.044505 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.044602 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.044882 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14740,"prepare.queue_time_us":219254,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":6359,"raft.queue_time_us":22122,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":4549,"thread_start_us":580534,"threads_started":2}
I20260709 13:06:10.022830 24624 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:09.854987 24535 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.011539 24421 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:10.082773 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.049993 24608 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.082607 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 693) took 1705 ms. Trace:
I20260709 13:06:10.057308 24512 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:10.081573 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 260362us
I20260709 13:06:09.814397 24610 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.113410 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.114717 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:10.114974 21932 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:06:10.061190 24533 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:10.117858 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.098839 24471 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:10.109623 22592 rpcz_store.cc:276] 0709 13:06:08.377455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.383616 (+  6161us) service_pool.cc:224] Handling call
0709 13:06:08.383936 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.384373 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:08.478942 (+ 94569us) log.cc:844] Serialized 14 byte log entry
0709 13:06:08.653204 (+174262us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.653718 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.653867 (+   149us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:08.654144 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 13:06:08.655609 (+  1465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.655679 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.082385 (+1426706us) raft_consensus.cc:1685] finished
0709 13:06:10.082411 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.082550 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1259,"prepare.queue_time_us":223674,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":9243,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":11966,"spinlock_wait_cycles":34915456,"thread_start_us":205161,"threads_started":2,"wal-append.queue_time_us":305056}
W20260709 13:06:10.119331 24627 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.307s	user 0.000s	sys 0.000s
I20260709 13:06:10.077608 24444 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:06:10.080284 24569 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.080072 24086 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.330802 21930 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:10.074447 22584 rpcz_store.cc:269] 0709 13:06:08.385828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.389726 (+  3898us) service_pool.cc:224] Handling call
0709 13:06:10.021497 (+1631771us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:10.092918 24629 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.281s	user 0.000s	sys 0.000s
W20260709 13:06:10.255601 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.288811 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1717) took 2617 ms. Trace:
W20260709 13:06:10.333012 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 274230us
I20260709 13:06:10.075593 24630 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.334388 21934 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:06:10.278254 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.080508 24383 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.195169 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:10.365151 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:10.270562 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:10.366528 21858 rpcz_store.cc:276] 0709 13:06:07.671717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.289465 (+617748us) service_pool.cc:224] Handling call
0709 13:06:09.661284 (+1371819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.666292 (+  5008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.288512 (+622220us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000591d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:10.288729 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3154020608}
W20260709 13:06:10.287336 24637 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:06:10.286427 24639 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 13:06:10.329663 24638 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:06:10.097208 24623 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:10.110522 24423 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:10.126302 24512 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:10.094041 24328 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:10.361881 24426 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.328605 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1989) took 2487 ms. Trace:
I20260709 13:06:10.089742 24626 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.284s
I20260709 13:06:10.111261 24608 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:10.375239 21935 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:10.369894 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.094139 24635 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:10.182114 24572 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.199s
I20260709 13:06:10.285058 24592 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.405714 24535 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.372493 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:10.380844 24623 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.378240 24383 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:10.099450 24624 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.375717 24533 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.093672 24625 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:10.400065 24642 raft_consensus.cc:670] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } [suppressed 1 similar messages]
I20260709 13:06:10.125483 24477 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.342321 24471 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.388181 21930 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:10.407469 24592 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.411276 24643 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:06:10.413421 24615 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:10.416548 24421 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:10.394723 21927 rpcz_store.cc:276] 0709 13:06:07.841417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.980344 (+138927us) service_pool.cc:224] Handling call
0709 13:06:07.980680 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.039399 (+ 58719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.045280 (+  5881us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.157985 (+112705us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.475695 (+317710us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:08.493155 (+ 17460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.498291 (+  5136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.192758 (+1694467us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b24001bef18. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d720fb0 00007f3b70397488 00007f3b70397be8 00007f3b7036f398 00007f3b7036a76a 00007f3b70369bb6
0709 13:06:10.314094 (+121336us) raft_consensus.cc:1685] finished
0709 13:06:10.314122 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.328512 (+ 14390us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66374,"raft.queue_time_us":56496,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":21061,"spinlock_wait_cycles":50760960,"thread_start_us":294648,"threads_started":1,"wal-append.queue_time_us":350619}
W20260709 13:06:10.419974 22003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 1942) took 2383 ms (client timeout 1593 ms). Trace:
I20260709 13:06:10.389899 24086 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:10.551441 24556 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.909s	user 0.013s	sys 0.005s
I20260709 13:06:10.403275 24635 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.428944 21858 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:10.430372 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1714) took 3117 ms. Trace:
I20260709 13:06:10.457841 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 450) took 2707 ms. Trace:
I20260709 13:06:10.556413 21931 rpcz_store.cc:276] 0709 13:06:07.750508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.752015 (+  1507us) service_pool.cc:224] Handling call
0709 13:06:07.776778 (+ 24763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.531465 (+754687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.457617 (+1926152us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400429ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:10.457779 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678417280}
I20260709 13:06:09.863730 22007 rpcz_store.cc:276] 0709 13:06:08.766046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.464827 (+698781us) service_pool.cc:224] Handling call
0709 13:06:09.465393 (+   566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.465660 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:09.720071 (+254411us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.814946 (+ 94875us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.815017 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:09.815066 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.815137 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.815251 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":311882,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":3562,"thread_start_us":306325,"threads_started":2}
I20260709 13:06:10.456292 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 537) took 2206 ms. Trace:
I20260709 13:06:10.484853 24471 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:10.431288 24383 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:10.437683 24591 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 2.059s	user 0.001s	sys 0.015s
I20260709 13:06:10.458266 24512 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.485792 24477 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.431648 24623 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.486022 24643 consensus_queue.cc:1243] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:06:10.451794 24576 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:10.457096 24559 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.000s
I20260709 13:06:10.493261 24533 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.396538 24426 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.430610 24535 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.484287 24630 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:10.550133 22003 rpcz_store.cc:269] 0709 13:06:08.036153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.041971 (+  5818us) service_pool.cc:224] Handling call
0709 13:06:10.419559 (+2377588us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540042c2d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:10.419880 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502628608}
I20260709 13:06:10.425995 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 567) took 1915 ms. Trace:
W20260709 13:06:10.554508 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.584618 22590 rpcz_store.cc:276] 0709 13:06:08.510873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.511135 (+   262us) service_pool.cc:224] Handling call
0709 13:06:08.511544 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.633116 (+121572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.633164 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:08.633239 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:08.633292 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.425738 (+1792446us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540055ffd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:10.425789 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.425925 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278882048}
I20260709 13:06:10.555738 21857 rpcz_store.cc:276] 0709 13:06:07.312528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.983951 (+671423us) service_pool.cc:224] Handling call
0709 13:06:07.984227 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.354253 (+370026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.430096 (+2075843us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54003d3954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:10.430299 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849023872}
I20260709 13:06:10.586484 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:10.558657 22004 rpcz_store.cc:276] 0709 13:06:08.250111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.250965 (+   854us) service_pool.cc:224] Handling call
0709 13:06:08.251348 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.455693 (+204345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.456031 (+2000338us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400425ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:10.456229 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356785920}
I20260709 13:06:10.560825 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:10.587069 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.554672 24428 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.747s	user 0.010s	sys 0.009s
W20260709 13:06:10.552057 24588 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.025s	user 0.009s	sys 0.012s
I20260709 13:06:10.587518 24383 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.588383 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.589035 22004 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:10.588486 24588 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.590031 22587 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:10.590044 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:10.590997 22004 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.592473 24588 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:10.555929 24585 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.932s	user 0.014s	sys 0.000s
I20260709 13:06:10.555343 24086 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.551994 24648 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.618110 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.578843 21858 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
W20260709 13:06:10.583312 24662 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:06:10.586663 21857 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [3.5-5.6]
W20260709 13:06:10.454743 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 341) took 3857 ms (client timeout 2202 ms). Trace:
W20260709 13:06:10.621097 21851 rpcz_store.cc:269] 0709 13:06:06.597218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.931835 (+334617us) service_pool.cc:224] Handling call
0709 13:06:10.454128 (+3522293us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540010ca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:10.454575 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420235136}
I20260709 13:06:10.587441 22007 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:10.604071 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:10.572561 24495 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:10.626045 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.622754 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.624203 21858 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:10.627738 22007 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:06:10.624181 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:10.631721 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.632304 22007 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:10.633157 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.633693 22007 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:10.626232 24495 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:10.593443 24535 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.636979 24658 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:10.637606 24588 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:10.636545 24495 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.638276 24572 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:10.638203 24535 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.639158 24572 consensus_queue.cc:1243] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:10.637746 24658 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:10.599041 24533 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.640479 24658 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.599946 24621 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:10.581238 24559 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.600162 24635 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:10.599727 24477 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.628834 24648 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.645849 22004 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:10.679895 22004 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:10.636991 21927 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:10.599973 24234 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:06:10.599004 24512 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.629788 24585 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:10.676429 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:10.667549 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.685248 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.668273 24478 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:10.691068 21931 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:10.694453 24585 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:10.694900 24630 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.695329 24536 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:10.695873 24624 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:10.695508 24585 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.698167 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.687516 24426 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.699257 21930 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:10.646400 24086 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.707466 24576 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.708182 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.712498 21935 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:10.716565 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.718366 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:10.712883 24557 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:10.717710 22004 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:10.691906 24478 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:10.722280 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:10.723076 22004 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:10.741055 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.726919 24592 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.751418 24614 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:10.769198 24559 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:10.769989 24660 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:06:10.766259 24614 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.770874 24614 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:10.771838 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1716) took 3100 ms. Trace:
I20260709 13:06:10.772003 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:10.772560 22005 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.773118 24656 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:10.773687 24656 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.775100 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:10.775823 22003 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:10.776751 22004 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:10.772221 21853 rpcz_store.cc:276] 0709 13:06:07.671449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.273851 (+602402us) service_pool.cc:224] Handling call
0709 13:06:08.274125 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.666417 (+1392292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.771627 (+1105210us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540018ecd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:10.771786 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3182512512}
I20260709 13:06:10.784648 22005 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:10.788718 22005 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:10.721359 24576 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.754434 24556 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:06:10.795868 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.767251 24623 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:10.799724 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:10.800303 21933 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:10.778296 24608 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.807312 24648 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.799105 24474 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.825169 24656 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:10.825487 24086 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:10.826399 24608 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:10.827556 24600 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 2.101s	user 0.012s	sys 0.005s
I20260709 13:06:10.827916 24262 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.828822 24262 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.830750 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.831041 24576 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:10.799626 24478 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.832132 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:10.832616 24478 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:10.844458 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1957) took 1031 ms. Trace:
I20260709 13:06:10.845112 22007 rpcz_store.cc:276] 0709 13:06:09.813434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.634214 (+820780us) service_pool.cc:224] Handling call
0709 13:06:10.634485 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.634823 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.634851 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.671936 (+ 37085us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.741962 (+ 70026us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:10.742024 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.742105 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.844140 (+102035us) raft_consensus.cc:1685] finished
0709 13:06:10.844167 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.844374 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":18790,"thread_start_us":52519,"threads_started":1,"wal-append.queue_time_us":86163}
I20260709 13:06:10.849077 24471 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:10.850811 21856 raft_consensus.cc:2368] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6.
W20260709 13:06:10.853269 21974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 2.152s (SENT) [suppressed 17 similar messages]
I20260709 13:06:10.835520 24474 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:10.856248 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:10.859192 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:10.859886 21858 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:06:10.862075 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.878962 24477 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.837045 24663 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.160s	user 0.000s	sys 0.002s
I20260709 13:06:10.879647 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:11.085996 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:10.931161 22592 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:11.054427 21933 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:11.035708 21851 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:11.128610 22006 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:11.128980 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:10.964790 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.089479 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.130903 22591 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:11.029757 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.082440 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 2.894e+06us
I20260709 13:06:11.136106 22592 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:10.929214 22590 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:10.921924 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 1992) took 3079 ms. Trace:
I20260709 13:06:11.082069 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:11.071688 24536 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.138267 21928 rpcz_store.cc:276] 0709 13:06:07.842303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.183539 (+341236us) service_pool.cc:224] Handling call
0709 13:06:09.444279 (+1260740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.468133 (+ 23854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.468164 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.196903 (+728739us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:06:10.197440 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.365187 (+167747us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:10.365226 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.365287 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.455587 (+ 90300us) raft_consensus.cc:1685] finished
0709 13:06:10.455656 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.921669 (+466013us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003075d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:10.921845 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23335,"raft.queue_time_us":54525,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":28823,"spinlock_wait_cycles":2906412672,"thread_start_us":167228,"threads_started":1,"wal-append.queue_time_us":254551}
I20260709 13:06:10.880046 24648 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.143507 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.143970 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.145097 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:10.879607 24477 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.145359 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:11.140983 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:11.149431 22557 connection.cc:256] RPC call timeout handler was delayed by -1.1073429159999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
I20260709 13:06:11.089035 22584 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:10.930425 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:11.154121 21972 connection.cc:256] RPC call timeout handler was delayed by -2.4679979349999712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:06:11.154325 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:10.928169 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:10.960244 24559 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:11.154685 22557 connection.cc:256] RPC call timeout handler was delayed by -1.09294291599997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
I20260709 13:06:11.155165 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.155386 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.153344 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.147508 24536 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.159637 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:11.160429 21972 connection.cc:256] RPC call timeout handler was delayed by -2.4490979350000543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 13:06:11.141793 24292 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.153324 24426 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:11.125972 24635 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:06:11.165335 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:11.165658 22591 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:06:11.167727 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:11.169402 21858 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:11.208598 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:11.208851 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.209998 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:11.210131 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:11.210754 21858 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.151288 24570 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.151806 24608 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.171290 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:11.161947 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.176259 24536 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:11.231326 22589 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:11.177976 21972 connection.cc:256] RPC call timeout handler was delayed by -2.3527979350000123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 13:06:11.177346 22586 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 574) took 1649 ms (client timeout 2152 ms). Trace:
I20260709 13:06:11.179028 24328 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.167927 24559 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:11.232986 22586 rpcz_store.cc:269] 0709 13:06:09.527784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.927842 (+1400058us) service_pool.cc:224] Handling call
0709 13:06:11.177250 (+249408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.152773 24648 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.183509 24477 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:11.187952 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.188982 24602 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 2.365s	user 0.004s	sys 0.007s
I20260709 13:06:11.192515 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 572) took 2408 ms. Trace:
I20260709 13:06:11.193883 24624 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:11.194756 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:11.166617 24673 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:06:11.211094 22557 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:11.231685 21853 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:11.232359 22006 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:11.178186 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:11.232882 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.232944 21972 connection.cc:256] RPC call timeout handler was delayed by -2.2717979349999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
I20260709 13:06:11.169365 24471 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:11.212110 22585 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:11.294678 21972 connection.cc:256] RPC call timeout handler was delayed by -2.22319793500003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
I20260709 13:06:11.212518 24445 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.237648 22587 rpcz_store.cc:276] 0709 13:06:08.784554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.492537 (+707983us) service_pool.cc:224] Handling call
0709 13:06:09.492861 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.590439 (+1097578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.192233 (+601794us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54005631d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:11.192442 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2522752768}
W20260709 13:06:11.296293 21972 connection.cc:256] RPC call timeout handler was delayed by -2.03059793500006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
W20260709 13:06:11.305164 21972 connection.cc:256] RPC call timeout handler was delayed by -1.9342979350000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
W20260709 13:06:11.305742 21972 connection.cc:256] RPC call timeout handler was delayed by -1.8694979350000267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21810000000000002s for a response
W20260709 13:06:11.306124 21972 connection.cc:256] RPC call timeout handler was delayed by -1.832597934999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
W20260709 13:06:11.306435 21972 connection.cc:256] RPC call timeout handler was delayed by -1.3042979350000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28090000000000004s for a response
I20260709 13:06:11.307039 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:11.309264 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:11.212371 24429 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:11.230687 24493 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:11.304370 24676 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:06:11.316385 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 105334us
I20260709 13:06:11.335625 21858 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:11.317916 24656 raft_consensus.cc:2764] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:11.317574 24477 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:11.203279 24292 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.355350 24445 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:11.357439 22589 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:11.415447 21856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 359) took 1689 ms (client timeout 1934 ms). Trace:
I20260709 13:06:11.362983 24575 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.318970 24493 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.317436 24536 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:06:11.320218 22585 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 694) took 2941 ms (client timeout 2134 ms). Trace:
I20260709 13:06:11.395830 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.424714 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:11.426070 21854 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:11.213534 24426 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.236855 24608 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.237915 24328 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.234205 24570 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.314335 24624 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:11.410342 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 273328us
I20260709 13:06:11.459905 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:11.462152 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 128033us
I20260709 13:06:11.464314 22588 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:11.465137 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:11.353894 21900 connection.cc:256] RPC call timeout handler was delayed by -1.1718127749999212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
I20260709 13:06:11.466881 22587 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:11.372115 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:11.415971 21856 rpcz_store.cc:269] 0709 13:06:09.725866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.187883 (+462017us) service_pool.cc:224] Handling call
0709 13:06:11.388055 (+1200172us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540028c654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:11.415379 (+ 27324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105579904}
I20260709 13:06:11.474116 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:11.427417 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:11.423816 24493 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:11.430089 24675 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:06:11.430435 24536 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.419028 24477 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:11.434414 24614 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.004s
I20260709 13:06:11.423249 24610 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.435539 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1987) took 1037 ms. Trace:
I20260709 13:06:11.411247 24635 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:11.412140 24529 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:11.438882 24648 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:11.440061 24559 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.440855 24457 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:11.479331 24426 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.446309 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:11.469180 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.494491 21932 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:11.463505 22006 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:11.470350 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.471555 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:11.474860 21855 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:11.475190 21933 raft_consensus.cc:2410] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:06:11.407290 24267 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:11.441431 22585 rpcz_store.cc:269] 0709 13:06:08.378725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.380920 (+  2195us) service_pool.cc:224] Handling call
0709 13:06:10.930269 (+2549349us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:11.317289 (+387020us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10615552}
I20260709 13:06:11.484221 24614 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.499696 24267 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:11.499924 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.500864 22585 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:11.501160 24635 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:11.501927 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.502179 24635 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:11.502652 24635 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:11.476043 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.497612 24506 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.503654 24506 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.504156 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:11.503759 24635 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.505366 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.477727 24656 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:11.478163 24478 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.002s
I20260709 13:06:11.478804 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.506542 24685 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.506923 21931 raft_consensus.cc:2368] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
I20260709 13:06:11.507682 24328 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:11.507845 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:11.478895 24624 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:11.508527 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.509974 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.511077 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.511884 22587 raft_consensus.cc:2410] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:11.512633 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.479965 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.431263 24575 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.513325 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.514066 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:11.514473 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 354) took 1842 ms (client timeout 1805 ms). Trace:
I20260709 13:06:11.514676 22009 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:06:11.513325 21934 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
W20260709 13:06:11.514936 21855 rpcz_store.cc:269] 0709 13:06:09.672404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.672522 (+   118us) service_pool.cc:224] Handling call
0709 13:06:11.371874 (+1699352us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:11.514401 (+142527us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22054272}
I20260709 13:06:11.516085 24575 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:11.483964 22004 rpcz_store.cc:276] 0709 13:06:10.398126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.779485 (+381359us) service_pool.cc:224] Handling call
0709 13:06:10.779865 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.781763 (+  1898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.781791 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.782497 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.987414 (+204917us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:10.987474 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.987570 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.435277 (+447707us) raft_consensus.cc:1685] finished
0709 13:06:11.435299 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.435473 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1355,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":204441,"threads_started":1,"wal-append.queue_time_us":429061}
I20260709 13:06:11.517095 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.490015 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.444172 21851 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:11.482789 24477 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:11.481385 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.521571 21928 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:11.486186 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.522760 22584 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 3.
I20260709 13:06:11.523144 24559 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.523972 24559 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.527627 24559 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:11.493383 24529 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:11.534198 24529 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.535622 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:11.537992 24608 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:11.539988 24495 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:11.498226 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:11.506225 24656 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:11.541504 22007 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:11.541769 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:11.541913 24656 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.458150 24429 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.493844 24457 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:11.502682 22585 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:11.496738 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:11.540799 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.517059 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:11.552312 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.553289 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 353) took 1881 ms (client timeout 2426 ms). Trace:
I20260709 13:06:11.556149 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.547621 22556 leader_election.cc:400] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6.
I20260709 13:06:11.573865 22556 leader_election.cc:403] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.574518 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.575567 22587 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:11.576642 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:11.578830 21825 connection.cc:256] RPC call timeout handler was delayed by -1.2343655180000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3205s for a response
I20260709 13:06:11.562343 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1964) took 1015 ms. Trace:
I20260709 13:06:11.528016 24292 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.591866 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.595916 24457 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.486159 24536 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.586364 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.588034 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.621901 21930 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:06:11.573105 21860 rpcz_store.cc:269] 0709 13:06:09.672113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.684608 (+ 12495us) service_pool.cc:224] Handling call
0709 13:06:11.541604 (+1856996us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:11.553176 (+ 11572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38144896}
I20260709 13:06:11.599908 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:11.514861 24570 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.625782 22004 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:11.583395 22007 rpcz_store.cc:276] 0709 13:06:10.547285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.542115 (+994830us) service_pool.cc:224] Handling call
0709 13:06:11.561626 (+ 19511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.561842 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.561867 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.561913 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:11.561946 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.562008 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.562141 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.599602 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.608366 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:11.588361 24571 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.629624 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:11.633473 22004 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:11.633991 22004 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:11.634275 24426 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:11.630765 24471 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:11.634964 22004 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.636325 24471 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:11.636153 22004 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:11.637472 24471 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:11.640362 24536 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:11.611477 24614 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.644459 22002 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:11.589670 24656 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:11.610594 22009 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:11.609680 22592 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:11.613703 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.651471 24642 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:11.652483 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.618983 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.572983 24635 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.562047 24648 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:11.656368 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:11.673920 21931 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:11.663648 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.667403 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:11.656836 22009 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:11.676091 24570 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:11.679040 24493 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.676765 24684 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:11.681249 24684 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:11.681960 24684 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:11.682950 24684 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.684418 24684 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:11.695009 24679 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.494s	user 0.000s	sys 0.002s
I20260709 13:06:11.696961 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.697777 22586 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:11.698776 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:11.577792 24429 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:11.699337 22586 raft_consensus.cc:2376] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8. Re-sending same reply.
I20260709 13:06:11.700596 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.701426 24429 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.702122 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.702508 22589 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:11.703112 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:11.705569 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1994) took 1280 ms. Trace:
I20260709 13:06:11.706154 22005 rpcz_store.cc:276] 0709 13:06:10.425760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.796977 (+371217us) service_pool.cc:224] Handling call
0709 13:06:10.797297 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.800077 (+  2780us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:10.800114 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.806820 (+  6706us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.988045 (+181225us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:10.988093 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.988174 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.694256 (+706082us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:11.705270 (+ 11014us) raft_consensus.cc:1685] finished
0709 13:06:11.705292 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.705502 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":19212,"spinlock_wait_cycles":22790016,"thread_start_us":146179,"threads_started":1,"wal-append.queue_time_us":307480}
I20260709 13:06:11.660281 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:11.713323 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:11.716668 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:11.615976 24457 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.652773 24642 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:11.720819 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60762.5us
I20260709 13:06:11.724449 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:11.703138 22586 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:11.729966 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:11.718776 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 409564us
I20260709 13:06:11.673620 24648 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:11.736259 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:11.737341 22007 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:11.746950 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:11.714478 24548 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:11.752785 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:11.720373 24250 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:11.781862 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:11.627452 24477 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:11.793362 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.670837 24426 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:06:11.794333 24328 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:11.794404 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.806833 22586 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:11.799474 24474 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:11.743685 24560 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:11.714200 24625 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:06:11.743532 24624 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:11.735240 24608 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:11.854305 21931 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:11.744860 24648 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:11.745154 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:11.779582 24548 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.779209 24536 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:11.826310 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 366617us
W20260709 13:06:11.836584 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 106731us
I20260709 13:06:11.855710 24426 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:11.854782 24560 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:11.885486 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.888984 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.744592 24557 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:06:11.909068 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:11.910136 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:11.918103 22590 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:11.793938 24656 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:06:11.814455 24691 raft_consensus.cc:2764] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:11.824545 24624 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:11.928346 24624 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.922832 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.935534 22586 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:11.923615 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.905524 24687 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:11.935575 24529 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:11.816094 24328 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:06:11.907682 24548 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:11.943943 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 364266us
W20260709 13:06:11.951143 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.954788 21825 connection.cc:256] RPC call timeout handler was delayed by -1.1578655179999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.329s for a response
I20260709 13:06:11.956571 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:11.957509 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 175763us
I20260709 13:06:11.956545 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:11.958479 22586 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:11.945299 21927 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:11.955494 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 214251us
W20260709 13:06:11.951169 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 177237us
I20260709 13:06:11.974747 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:11.988421 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.989590 21972 leader_election.cc:400] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:06:11.985945 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:12.011915 21972 leader_election.cc:403] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:12.015578 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:12.017372 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 98405.4us
W20260709 13:06:12.017850 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 109686us
I20260709 13:06:12.018898 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:11.956969 24529 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:06:11.977016 24641 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:12.020046 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:12.042621 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.012418 24250 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:12.012662 24471 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.965574 24536 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:06:12.017998 24691 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6.
I20260709 13:06:12.043322 24581 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:12.090806 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75339.4us
I20260709 13:06:12.093417 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:12.056581 24477 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:12.021499 24560 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.054374 24684 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.060050 24328 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:12.061972 24692 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:12.133440 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.197149 22009 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:06:12.198772 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 105425us
W20260709 13:06:12.136946 24712 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.206s	user 0.000s	sys 0.003s
W20260709 13:06:12.174050 24571 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.004s
I20260709 13:06:12.210891 24571 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6.
I20260709 13:06:12.085364 24429 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.166497 24715 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 13:06:12.208226 24664 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.582s	user 0.006s	sys 0.000s
I20260709 13:06:12.208513 24687 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:12.102747 24685 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 13:06:12.215968 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:12.130575 24421 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:12.087903 24608 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:12.229416 24635 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.575s	user 0.012s	sys 0.004s
W20260709 13:06:12.229549 24495 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.689s	user 0.011s	sys 0.006s
I20260709 13:06:12.196005 24684 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:12.137878 24250 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.200899 24692 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:12.123373 24564 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.196338 24714 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:12.200459 24328 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:12.232762 24457 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.000s
I20260709 13:06:12.258505 24477 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:12.263585 24718 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:06:12.251932 24421 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:12.265522 24642 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.265797 24262 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.201097 24720 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:12.250605 24685 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:12.268605 24684 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.299297 24624 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.269990 24692 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:12.302387 24548 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.233671 24429 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:12.281783 24719 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 13:06:12.285698 24608 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.272090 24714 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.276134 24477 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:12.310284 24477 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.292912 24648 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:12.293928 22593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 663) took 4108 ms (client timeout 2188 ms). Trace:
W20260709 13:06:12.302881 24689 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.400s	user 0.002s	sys 0.000s
W20260709 13:06:12.317358 22593 rpcz_store.cc:269] 0709 13:06:08.185557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.185680 (+   123us) service_pool.cc:224] Handling call
0709 13:06:12.293705 (+4108025us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54006009d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.293856 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2840378112}
I20260709 13:06:12.306887 24429 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:12.307819 24493 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.313397 24421 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.314049 24548 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:12.314631 24704 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 13:06:12.310835 24714 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:12.329918 24506 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.307824 24624 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.383239 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1728) took 1238 ms. Trace:
I20260709 13:06:12.403131 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 357) took 2729 ms. Trace:
I20260709 13:06:12.401016 21858 rpcz_store.cc:276] 0709 13:06:11.144172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.355517 (+211345us) service_pool.cc:224] Handling call
0709 13:06:11.356656 (+  1139us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:11.358871 (+  2215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.359010 (+   139us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:11.360393 (+  1383us) log.cc:844] Serialized 26 byte log entry
0709 13:06:11.610911 (+250518us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:11.873435 (+262524us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.915305 (+ 41870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.915406 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.302230 (+386824us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:12.381696 (+ 79466us) raft_consensus.cc:1685] finished
0709 13:06:12.381998 (+   302us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.382507 (+   509us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34392,"prepare.queue_time_us":262370,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":50979,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":238,"spinlock_wait_cycles":13883776,"thread_start_us":495113,"threads_started":2,"wal-append.queue_time_us":433948}
I20260709 13:06:12.275189 24328 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:12.405692 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 382262us
W20260709 13:06:12.189921 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 169983us
I20260709 13:06:12.281812 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:12.409516 24478 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:12.262318 22006 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
W20260709 13:06:12.414012 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.414856 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.391737 21932 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:12.254069 21972 leader_election.cc:400] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
I20260709 13:06:12.418903 21972 leader_election.cc:403] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:12.420440 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 166503us
I20260709 13:06:12.398218 24421 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:12.385799 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 365969us
I20260709 13:06:12.335568 24648 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:12.266930 22587 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:12.343865 24692 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.404917 24578 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:12.421722 24720 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5.
W20260709 13:06:12.422217 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.411314 24478 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:12.427105 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.427698 22009 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:12.430874 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 413625us
I20260709 13:06:12.404096 21859 rpcz_store.cc:276] 0709 13:06:09.673523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.722073 (+ 48550us) service_pool.cc:224] Handling call
0709 13:06:09.722479 (+   406us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:09.749035 (+ 26556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.749080 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:09.750364 (+  1284us) log.cc:844] Serialized 78 byte log entry
0709 13:06:10.102430 (+352066us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:10.179503 (+ 77073us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.387375 (+1207872us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b18000a3ae0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:06:11.967273 (+579898us) log.cc:844] Serialized 14 byte log entry
0709 13:06:12.400953 (+433680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.401146 (+   193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.401574 (+   428us) raft_consensus.cc:1685] finished
0709 13:06:12.401590 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.402906 (+  1316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":391451,"prepare.queue_time_us":137279,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":110,"raft.queue_time_us":546668,"raft.run_cpu_time_us":771,"raft.run_wall_time_us":31396,"spinlock_wait_cycles":2802048,"thread_start_us":868024,"threads_started":4,"wal-append.queue_time_us":812251,"wal-append.run_cpu_time_us":2858,"wal-append.run_wall_time_us":1043409}
I20260709 13:06:12.432353 24585 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:06:12.441922 24578 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:12.462100 22584 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:12.471359 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:12.472141 24578 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.473202 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1949) took 4006 ms. Trace:
I20260709 13:06:12.475144 22008 rpcz_store.cc:276] 0709 13:06:08.462499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.464191 (+  1692us) service_pool.cc:224] Handling call
0709 13:06:08.493712 (+ 29521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.723752 (+1230040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.456214 (+2732462us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540033ead4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:12.468375 (+ 12161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2772948736}
I20260709 13:06:12.478082 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:12.480556 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:12.486778 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:12.492149 24578 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:12.496793 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 700) took 1032 ms. Trace:
I20260709 13:06:12.483222 24692 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.497442 22589 rpcz_store.cc:276] 0709 13:06:11.464586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.464771 (+   185us) service_pool.cc:224] Handling call
0709 13:06:12.496300 (+1031529us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400668354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.496563 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543013120}
I20260709 13:06:12.501621 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:12.503259 22586 raft_consensus.cc:2410] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:12.508514 24608 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:12.509490 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.515168 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.516443 21857 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:12.519191 22007 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:12.529855 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1912) took 4611 ms. Trace:
I20260709 13:06:12.530916 21859 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:12.536334 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:12.539079 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2006) took 2426 ms. Trace:
I20260709 13:06:12.530647 21852 rpcz_store.cc:276] 0709 13:06:07.918467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.336359 (+417892us) service_pool.cc:224] Handling call
0709 13:06:08.352208 (+ 15849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.526648 (+1174440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.529091 (+3002443us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540008a7d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:12.529627 (+   536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2648618240}
I20260709 13:06:12.543960 24569 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:12.545473 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:12.545922 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 68059.7us
I20260709 13:06:12.547665 21929 rpcz_store.cc:276] 0709 13:06:10.112017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.112139 (+   122us) service_pool.cc:224] Handling call
0709 13:06:10.112451 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.545599 (+433148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.545654 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:10.545712 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:10.548964 (+  3252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.537512 (+1988548us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54002e7cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:12.537626 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.538597 (+   971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950514944}
I20260709 13:06:12.551044 24720 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:12.560343 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:12.560523 24720 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:12.561369 24720 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:12.564625 24635 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.565336 24716 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:12.566061 24716 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:12.567078 24471 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:12.567632 24471 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:12.568073 24716 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.569963 21852 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:12.570686 24716 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:12.565346 24635 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.572779 24635 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.577386 24684 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.578226 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:12.579226 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:12.579862 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 704) took 1108 ms. Trace:
W20260709 13:06:12.584308 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.593168 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.580226 22591 rpcz_store.cc:276] 0709 13:06:11.473762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.577650 (+1103888us) service_pool.cc:224] Handling call
0709 13:06:12.579780 (+  2130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.608129 24715 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:12.608985 22002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 547) took 1489 ms (client timeout 1934 ms). Trace:
W20260709 13:06:12.609442 24678 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.202s	user 0.003s	sys 0.003s
W20260709 13:06:12.609632 22002 rpcz_store.cc:269] 0709 13:06:11.118947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.426941 (+307994us) service_pool.cc:224] Handling call
0709 13:06:12.608213 (+1181272us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540040c754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.608559 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386430336}
I20260709 13:06:12.613093 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 1963) took 2271 ms. Trace:
I20260709 13:06:12.614360 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:12.559494 24729 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.120s	user 0.000s	sys 0.003s
I20260709 13:06:12.615752 21926 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.617547 21853 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:12.580888 24684 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:12.622321 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 131536us
I20260709 13:06:12.624033 24478 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:12.613564 22003 rpcz_store.cc:276] 0709 13:06:10.341341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.776604 (+435263us) service_pool.cc:224] Handling call
0709 13:06:10.777051 (+   447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:10.777477 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.777508 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:10.783829 (+  6321us) log.cc:844] Serialized 52 byte log entry
0709 13:06:11.079199 (+295370us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:11.083360 (+  4161us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.085991 (+  2631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.086489 (+   498us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.603224 (+1516735us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:12.612236 (+  9012us) raft_consensus.cc:1685] finished
0709 13:06:12.612362 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.612780 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1243,"prepare.queue_time_us":4735,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":97,"raft.queue_time_us":769,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":208,"spinlock_wait_cycles":30864256,"thread_start_us":112939,"threads_started":1,"wal-append.queue_time_us":522135}
I20260709 13:06:12.627293 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 573) took 3120 ms. Trace:
I20260709 13:06:12.627755 22588 rpcz_store.cc:276] 0709 13:06:09.506593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.506748 (+   155us) service_pool.cc:224] Handling call
0709 13:06:09.535723 (+ 28975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.465419 (+1929696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.626214 (+1160795us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54004c22d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:12.626728 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4434371840}
I20260709 13:06:12.629683 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:12.631500 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 210503us
I20260709 13:06:12.646201 24624 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:12.647481 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.647332 24624 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:12.652128 21857 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:12.654075 24624 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:12.655045 22006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 551) took 1508 ms (client timeout 1932 ms). Trace:
I20260709 13:06:12.655376 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 1985) took 2982 ms. Trace:
I20260709 13:06:12.655894 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.651260 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.656908 21857 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 2.
I20260709 13:06:12.657641 22002 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
W20260709 13:06:12.655495 22006 rpcz_store.cc:269] 0709 13:06:11.146607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.495860 (+349253us) service_pool.cc:224] Handling call
0709 13:06:12.654687 (+1158827us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54004380d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.654859 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569396992}
I20260709 13:06:12.649431 24720 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.659602 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 714) took 1098 ms. Trace:
I20260709 13:06:12.660082 22587 rpcz_store.cc:276] 0709 13:06:11.561309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.616110 (+ 54801us) service_pool.cc:224] Handling call
0709 13:06:12.659221 (+1043111us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.659435 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233719040}
I20260709 13:06:12.660665 24720 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:12.660475 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:12.664733 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 584) took 1174 ms. Trace:
I20260709 13:06:12.665510 22592 rpcz_store.cc:276] 0709 13:06:11.490107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.505118 (+ 15011us) service_pool.cc:224] Handling call
0709 13:06:12.664007 (+1158889us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54004c2f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.664475 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238314752}
I20260709 13:06:12.667922 22590 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:12.668882 21853 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:06:12.670794 22007 pending_rounds.cc:85] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 2
I20260709 13:06:12.671214 22007 pending_rounds.cc:107] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:06:12.671248 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 715) took 1101 ms. Trace:
I20260709 13:06:12.671686 22007 raft_consensus.cc:2889] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:12.671749 22590 rpcz_store.cc:276] 0709 13:06:11.570704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.941410 (+370706us) service_pool.cc:224] Handling call
0709 13:06:12.666791 (+725381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.668462 (+  1671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.670967 (+  2505us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54004c2f50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.671169 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667070848}
I20260709 13:06:12.655965 22001 rpcz_store.cc:276] 0709 13:06:09.673290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.705536 (+ 32246us) service_pool.cc:224] Handling call
0709 13:06:10.427994 (+722458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.428522 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:10.445393 (+ 16871us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.549552 (+104159us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.571459 (+ 21907us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.571605 (+   146us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:10.571653 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.571727 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.207801 (+1636074us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b24001d38b8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d72109d 00007f3b7035310e 00007f3b7036a88c 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221
0709 13:06:12.266272 (+ 58471us) raft_consensus.cc:1685] finished
0709 13:06:12.390674 (+124402us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.654995 (+264321us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b540042c2d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.655198 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10425,"prepare.queue_time_us":16651,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":2478,"raft.queue_time_us":11661,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":3671,"spinlock_wait_cycles":1671393792,"thread_start_us":93080,"threads_started":1,"wal-append.queue_time_us":129627}
I20260709 13:06:12.674003 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.674275 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:12.675151 21857 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:12.676614 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.677487 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:12.678800 24715 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:12.679752 21853 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:12.682406 24648 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:12.683079 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:12.686271 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2037) took 1059 ms. Trace:
I20260709 13:06:12.688632 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.690577 21858 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:06:12.688283 21930 rpcz_store.cc:276] 0709 13:06:11.626387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.626507 (+   120us) service_pool.cc:224] Handling call
0709 13:06:12.682563 (+1056056us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:12.686026 (+  3463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71114496}
I20260709 13:06:12.688524 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.693948 22000 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:12.694628 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 555) took 1019 ms. Trace:
I20260709 13:06:12.695052 22000 rpcz_store.cc:276] 0709 13:06:11.675372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.692009 (+ 16637us) service_pool.cc:224] Handling call
0709 13:06:12.688114 (+996105us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:12.694531 (+  6417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61815040}
I20260709 13:06:12.695497 24477 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.698915 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.701594 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.702417 21858 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:12.702463 24478 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.706018 24495 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.702046 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2038) took 1039 ms. Trace:
I20260709 13:06:12.707032 24495 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.609746 24471 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.688943 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:12.709995 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.710157 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.711019 21859 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:12.711337 21858 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:12.712003 24495 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.716045 24471 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:12.725932 24477 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:12.707087 21932 rpcz_store.cc:276] 0709 13:06:11.662228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.662508 (+   280us) service_pool.cc:224] Handling call
0709 13:06:12.701762 (+1039254us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b540003d254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:12.701952 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1643778944}
I20260709 13:06:12.739362 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:12.705629 24478 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.766621 22001 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:12.767020 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:12.768024 22006 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.769378 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.769431 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.772673 22002 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:12.746308 24720 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:12.790890 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 139790us
I20260709 13:06:12.793295 21926 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:12.791484 24715 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.700883 24328 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:12.817991 24506 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:12.819365 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:12.818629 22009 pending_rounds.cc:85] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
I20260709 13:06:12.820725 22009 pending_rounds.cc:107] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:06:12.797948 24684 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.796271 24165 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.821141 22009 raft_consensus.cc:2889] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:12.801522 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 1998) took 4497 ms (client timeout 1743 ms). Trace:
I20260709 13:06:12.799526 24608 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.824954 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:12.824653 21926 rpcz_store.cc:269] 0709 13:06:08.300801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.379222 (+ 78421us) service_pool.cc:224] Handling call
0709 13:06:12.201828 (+3822606us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b58006539d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7072b0aa 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d327c 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c
0709 13:06:12.614102 (+412274us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54001d2a20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70744a1d 00007f3b7072b1a4 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d327c 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591
0709 13:06:12.798072 (+183970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5013649920}
I20260709 13:06:12.797823 24624 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:12.834990 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:12.835815 24523 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.836298 21927 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:12.831054 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:12.841677 21859 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 3: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:12.843668 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:12.844173 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:12.847682 24267 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:12.848945 24267 consensus_queue.cc:1243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:12.851615 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:12.852706 22005 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:12.854044 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:12.856112 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:12.857714 24533 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:12.858341 24533 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.858798 24533 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:12.862121 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 184902us
I20260709 13:06:12.863710 24715 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.812381 24166 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.873615 24166 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.854290 24506 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:12.879691 24506 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.854784 24608 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.827550 24421 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:06:12.886926 24421 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.891569 24569 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:12.892563 24569 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.893790 24495 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:12.895190 24569 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.895432 24495 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:12.895910 24495 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:12.896014 24684 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.844777 22001 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.888263 24421 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.876636 24166 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.838722 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:12.903859 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.909204 24495 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.910594 24478 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:12.911077 24495 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:12.912156 24721 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:12.912443 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 96175.2us
I20260709 13:06:12.913017 24721 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:12.915270 24328 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:12.884802 24608 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:12.918644 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.942744 24715 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:12.943639 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99656.2us
I20260709 13:06:12.946146 24720 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:12.946700 24720 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:12.946817 24684 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:12.950487 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:12.952088 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.950605 21853 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:12.953774 21857 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:12.953864 21928 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:12.954404 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:12.955462 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:12.955643 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:12.956892 21860 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:06:12.962762 21858 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 13:06:12.964136 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.945865 21929 raft_consensus.cc:2410] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:12.945886 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:12.961501 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:12.801362 24635 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:12.968582 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:12.911540 24421 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.017124 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:13.017748 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:13.020797 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 164784us
I20260709 13:06:12.836694 24523 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:13.033010 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169757us
I20260709 13:06:12.912082 24624 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:13.008215 24569 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:12.839937 24165 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.050294 22003 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:13.044879 21852 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:13.069818 21859 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:06:13.074599 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:13.068819 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.063908 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:13.070365 21859 pending_rounds.cc:85] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 2
I20260709 13:06:13.049350 24569 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:13.089972 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.087971 21930 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:13.092227 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:13.081832 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:13.097714 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.099431 21859 pending_rounds.cc:107] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:13.099820 21859 raft_consensus.cc:2889] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:13.092643 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:13.108230 24734 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.178s	user 0.000s	sys 0.002s
I20260709 13:06:13.108897 24328 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.110001 24328 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.110399 24635 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:13.111733 24731 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.171s	user 0.001s	sys 0.001s
I20260709 13:06:13.110605 24328 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.093156 24569 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.127278 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:13.130947 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:13.131928 21860 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:13.139472 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:13.142098 24165 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:13.142261 24478 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:13.144183 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:13.155895 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:13.157513 21932 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:13.159154 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:13.165069 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:13.165987 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 115141us
I20260709 13:06:13.165949 21932 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:13.124863 24721 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.155489 24609 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.150527 24267 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.197507 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:13.158007 24421 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:13.184469 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:13.205695 21928 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.183492 21852 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:13.208514 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.185920 21929 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:13.147819 24608 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:13.214984 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.215158 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.221643 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:13.222687 21932 raft_consensus.cc:2410] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:13.198400 24609 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.221783 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.243489 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:13.247531 22592 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.244484 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:13.221796 24714 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.010s
I20260709 13:06:13.171969 24533 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:13.257313 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 200702us
I20260709 13:06:13.258903 22008 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:13.259431 22587 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:13.260073 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:13.260775 21932 raft_consensus.cc:2376] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6. Re-sending same reply.
I20260709 13:06:13.252502 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:13.275758 22003 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:13.247920 24609 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:13.279785 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 176958us
I20260709 13:06:13.285513 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:13.262321 24730 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.328s	user 0.000s	sys 0.006s
I20260709 13:06:13.251041 24635 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:13.279532 24608 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:13.243001 24477 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.287355 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.293622 21859 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:13.297250 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:13.299155 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.303539 21933 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:13.308944 21857 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:13.281517 24421 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:13.238420 24267 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.315485 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:13.316992 24608 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.304617 24471 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:13.322381 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.325031 21933 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:13.328336 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2033) took 1722 ms. Trace:
W20260709 13:06:13.329192 24737 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:06:13.328848 21935 rpcz_store.cc:276] 0709 13:06:11.557336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.557456 (+   120us) service_pool.cc:224] Handling call
0709 13:06:11.557726 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:11.558135 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.558183 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:11.558759 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:06:12.662977 (+1104218us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b18002e4f20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:06:12.801396 (+138419us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:12.818214 (+ 16818us) log.cc:844] Serialized 14 byte log entry
0709 13:06:12.838012 (+ 19798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.855990 (+ 17978us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.871198 (+ 15208us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b18002e4f20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:13.268569 (+397371us) raft_consensus.cc:1685] finished
0709 13:06:13.268593 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.279999 (+ 11406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1002,"prepare.queue_time_us":4921,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":112,"raft.queue_time_us":15248,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":34130,"spinlock_wait_cycles":15038208,"thread_start_us":82423,"threads_started":1,"wal-append.queue_time_us":1249530}
I20260709 13:06:13.331496 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.332363 21935 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:13.336289 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:13.336987 21930 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.300460 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.335300 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.351356 22000 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:13.344625 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:13.353403 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 214189us
I20260709 13:06:13.281031 24714 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.359994 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:13.370908 22006 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.003s
I20260709 13:06:13.371347 22006 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:13.372238 24533 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.373569 22006 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:13.334122 24421 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:13.346314 24624 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.377671 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:13.380790 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 124219us
I20260709 13:06:13.376418 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.381560 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.308229 24644 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.104s
W20260709 13:06:13.361106 21927 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.006s
I20260709 13:06:13.386852 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:13.394595 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47849.7us
I20260709 13:06:13.403117 24471 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:13.423916 21928 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:13.428010 22008 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:13.418515 21859 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:13.417979 22007 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:13.431957 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:13.443493 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.453902 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 222254us
I20260709 13:06:13.395620 21927 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:13.439440 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.463860 22005 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:13.475221 21929 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:13.477176 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:13.478168 24714 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:13.478660 21934 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:13.478925 24721 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.479689 24721 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:13.483713 24267 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.427668 24471 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:13.476809 21929 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:13.505285 24735 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:13.515487 22585 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:13.485397 24421 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.525913 21927 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:06:13.527266 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:13.486771 24471 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:13.530751 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:13.531075 24721 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:13.515538 22586 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:06:13.551268 22586 pending_rounds.cc:85] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 5
I20260709 13:06:13.551595 22586 pending_rounds.cc:107] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:06:13.551839 21935 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:13.552083 22586 raft_consensus.cc:2889] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:13.494995 21931 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:13.552568 21930 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:13.539763 24569 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.542258 24523 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:13.544950 24743 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:06:13.538925 24751 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:13.484963 24267 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:13.562723 22001 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.003s
I20260709 13:06:13.585170 22001 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:13.574065 21860 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:13.589505 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.589437 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 235016us
I20260709 13:06:13.571521 22592 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:13.605794 22003 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:13.607372 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:13.608211 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.609016 22004 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:13.610534 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:13.613447 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:13.614403 22004 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:13.616037 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.617111 22003 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:13.618305 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.620621 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:13.625634 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.627249 22587 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:13.627952 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:13.583187 24495 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.671s	user 0.018s	sys 0.000s
I20260709 13:06:13.630045 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:13.636813 22592 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:13.572374 24506 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:06:13.577176 21931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33884
I20260709 13:06:13.639336 21931 raft_consensus.cc:606] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:06:13.639716 21931 raft_consensus.cc:614] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:13.640831 24635 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:13.641424 24635 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:13.590859 24267 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.644629 24267 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:13.645974 22585 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:13.634617 24609 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.649977 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.649240 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:13.651418 22004 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:13.652024 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:13.654438 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.656965 22009 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:13.659835 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:13.670164 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 276323us
I20260709 13:06:13.672489 22004 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:13.671214 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 231058us
I20260709 13:06:13.662441 24624 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:13.661885 24165 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:06:13.634675 24328 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:13.681236 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 209991us
I20260709 13:06:13.700012 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:13.700307 22004 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:13.701076 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:13.662585 24523 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:06:13.680074 24624 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:13.710918 24720 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.764s	user 0.015s	sys 0.000s
I20260709 13:06:13.702354 24569 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:13.726969 24609 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:13.730104 22009 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:13.730897 24609 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.731536 24609 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:13.731994 24609 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.734189 24328 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:13.734515 24752 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.735193 24752 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.735244 24328 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.735785 24752 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.736073 24328 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.737239 22004 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:13.759096 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.727752 24624 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:13.742496 24744 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.354s	user 0.004s	sys 0.000s
W20260709 13:06:13.767758 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50021.1us
W20260709 13:06:13.758208 24749 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:06:13.772640 22003 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:13.778854 22000 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:13.742318 24720 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.784147 24721 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.797734 24721 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:13.801862 24754 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:13.810853 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 356000us
I20260709 13:06:13.808983 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:13.812441 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:13.773788 24471 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.831007 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:13.835999 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:13.850607 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:13.851764 22004 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:13.860157 22000 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:06:13.894922 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.800894 24721 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:13.849594 24477 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.003s
I20260709 13:06:13.942390 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2045) took 1005 ms. Trace:
I20260709 13:06:13.945138 21932 rpcz_store.cc:276] 0709 13:06:12.935798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.402546 (+466748us) service_pool.cc:224] Handling call
0709 13:06:13.441757 (+ 39211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:13.638086 (+196329us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:13.638493 (+   407us) log.cc:844] Serialized 14 byte log entry
0709 13:06:13.858083 (+219590us) log.cc:844] Serialized 14 byte log entry
0709 13:06:13.941076 (+ 82993us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:13.941152 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:13.941215 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.941308 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.941470 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60941,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":84209,"thread_start_us":218149,"threads_started":1}
W20260709 13:06:13.948232 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 138099us
W20260709 13:06:13.950875 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 220877us
I20260709 13:06:13.948853 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:13.953895 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 103404us
I20260709 13:06:13.917392 24714 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:13.967607 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 266637us
W20260709 13:06:13.961750 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 235238us
I20260709 13:06:13.942371 24757 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:13.968763 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:13.951874 24477 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:13.973225 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 459) took 1066 ms. Trace:
I20260709 13:06:13.980070 21926 rpcz_store.cc:276] 0709 13:06:12.906788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.909123 (+  2335us) service_pool.cc:224] Handling call
0709 13:06:12.909555 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.914306 (+  4751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.914346 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.915018 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.080711 (+165693us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:13.080844 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.080971 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.733437 (+652466us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:13.967114 (+233677us) raft_consensus.cc:1685] finished
0709 13:06:13.967138 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.973144 (+  6006us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4196,"raft.queue_time_us":5728,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"spinlock_wait_cycles":7053056,"thread_start_us":164295,"threads_started":1,"wal-append.queue_time_us":367162}
I20260709 13:06:13.841089 24471 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:13.960762 21825 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:13.958865 24714 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.988693 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:13.990494 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:13.984562 24755 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.968796 24608 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:13.967226 24721 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:14.000043 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 188044us
I20260709 13:06:14.004736 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:13.999945 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:13.993022 24714 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.970070 24692 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:06:14.023108 24692 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:14.009608 24685 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.195s
I20260709 13:06:14.026712 24471 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.026871 24766 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.027501 24766 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:14.028870 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 83874.1us
I20260709 13:06:14.002288 24721 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.031883 24721 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.031714 24752 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.024839 24692 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:06:14.032783 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 71206.6us
I20260709 13:06:14.006026 24478 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:14.009871 24753 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:14.061900 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:14.014012 24684 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.015933 24328 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.065122 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:14.070713 24687 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:06:14.042176 24692 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:14.048713 24753 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.006429 24757 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:06:14.090785 21825 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.074538 24687 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:14.077378 24762 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:06:14.078534 24478 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:14.079967 24772 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.014504 24635 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.001829 24755 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:14.085345 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.072309 24720 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.067073 24756 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:14.080943 24471 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:14.100332 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.085620 24760 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:14.092962 24753 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:14.068461 24477 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.102700 24758 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.103240 24772 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:14.104784 24764 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:06:14.093495 24687 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:14.090996 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 100690us
I20260709 13:06:14.105471 24762 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:14.099488 24471 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.104852 24766 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:14.099341 24756 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.097982 24328 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:06:14.117693 24720 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.124413 24471 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.120922 24762 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:14.135037 24766 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.126212 24756 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.168272 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:14.175894 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.169597 24777 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:14.202812 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.132877 24774 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:06:14.167632 24762 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:14.137562 24267 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.179590 24721 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:14.115630 24692 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.213466 24777 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.237317 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:14.247225 22002 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:14.249176 24523 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:14.251255 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75496us
I20260709 13:06:14.252415 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.180047 24685 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:06:14.254585 24685 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:14.255110 24685 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:14.258009 24495 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:06:14.261099 24776 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 13:06:14.262359 24776 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:14.262948 24267 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.263988 24776 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:14.264427 24776 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:14.266414 24780 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:06:14.173647 24684 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.072s
W20260709 13:06:14.269557 24768 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:06:14.269876 24684 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:14.270689 24684 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:14.275285 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44940.4us
I20260709 13:06:14.276588 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.278379 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.278674 24770 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:14.279143 24770 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:14.280294 24770 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.268678 24778 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 13:06:14.242770 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.158329 24477 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.246107 24721 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.218375 24716 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:14.127512 24758 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.177843 24775 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:06:14.272190 24684 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.287945 24608 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:14.301250 24609 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 13:06:14.179330 24766 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.286948 24754 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.333184 24754 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.336022 24754 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.284067 24777 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:14.256464 24685 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.338196 24755 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.339952 24495 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.301878 24569 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:14.301931 24752 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.295267 24267 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:14.285552 24421 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:06:14.316509 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 194242us
I20260709 13:06:14.284447 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.302404 24780 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:14.245221 24760 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:06:14.307153 24721 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:14.323215 21753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 305) took 19492 ms (client timeout 15000 ms). Trace:
I20260709 13:06:14.323107 21758 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 4 to 6, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 6 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.341586 24692 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:14.341842 24772 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.342386 24759 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:14.343308 24477 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.348287 24766 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:14.350067 24758 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.311681 24735 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:14.352481 24624 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.004s
I20260709 13:06:14.389025 24624 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.391615 24624 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:14.311805 24716 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:14.393532 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 109314us
I20260709 13:06:14.394258 24716 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.353153 24609 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.397393 24609 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:14.312908 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 96759.8us
W20260709 13:06:14.356719 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 310376us
W20260709 13:06:14.359211 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 106921us
I20260709 13:06:14.399492 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.400673 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.320984 24635 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:14.373716 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 154781us
I20260709 13:06:14.401993 24635 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:14.402772 24635 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:14.403935 24635 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.358870 24778 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.362090 24754 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.292431 24687 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.405583 24774 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.406219 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.406708 24687 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.405545 24754 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.409556 24754 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:14.410583 24783 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.361629 24770 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:14.364039 24720 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:14.416827 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 347483us
I20260709 13:06:14.370885 24421 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:14.433393 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.438158 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.439159 22590 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.419063 21758 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 4, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 4 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.434484 22591 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:14.446131 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46925.8us
I20260709 13:06:14.448128 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.450297 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.366580 24776 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:14.445308 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 219378us
I20260709 13:06:14.450954 21758 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 3 to 5, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 5 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:14.463119 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:14.469944 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.470599 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:14.471469 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:14.474702 21932 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:14.425575 24760 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:06:14.422844 24720 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.481045 22000 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:14.481953 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:14.481190 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.482085 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:14.482838 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.483590 24776 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.484261 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:14.405748 24778 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:14.462738 21758 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:14.490339 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.497390 21931 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:14.506505 22588 raft_consensus.cc:2368] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:14.503232 21928 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:06:14.371649 21753 rpcz_store.cc:269] 0709 13:05:54.826683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.827000 (+   317us) service_pool.cc:224] Handling call
0709 13:06:14.314658 (+19487658us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:07.849582 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:07.965676 (+116094us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:07.965769 (+    93us) write_op.cc:435] Acquired schema lock
0709 13:06:07.965797 (+    28us) tablet.cc:662] Decoding operations
0709 13:06:08.353378 (+387581us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:08.353457 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:08.353488 (+    31us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:06:08.354729 (+  1241us) tablet.cc:701] Row locks acquired
0709 13:06:08.354783 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:06:08.355083 (+   300us) write_op.cc:270] Start()
0709 13:06:08.355283 (+   200us) write_op.cc:276] Timestamp: P: 1783602368354889 usec, L: 0
0709 13:06:08.355308 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:06:08.355913 (+   605us) log.cc:844] Serialized 5687 byte log entry
0709 13:06:10.781783 (+2425870us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:11.167250 (+385467us) op_driver.cc:464] REPLICATION: finished
0709 13:06:11.529588 (+362338us) write_op.cc:301] APPLY: starting
0709 13:06:11.530937 (+  1349us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:11.531425 (+   488us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:11.531454 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:11.889070 (+357616us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:11.939283 (+ 50213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:06:11.939347 (+    64us) write_op.cc:312] APPLY: finished
0709 13:06:11.939765 (+   418us) log.cc:844] Serialized 101 byte log entry
0709 13:06:11.940015 (+   250us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:11.940187 (+   172us) write_op.cc:454] Released schema lock
0709 13:06:11.940244 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:11.940689 (+   445us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:11.940726 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16335,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":321112,"apply.run_cpu_time_us":9820,"apply.run_wall_time_us":446234,"num_ops":14,"prepare.queue_time_us":167422,"prepare.run_cpu_time_us":21560,"prepare.run_wall_time_us":681999,"raft.queue_time_us":300578,"raft.run_cpu_time_us":23320,"raft.run_wall_time_us":219476,"replication_time_us":2794439,"spinlock_wait_cycles":87487744,"thread_start_us":450008,"threads_started":4,"wal-append.queue_time_us":280402,"wal-append.run_cpu_time_us":31705,"wal-append.run_wall_time_us":4310175}]]}
I20260709 13:06:14.507503 21758 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 3 to 5, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.486701 22592 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 3.
W20260709 13:06:14.509943 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 98185.7us
I20260709 13:06:14.510490 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:14.513886 21935 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:14.515355 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:14.480194 24735 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.521064 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:14.495288 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.522542 22590 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:14.529866 21927 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:14.530349 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.531188 22584 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:14.533135 24624 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:14.487025 24165 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:06:14.536985 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 101632us
I20260709 13:06:14.521219 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.544634 22592 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:14.546542 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 145969us
W20260709 13:06:14.549628 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65008.1us
I20260709 13:06:14.486269 24787 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:14.544631 21758 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 4 to 8, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 8 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.488724 24795 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:14.479902 24759 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:06:14.496114 24793 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.559128 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:14.563073 22000 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:06:14.563833 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 81156.6us
I20260709 13:06:14.566263 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:14.564407 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:14.568622 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:14.568812 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:14.569311 22008 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:14.560551 21758 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.571656 22592 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.572335 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.490425 24780 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:06:14.489037 24775 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:06:14.582911 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.584635 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.575613 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.578379 22000 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:14.576750 21758 catalog_manager.cc:5697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 4, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 4 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:14.580883 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.580823 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 93058.4us
I20260709 13:06:14.559473 24795 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.569749 24783 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.600656 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:14.602983 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:14.549070 24714 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.603629 22008 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:14.606583 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.607240 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.609184 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.610492 24790 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:14.610891 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:14.611590 24790 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:14.612460 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:14.613157 24792 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:14.614143 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.614799 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:14.610561 22584 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:14.615803 24569 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:14.616263 24569 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:14.609418 24782 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.268s	user 0.002s	sys 0.000s
I20260709 13:06:14.621647 24778 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:14.596112 24787 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:14.647925 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:14.602542 24795 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.690922 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:14.612660 24790 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:14.691716 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.694265 21806 auto_rebalancer.cc:893] Removing replace marker failed: Replica 821b3831b8a348c0826bd3a20a857f00 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:06:14.694829 21806 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:06:14.695075 24784 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:14.696599 24685 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:14.641921 24760 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.637719 21758 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 5, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:14.695111 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:06:14.718400 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:14.671186 24735 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:14.731290 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 171615us
I20260709 13:06:14.716874 21758 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:14.578541 24421 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:14.723227 24782 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:06:14.663313 24792 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:14.622452 24716 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:06:14.737324 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:14.746743 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.737206 24756 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.016s
I20260709 13:06:14.762735 21758 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 2, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 2 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:14.736176 24784 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:14.731117 24684 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:14.781451 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.652351 24777 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.779835 21758 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:14.783766 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46537.6us
W20260709 13:06:14.784162 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 209024us
I20260709 13:06:14.789597 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:14.804255 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:14.790177 24760 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:14.805938 24796 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:06:14.807900 24786 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:14.808504 24786 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.808903 24786 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.738355 24772 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:06:14.809728 24495 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:14.810729 24495 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.778796 24756 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.811300 24495 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.813380 24756 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.813228 24720 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:14.816181 24721 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.816474 24787 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.819095 21929 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:14.821290 22587 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:14.821897 24777 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:14.822885 22004 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:14.815260 22001 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:14.824146 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:14.804761 24798 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:06:14.782491 24784 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:14.831139 24766 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:14.841159 24721 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:14.764647 24792 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.844106 22004 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:14.831925 21758 catalog_manager.cc:5697] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 5, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.853796 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.858582 24756 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:06:14.846581 24777 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:06:14.862516 24753 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.745s	user 0.018s	sys 0.008s
W20260709 13:06:14.898154 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 139139us
I20260709 13:06:14.899706 21758 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:14.903548 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.910888 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 344669us
I20260709 13:06:14.907534 21758 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 4, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 4 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.860392 24784 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:14.915932 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 85418.5us
I20260709 13:06:14.870262 24720 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:14.923780 22008 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:14.929080 22008 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:14.929538 22008 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:14.929766 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:14.930773 22008 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.931970 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:14.925171 24762 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.643s	user 0.017s	sys 0.007s
W20260709 13:06:14.934991 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.932118 22008 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:14.847615 24569 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:14.920804 24798 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:06:14.933594 24687 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.526s	user 0.016s	sys 0.008s
I20260709 13:06:14.921993 24766 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:14.935703 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:14.941545 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:14.968111 22003 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:06:14.968765 22003 pending_rounds.cc:85] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 4
I20260709 13:06:14.969653 22003 pending_rounds.cc:107] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:06:14.970147 22003 raft_consensus.cc:2889] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.959849 21758 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 3, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 3 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.972301 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:14.933599 24762 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.974786 21758 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:14.976178 22590 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.912972 24753 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.979189 24421 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:14.983961 24692 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:14.994969 24421 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:14.992125 21758 catalog_manager.cc:5697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 1 to 4, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 4 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.984135 24506 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:14.998076 24805 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:14.997756 24692 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:14.998373 24421 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.998461 24506 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:14.968497 24714 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:15.004233 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44777.4us
I20260709 13:06:15.005350 24692 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:15.006704 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80762.2us
I20260709 13:06:15.016167 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:15.000041 24720 raft_consensus.cc:2764] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:15.019762 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:15.021306 22592 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:06:15.025763 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:15.033309 22585 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:14.992519 24752 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:14.993597 24760 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:15.037712 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 106106us
I20260709 13:06:15.038615 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:15.043394 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.044081 21926 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.034807 21758 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:15.048686 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:15.030067 24808 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:06:15.054116 21758 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 4 to 5, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.057994 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.059283 22008 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:06:15.059657 22008 pending_rounds.cc:85] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 2
I20260709 13:06:15.059914 22008 pending_rounds.cc:107] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:15.060228 22008 raft_consensus.cc:2889] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.042846 24781 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.039546 24777 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:15.069459 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:48000
W20260709 13:06:15.085971 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73330.9us
W20260709 13:06:15.070132 24471 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.890s	user 0.015s	sys 0.000s
I20260709 13:06:15.082919 21934 raft_consensus.cc:606] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:15.075002 24790 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:15.081388 24815 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:06:15.073904 24756 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.094537 21934 raft_consensus.cc:614] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:15.129462 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:15.082458 24759 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:06:15.140157 21926 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
W20260709 13:06:15.151104 24814 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:06:15.153537 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:15.156409 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 106575us
W20260709 13:06:15.158010 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:15.160321 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 87681.6us
I20260709 13:06:15.162957 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:15.133456 24755 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:15.159988 24770 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.009s
I20260709 13:06:15.100641 24471 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.155797 24760 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:15.190953 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 108306us
I20260709 13:06:15.165588 24751 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.240828 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:15.175736 24821 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.079s	user 0.006s	sys 0.000s
I20260709 13:06:15.132710 24795 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:15.244516 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71732.9us
I20260709 13:06:15.099417 24766 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.170984 24755 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:15.247049 22000 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 13:06:15.236141 21807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59644.6us
W20260709 13:06:15.248768 24820 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:06:15.243165 24751 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:15.164961 24777 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:06:15.286724 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:15.287175 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49418.8us
I20260709 13:06:15.283459 22000 pending_rounds.cc:85] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 3
I20260709 13:06:15.182003 24760 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.305809 22000 pending_rounds.cc:107] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.4
W20260709 13:06:15.305932 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 65300.2us
I20260709 13:06:15.306420 22000 raft_consensus.cc:2889] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:15.252964 22584 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.012s
I20260709 13:06:15.310979 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:15.259285 24685 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.010s
I20260709 13:06:15.237895 24523 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:15.316119 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:15.321995 24822 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:15.322053 24770 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:15.322739 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:15.322921 24770 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:15.323417 24756 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:15.323967 24770 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.324472 24751 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.253656 24506 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:15.327935 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.502314 21860 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:15.330456 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:15.502547 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2024) took 1044 ms. Trace:
I20260709 13:06:15.502943 22003 rpcz_store.cc:276] 0709 13:06:14.458068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.458366 (+   298us) service_pool.cc:224] Handling call
0709 13:06:14.476000 (+ 17634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.977288 (+501288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.502312 (+525024us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b540038b3d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:15.502489 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129879424}
I20260709 13:06:15.338171 24752 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:15.327538 24795 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:15.330791 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.468914 24735 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:15.514142 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:15.514557 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.337610 24785 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:15.326633 24523 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.377722 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.314841 24787 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:06:15.305735 24760 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.388052 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:15.389993 22000 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:15.391338 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:15.384678 24684 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.596s	user 0.007s	sys 0.007s
I20260709 13:06:15.391398 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.534220 21859 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:15.253451 24792 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:15.393275 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.281576 24762 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:15.409374 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.408068 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.408802 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.267863 24495 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:15.425436 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 114514us
I20260709 13:06:15.260906 24786 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.313818 22584 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:15.384723 24790 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:15.518718 24770 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:15.458148 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.394876 24533 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:15.445187 21934 raft_consensus.cc:1217] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[4.4-6.5]   Dedup: 4.4->[6.5-6.5]
I20260709 13:06:15.563796 21934 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:06:15.563911 21935 raft_consensus.cc:2410] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:06:15.464464 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.394622 24756 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:15.437709 24781 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.414084 24769 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:15.469075 24720 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:15.473343 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.407186 21928 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:06:15.438625 24783 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:15.494145 21933 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:15.502962 22586 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:15.503950 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:15.569393 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65547.2us
I20260709 13:06:15.570801 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.571539 21935 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:15.517422 21932 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:06:15.498914 24776 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.982s	user 0.008s	sys 0.007s
I20260709 13:06:15.601953 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2033) took 1121 ms. Trace:
I20260709 13:06:15.531747 22003 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:15.535260 22009 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:15.506958 24421 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:06:15.537611 22002 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:15.541394 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:15.542794 21851 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:06:15.542694 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:15.493583 24777 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:06:15.605288 21859 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.439257 24755 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.540163 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.328178 24569 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:15.535637 24751 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.561280 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.614542 21853 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:15.615382 21860 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:15.518155 24785 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:15.461239 21926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "430edce57310489986e4f739148461a6"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:06:15.620656 21926 raft_consensus.cc:606] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:15.621414 21926 raft_consensus.cc:614] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:15.538977 24766 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.568202 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:15.568552 21931 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:15.627686 21852 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:15.575358 21930 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:15.581748 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:15.573688 24770 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.558148 24471 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:15.585943 24756 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.587924 24769 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.590925 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2027) took 1111 ms. Trace:
I20260709 13:06:15.655786 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.656443 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:15.592805 22592 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:15.594350 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.596310 21935 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:15.596406 24788 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:15.597645 24827 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:15.600492 24165 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.609738 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:15.519130 24735 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.616039 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:15.617656 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.602519 22007 rpcz_store.cc:276] 0709 13:06:14.480623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.817096 (+336473us) service_pool.cc:224] Handling call
0709 13:06:14.817462 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.817689 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:14.914611 (+ 96922us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.601250 (+686639us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.601562 (+   312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.601617 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:15.601672 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.601756 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.601884 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":458176,"prepare.queue_time_us":671,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":1124,"raft.run_cpu_time_us":578,"raft.run_wall_time_us":580,"thread_start_us":318258,"threads_started":2,"wal-append.queue_time_us":369186}
I20260709 13:06:15.619521 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:15.619697 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 77087.2us
I20260709 13:06:15.620793 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.566696 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:15.626838 21933 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:15.626626 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:15.581806 24784 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.624s	user 0.004s	sys 0.008s
I20260709 13:06:15.641120 22589 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:15.643827 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:15.612607 24792 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:15.629175 24506 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:06:15.643471 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:15.648749 24831 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:15.653806 24832 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.654291 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:15.653868 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.657760 21927 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.655732 22006 rpcz_store.cc:276] 0709 13:06:14.479087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.641984 (+162897us) service_pool.cc:224] Handling call
0709 13:06:14.696311 (+ 54327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:14.696787 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.696822 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:14.697574 (+   752us) log.cc:844] Serialized 52 byte log entry
0709 13:06:14.823979 (+126405us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:14.869648 (+ 45669us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.589874 (+720226us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b18003c5740. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf 00007f3b707e9038
0709 13:06:15.590206 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.590490 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.590566 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.590690 (+   124us) raft_consensus.cc:1685] finished
0709 13:06:15.590705 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.590865 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":723090,"prepare.queue_time_us":768200,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":7006,"raft.queue_time_us":11340,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":34377,"spinlock_wait_cycles":6763776,"thread_start_us":125946,"threads_started":1,"wal-append.queue_time_us":232039}
I20260709 13:06:15.628258 24533 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:15.567842 24790 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.704466 22009 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:15.630776 24762 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:15.661296 24788 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:15.616132 24783 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.644677 24755 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.663353 24165 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:15.628501 24752 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 13:06:15.740586 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.762154 21825 leader_election.cc:400] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:15.693346 21854 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:15.667119 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:15.709698 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.753121 21932 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:15.768838 21852 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
I20260709 13:06:15.665331 21931 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:15.771235 21926 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:15.778187 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162208us
I20260709 13:06:15.782374 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.783107 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:15.747943 24684 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.807253 24810 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:15.810103 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.810760 21931 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.804896 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.745409 24792 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:15.814213 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 108808us
I20260709 13:06:15.655480 24766 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.822676 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:15.766872 24762 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:15.826594 21927 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
W20260709 13:06:15.817195 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 207557us
I20260709 13:06:15.819340 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.782481 24800 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.782020 24506 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.821785 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:15.829262 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.805825 24776 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.704200 24784 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.786056 24833 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:15.833575 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.837450 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:15.833854 21931 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:15.838845 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.840247 21931 raft_consensus.cc:2368] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:15.804605 24471 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:06:15.845062 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.851014 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.856452 21932 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:15.659957 24785 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.829892 24762 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.859844 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:15.861469 24830 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:06:15.861575 24790 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:15.865294 21926 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 3.
W20260709 13:06:15.909955 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66751.1us
I20260709 13:06:15.866619 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.887559 24783 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:15.766281 24831 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.868393 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.766317 24832 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:15.873279 24772 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.874128 24533 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.876502 24735 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:15.835860 24495 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:15.878183 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 702) took 1471 ms. Trace:
I20260709 13:06:15.837253 24800 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.882251 24716 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.885739 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:15.889789 24786 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:15.906055 21927 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:15.955973 21854 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:15.867153 24787 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:15.911332 21933 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:15.909122 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:15.912034 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 74647.8us
I20260709 13:06:15.839473 24833 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.922312 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:15.827126 24810 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:06:15.817247 24684 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.926321 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:15.929078 22585 rpcz_store.cc:276] 0709 13:06:14.408668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.410025 (+  1357us) service_pool.cc:224] Handling call
0709 13:06:14.410455 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.033817 (+623362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.877440 (+843623us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54006676d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:15.877715 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431879040}
I20260709 13:06:15.844910 24506 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:15.931285 24837 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:06:15.858386 24644 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:06:15.830492 24788 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:15.932279 21900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.525s (SENT) [suppressed 23 similar messages]
I20260709 13:06:15.890466 24165 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.932209 24792 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.885875 24840 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:15.938194 24784 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:15.929526 24783 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:15.942679 24832 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:15.858235 24721 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.949625 22586 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:15.965867 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.969396 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:15.916488 24720 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:15.970927 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 572) took 1035 ms. Trace:
I20260709 13:06:15.971802 22009 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:15.849269 24776 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:15.979902 21928 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:16.413800 21933 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:16.014488 21851 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:16.011191 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:16.006313 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 184611us
I20260709 13:06:16.033185 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:16.102029 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 370716us
I20260709 13:06:16.549572 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.952021 24758 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.144477 22591 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.005s
I20260709 13:06:16.543645 22001 rpcz_store.cc:276] 0709 13:06:14.935105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.070541 (+135436us) service_pool.cc:224] Handling call
0709 13:06:15.081888 (+ 11347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.082116 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:15.601100 (+518984us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.970558 (+369458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.970612 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:15.970662 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.970750 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.970875 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":518653,"prepare.queue_time_us":524623,"prepare.run_cpu_time_us":179,"prepare.run_wall_time_us":177,"raft.queue_time_us":125811,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":109,"thread_start_us":369073,"threads_started":2,"wal-append.queue_time_us":243766}
I20260709 13:06:15.952339 24772 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.061354 21931 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:16.551935 21853 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:16.554593 22591 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:06:16.538057 21934 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.824s	user 0.008s	sys 0.004s
I20260709 13:06:16.556300 21932 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:16.558066 22009 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:16.558458 21859 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.952s	user 0.010s	sys 0.005s
I20260709 13:06:16.549775 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 717) took 1220 ms. Trace:
I20260709 13:06:16.032676 24769 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:16.563678 21933 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:16.573791 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.978145 24735 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:15.970780 24833 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:16.230187 24845 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 13:06:16.270606 24760 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
I20260709 13:06:16.594625 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.594354 24760 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.032721 24495 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.563s
I20260709 13:06:16.587495 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2062) took 1080 ms. Trace:
I20260709 13:06:16.597180 21930 rpcz_store.cc:276] 0709 13:06:15.507056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.507203 (+   147us) service_pool.cc:224] Handling call
0709 13:06:15.574895 (+ 67692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.628269 (+ 53374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.587263 (+958994us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b54002e9ad0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:16.587376 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154962304}
I20260709 13:06:16.594379 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.600721 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1689) took 1060 ms. Trace:
I20260709 13:06:16.585065 21859 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:16.584513 22586 rpcz_store.cc:276] 0709 13:06:15.329780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.330182 (+   402us) service_pool.cc:224] Handling call
0709 13:06:16.549710 (+1219528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.564817 24772 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:16.557150 24783 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.605031 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:15.952035 24716 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:16.595929 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.582504 21934 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:16.561380 24506 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:15.978689 24687 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.602077 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.601996 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.624197 21857 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.601982 24165 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.605177 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 384) took 1102 ms. Trace:
I20260709 13:06:16.635747 21859 rpcz_store.cc:276] 0709 13:06:15.499376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.541074 (+ 41698us) service_pool.cc:224] Handling call
0709 13:06:16.601607 (+1060533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.231611 24800 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:16.606776 21860 raft_consensus.cc:2368] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:16.365157 24766 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:16.601147 21928 rpcz_store.cc:276] 0709 13:06:15.540881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.926194 (+385313us) service_pool.cc:224] Handling call
0709 13:06:15.926576 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.544427 (+617851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.600481 (+ 56054us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54003539d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:16.600666 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122229120}
W20260709 13:06:16.607443 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.009s	sys 0.021s
I20260709 13:06:16.328800 24831 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:16.601114 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.609797 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1688) took 1093 ms. Trace:
W20260709 13:06:16.637113 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.010s	sys 0.021s
W20260709 13:06:16.637465 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 688604us
I20260709 13:06:16.590544 24832 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.588160 24735 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.643147 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.091396 24471 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.516s
W20260709 13:06:16.601986 24810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.010s	sys 0.031s
I20260709 13:06:16.584823 24758 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:16.633411 22587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 708) took 2186 ms (client timeout 1525 ms). Trace:
W20260709 13:06:16.633641 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81625.3us
I20260709 13:06:16.609092 24783 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.602959 24720 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:16.576294 24721 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.637326 21934 rpcz_store.cc:276] 0709 13:06:15.516023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.602693 (+ 86670us) service_pool.cc:224] Handling call
0709 13:06:16.609726 (+1007033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:16.591537 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.626011 24848 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.629081 21932 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:16.566622 24840 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:16.647086 24810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.010s	sys 0.032s
W20260709 13:06:16.648973 22587 rpcz_store.cc:269] 0709 13:06:14.437522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.437773 (+   251us) service_pool.cc:224] Handling call
0709 13:06:16.623170 (+2185397us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54007300d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:16.623380 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878197504}
I20260709 13:06:16.650295 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.652751 21852 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:16.653754 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.648635 24735 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:16.647897 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 715560us
I20260709 13:06:16.656653 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.658344 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.645117 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.636940 24844 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:16.675895 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.660284 21853 raft_consensus.cc:2368] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:16.674510 21860 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:16.653107 24751 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.032s	user 0.015s	sys 0.000s
I20260709 13:06:16.653970 24684 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:16.690460 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.693375 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.694653 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.695713 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.696388 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:16.697777 24687 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.697870 24776 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:16.698436 24755 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.926s	user 0.011s	sys 0.001s
I20260709 13:06:16.652290 24760 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:16.680507 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.681609 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 380) took 1137 ms. Trace:
I20260709 13:06:16.681183 24844 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:16.702361 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.703014 21853 raft_consensus.cc:2376] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5. Re-sending same reply.
I20260709 13:06:16.702641 24720 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.703799 24844 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:16.703985 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.704639 21853 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:16.705471 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2007) took 1166 ms. Trace:
I20260709 13:06:16.705845 22005 rpcz_store.cc:276] 0709 13:06:15.539565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.539705 (+   140us) service_pool.cc:224] Handling call
0709 13:06:15.557966 (+ 18261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.785695 (+227729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.785724 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.785776 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:15.785807 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.705266 (+919459us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400197dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:16.705315 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.705420 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523266816}
I20260709 13:06:16.682755 24753 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:16.682011 21743 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7d07aa0dcdd1484ab87925c9a4604b81 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:16.696869 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.709858 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:16.710505 21857 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:16.711808 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.712812 21935 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:16.685155 24846 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:16.687650 21929 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:16.704145 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2017) took 1099 ms. Trace:
I20260709 13:06:16.714984 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2067) took 1113 ms. Trace:
I20260709 13:06:16.715372 21929 rpcz_store.cc:276] 0709 13:06:15.601217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.976335 (+375118us) service_pool.cc:224] Handling call
0709 13:06:16.687242 (+710907us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:06:16.687465 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.714738 (+ 27273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.714933 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11091200}
I20260709 13:06:16.715939 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.717286 21934 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:06:16.713939 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:16.714829 22006 rpcz_store.cc:276] 0709 13:06:15.604886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.766093 (+161207us) service_pool.cc:224] Handling call
0709 13:06:15.785392 (+ 19299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.956243 (+170851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.956274 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.956318 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:15.956349 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.703938 (+747589us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400437bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:16.703982 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.704094 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392456576}
I20260709 13:06:16.719027 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:16.725582 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:16.725796 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.674459 24792 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.648952 24832 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.726269 22587 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:16.730369 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.731408 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.726413 21857 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:16.736122 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.736601 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.736795 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.659914 24855 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:06:16.736203 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:16.738027 22007 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.738886 22585 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:06:16.735615 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82121.8us
I20260709 13:06:16.739946 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:16.742400 22585 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:06:16.742939 22008 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:16.744057 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.744721 21860 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:16.745316 24758 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:16.745680 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.746826 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.747128 24840 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:16.747414 21860 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:16.660861 24848 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:16.747833 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 385) took 1248 ms. Trace:
I20260709 13:06:16.749353 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.749476 22002 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:16.742849 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:16.699698 21856 rpcz_store.cc:276] 0709 13:06:15.464203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.497922 (+ 33719us) service_pool.cc:224] Handling call
0709 13:06:15.540445 (+ 42523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.614417 (+ 73972us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:15.920528 (+306111us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.269243 (+348715us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:16.553753 (+284510us) log.cc:844] Serialized 26 byte log entry
0709 13:06:16.553904 (+   151us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:16.553938 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.554006 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.601690 (+ 47684us) raft_consensus.cc:1685] finished
0709 13:06:16.601708 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.601834 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":346748,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":341766,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":8677,"spinlock_wait_cycles":207744,"thread_start_us":591121,"threads_started":2,"wal-append.queue_time_us":652950}
I20260709 13:06:16.751978 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:16.710923 24766 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.741474 21743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 151347us
W20260709 13:06:16.729568 24762 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.809s	user 0.007s	sys 0.006s
I20260709 13:06:16.752625 22001 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:16.750244 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:16.753376 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.758749 21934 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:16.748373 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.748205 21854 rpcz_store.cc:276] 0709 13:06:15.499709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.628834 (+129125us) service_pool.cc:224] Handling call
0709 13:06:15.649786 (+ 20952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.956408 (+306622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.747540 (+791132us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54000fcb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:16.747771 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99466112}
I20260709 13:06:16.755458 22585 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:06:16.714478 24846 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:16.764572 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:16.755043 22587 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:16.760517 21851 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:16.760649 21860 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:16.761339 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.758749 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:16.768435 21854 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:16.741096 24844 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:16.773288 24844 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:16.764933 24762 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.773778 24844 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.773406 24766 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.775739 24762 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:16.776584 24856 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:06:16.777681 24832 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:16.778115 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:16.779760 24848 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:16.780377 24848 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.780706 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:16.765391 24795 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.781175 24784 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:16.781987 21856 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:16.781790 24795 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:16.782200 24784 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:16.782712 24784 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:16.783425 21856 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:06:16.783859 24795 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:16.784045 24847 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:16.784482 24847 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:16.785194 24772 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:16.659984 24716 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.773307 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:16.786058 24716 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.785567 24847 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.787083 24847 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.789414 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:16.789980 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.790133 22593 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:16.790424 24846 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.790897 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.758315 24840 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.791630 21929 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:16.791747 24846 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.740829 24792 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:16.789414 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:16.778307 24832 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:16.757404 24860 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:06:16.795873 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:16.784876 21856 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:06:16.796993 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.802698 22007 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:16.834705 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:16.961071 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:16.962314 22588 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
W20260709 13:06:16.977309 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.977888 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 143254us
I20260709 13:06:16.980280 21857 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:06:16.980796 21857 pending_rounds.cc:85] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 6
I20260709 13:06:16.981093 21857 pending_rounds.cc:107] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:06:16.981374 21857 raft_consensus.cc:2889] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:16.984031 21852 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:16.985055 21854 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:16.985261 24800 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:16.985795 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57966.7us
I20260709 13:06:17.042703 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:16.987030 24506 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:16.987733 24840 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:16.988947 24772 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:16.997349 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:16.993551 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:16.979646 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169356us
I20260709 13:06:17.048137 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:17.044842 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:17.051851 22593 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:16.805454 24831 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.795535 24848 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.093839 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:17.104384 22585 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:16.752280 24755 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.260s
W20260709 13:06:16.983994 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 309336us
I20260709 13:06:17.096961 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:17.042182 24800 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:17.137212 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89206.5us
I20260709 13:06:17.138038 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:17.052058 24795 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:17.091619 24810 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.783584 24784 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:17.106256 24506 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:17.091811 24852 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:17.146297 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.154125 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 166097us
W20260709 13:06:17.183029 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45070.2us
I20260709 13:06:17.148888 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:17.164108 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67259.4us
I20260709 13:06:17.143175 24800 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:17.239359 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.235818 21974 leader_election.cc:336] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.712s (SENT)
W20260709 13:06:16.807202 22591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 598) took 2018 ms (client timeout 1712 ms). Trace:
I20260709 13:06:17.265354 21823 leader_election.cc:400] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
W20260709 13:06:17.267877 22591 rpcz_store.cc:269] 0709 13:06:14.788789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.789955 (+  1166us) service_pool.cc:224] Handling call
0709 13:06:16.806941 (+2016986us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54006b3ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:16.807104 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845359360}
I20260709 13:06:17.104920 24862 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:06:17.272693 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:17.274395 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:17.268378 21823 leader_election.cc:403] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:06:17.120677 24776 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 13:06:17.271872 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:17.105350 24831 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:17.090410 24840 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:17.121740 24848 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:17.300637 22589 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:17.266811 24687 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:06:17.092845 24735 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:17.301270 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:17.232766 24506 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.235113 24684 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.304140 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:17.271215 24720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.012s	sys 0.050s
I20260709 13:06:17.235962 24861 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:17.307407 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 164498us
W20260709 13:06:17.343644 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 78555.3us
W20260709 13:06:17.311264 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 266502us
I20260709 13:06:17.269483 24753 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:17.331552 22592 raft_consensus.cc:2368] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8.
W20260709 13:06:17.350289 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.352761 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:17.356905 22003 raft_consensus.cc:2410] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:17.358507 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:17.360425 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 211648us
I20260709 13:06:17.233307 24810 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:17.362584 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:17.364925 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:17.373128 22001 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:17.330173 24758 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:17.369973 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:17.374434 22002 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.370810 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:17.336694 24720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.013s	sys 0.051s
I20260709 13:06:17.367767 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:17.375211 22008 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.334093 24506 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:17.311641 24865 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:17.266865 24866 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.116s
I20260709 13:06:17.386061 22004 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:17.389222 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:17.350333 24687 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:17.390520 22003 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:06:17.383316 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 113777us
I20260709 13:06:17.355767 24753 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.349900 24873 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:17.406150 22008 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:17.262602 24852 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:17.359606 24832 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:17.410380 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:17.450031 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1696) took 1867 ms (client timeout 2197 ms). Trace:
I20260709 13:06:17.380229 24855 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:06:17.313500 24870 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:06:17.342859 24684 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:17.450659 21931 rpcz_store.cc:269] 0709 13:06:15.582596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.843599 (+261003us) service_pool.cc:224] Handling call
0709 13:06:17.449784 (+1606185us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002e81d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:17.449948 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490575744}
W20260709 13:06:17.392325 24783 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.006s
W20260709 13:06:17.464754 21858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1741) took 2078 ms (client timeout 1699 ms). Trace:
W20260709 13:06:17.465341 21858 rpcz_store.cc:269] 0709 13:06:15.386751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.391722 (+  4971us) service_pool.cc:224] Handling call
0709 13:06:17.464444 (+2072722us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b540028c654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:17.464660 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897749376}
I20260709 13:06:17.462550 21927 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [3.5-6.7]
I20260709 13:06:17.471545 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:17.472523 22001 raft_consensus.cc:2368] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8.
I20260709 13:06:17.377216 24879 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:17.406162 24753 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.477141 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 492) took 1860 ms. Trace:
I20260709 13:06:17.416478 24720 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:17.499819 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 141365us
I20260709 13:06:17.456849 24874 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:17.507146 24878 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:06:17.509722 24868 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:06:17.402025 24886 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.125s
I20260709 13:06:17.475507 24783 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:17.359373 24861 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:17.461453 24873 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:17.734704 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.840023 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:17.592254 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 181958us
I20260709 13:06:17.480427 21927 rpcz_store.cc:276] 0709 13:06:15.617153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.975591 (+358438us) service_pool.cc:224] Handling call
0709 13:06:17.461899 (+1486308us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:06:17.462165 (+   266us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:17.469108 (+  6943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.477045 (+  7937us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12887296}
I20260709 13:06:17.565142 22002 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:17.666179 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:17.699548 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:17.700263 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:17.711371 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.487979 24872 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:17.876580 21857 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:17.872897 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 510344us
I20260709 13:06:17.820905 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:17.925459 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:17.687624 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1751) took 1004 ms. Trace:
W20260709 13:06:17.873148 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 372251us
W20260709 13:06:17.764144 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.926769 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:17.925028 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1959) took 1842 ms. Trace:
I20260709 13:06:17.925530 21824 leader_election.cc:400] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8.
I20260709 13:06:17.928544 21824 leader_election.cc:403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:06:17.928339 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:17.924182 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:17.875829 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 209704us
I20260709 13:06:17.930379 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:17.927904 21860 rpcz_store.cc:276] 0709 13:06:16.083116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.767422 (+684306us) service_pool.cc:224] Handling call
0709 13:06:16.978635 (+211213us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:17.053871 (+ 75236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.053900 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:17.122534 (+ 68634us) log.cc:844] Serialized 52 byte log entry
0709 13:06:17.341887 (+219353us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:17.849281 (+507394us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.924717 (+ 75436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.924790 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.924829 (+    39us) raft_consensus.cc:1685] finished
0709 13:06:17.924841 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.924963 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":326953,"raft.queue_time_us":117521,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":27269,"thread_start_us":526160,"threads_started":2,"wal-append.queue_time_us":454061}
I20260709 13:06:17.933279 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:17.933413 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:17.934991 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:17.935657 21860 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:17.860786 22586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.003s	sys 0.012s
W20260709 13:06:17.939908 22586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.004s	sys 0.012s
I20260709 13:06:17.481554 24877 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:17.955250 21898 leader_election.cc:400] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:17.955703 21898 leader_election.cc:403] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:17.559273 24787 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.584s	user 0.017s	sys 0.007s
I20260709 13:06:17.959796 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:17.948081 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:17.961508 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:17.659566 24880 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:06:17.966866 21857 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:17.662644 24888 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:17.945144 24478 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.005s	sys 0.026s
I20260709 13:06:17.598110 24879 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.967501 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:17.591159 24795 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:17.594012 24895 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:17.983548 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.514206 24735 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:17.686645 24889 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.594362 24864 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.867722 24891 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.926198 21851 rpcz_store.cc:276] 0709 13:06:16.683211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.794403 (+111192us) service_pool.cc:224] Handling call
0709 13:06:16.794711 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:16.979545 (+184834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.979587 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:17.092162 (+112575us) log.cc:844] Serialized 52 byte log entry
0709 13:06:17.374186 (+282024us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:17.374241 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.374339 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.687422 (+313083us) raft_consensus.cc:1685] finished
0709 13:06:17.687440 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.687568 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173890,"raft.queue_time_us":183192,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":90507,"threads_started":1,"wal-append.queue_time_us":435362}
I20260709 13:06:18.068159 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 578) took 1341 ms. Trace:
I20260709 13:06:18.001200 24888 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:17.803170 24831 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:17.589036 24720 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:17.515657 24883 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:18.026090 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:18.107472 21857 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:17.943257 24770 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.122823 22587 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:06:17.938657 24687 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:17.943436 24840 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:18.038506 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78776.2us
W20260709 13:06:18.037696 24758 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.014s	sys 0.032s
I20260709 13:06:17.954970 24608 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.048426 21860 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:17.977196 24523 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:17.996955 24870 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:18.027741 24884 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:06:18.018780 24478 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.005s	sys 0.026s
I20260709 13:06:18.003762 24787 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:17.997726 24874 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.684556 24800 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:18.156621 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.04137e+06us
I20260709 13:06:18.068823 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:17.821231 24848 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:18.070719 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:18.070979 24876 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:18.071197 24891 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.070899 24892 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:18.068656 22009 rpcz_store.cc:276] 0709 13:06:16.727164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.727400 (+   236us) service_pool.cc:224] Handling call
0709 13:06:16.727771 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.795127 (+ 67356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.795160 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.039660 (+244500us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.162419 (+122759us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:17.162482 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.975860 (+813378us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54003890d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:17.975950 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:18.067813 (+ 91863us) raft_consensus.cc:1685] finished
0709 13:06:18.067834 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.068091 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16303,"raft.queue_time_us":24406,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":120091,"spinlock_wait_cycles":86676864,"thread_start_us":14879,"threads_started":1,"wal-append.queue_time_us":160133}
I20260709 13:06:18.073993 24756 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:18.075806 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:18.076418 24860 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:18.082180 24783 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.028424 24810 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:17.688325 24832 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.035223 24795 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:18.112468 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:18.114280 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 160597us
W20260709 13:06:18.117022 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.120085 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 726) took 1411 ms. Trace:
W20260709 13:06:18.120759 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 189940us
I20260709 13:06:18.049230 24895 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:18.070798 24875 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:18.135978 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:18.039175 21858 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:06:18.094035 24852 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:18.112963 24755 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:18.094821 24888 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:18.094964 24904 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.145882 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:18.391187 24860 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:18.152468 24766 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:18.138626 24758 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.015s	sys 0.034s
I20260709 13:06:18.123282 24684 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.106006 24735 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:06:17.999065 24772 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:18.158452 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:18.290287 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:18.486395 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 196169us
W20260709 13:06:18.486706 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 328312us
I20260709 13:06:18.487233 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:18.487547 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:18.043751 24879 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.139434 24471 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.124042 24831 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:06:18.314581 24848 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.391342 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 315596us
W20260709 13:06:18.502146 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.160354 24716 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.374s	user 0.015s	sys 0.003s
I20260709 13:06:18.410547 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:18.421950 21852 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:18.317713 24870 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:18.428095 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 292144us
I20260709 13:06:18.424631 24687 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:06:18.106210 24864 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.145357 24608 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.379819 24756 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:17.965837 24877 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:18.442191 24857 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:06:18.442625 24897 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.443429 24903 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 13:06:18.446219 24854 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.009s	sys 0.025s
I20260709 13:06:18.123433 24883 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.448766 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2052) took 1091 ms. Trace:
I20260709 13:06:18.095005 24889 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.454607 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:18.474103 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.484741 24758 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:18.491751 24855 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:18.490659 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:18.541513 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 131036us
I20260709 13:06:18.490005 24873 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:18.586772 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:18.524674 24684 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.528777 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:18.431416 22592 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:18.482512 24766 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.533201 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:18.482178 24755 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:18.426079 24720 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.630236 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:18.545449 21855 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:18.549748 21852 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:18.524201 24772 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.563685 22008 rpcz_store.cc:276] 0709 13:06:17.358741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.450561 (+ 91820us) service_pool.cc:224] Handling call
0709 13:06:17.450997 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.470926 (+ 19929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.470970 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.471541 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.924955 (+453414us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:18.370638 (+445683us) log.cc:844] Serialized 14 byte log entry
0709 13:06:18.389868 (+ 19230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.390012 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:18.448497 (+ 58485us) raft_consensus.cc:1685] finished
0709 13:06:18.448524 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.448698 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38222,"prepare.queue_time_us":474694,"raft.queue_time_us":158647,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":54441,"spinlock_wait_cycles":25075968,"thread_start_us":785457,"threads_started":2,"wal-append.queue_time_us":642207}
I20260709 13:06:18.400071 24523 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:18.544741 24753 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.056s	user 0.008s	sys 0.000s
I20260709 13:06:18.598956 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:18.519038 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 113479us
I20260709 13:06:18.522593 24471 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.546908 24864 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.586202 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:18.545921 24899 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 13:06:18.619268 24873 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:18.547755 24900 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:18.534461 24735 consensus_queue.cc:1243] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:06:18.624898 22593 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:06:18.552599 24608 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.554674 24891 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.559794 24792 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:18.664201 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:18.670655 24792 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:18.542032 24800 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.672048 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:18.549095 24787 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.492677 24533 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:18.672863 24911 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.673334 24533 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:18.673403 24911 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:18.585302 24846 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.793s	user 0.007s	sys 0.006s
I20260709 13:06:18.674932 21851 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:18.672889 24787 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.677495 24787 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:18.619491 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.679087 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:18.526122 22556 connection.cc:256] RPC call timeout handler was delayed by -1.3304467330000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 13:06:18.679781 22556 connection.cc:256] RPC call timeout handler was delayed by -1.3304467330000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
I20260709 13:06:18.679585 24758 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.480572 24904 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.620195 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:18.681501 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:18.681834 21860 raft_consensus.cc:2376] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7. Re-sending same reply.
W20260709 13:06:18.634852 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.682716 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:18.642243 24716 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:18.683375 21899 connection.cc:256] RPC call timeout handler was delayed by -2.0122310509999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
I20260709 13:06:18.541795 24916 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:18.635669 24784 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.681517 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:18.641135 22008 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:18.679628 22006 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:18.684916 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.685148 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.591327 22009 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:18.557742 24867 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8.
I20260709 13:06:18.647365 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:18.694024 22591 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:18.648536 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:18.531296 24795 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:18.550084 24844 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.776s	user 0.007s	sys 0.005s
W20260709 13:06:18.654079 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55233.7us
I20260709 13:06:18.527256 24848 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.657384 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:18.639919 24165 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.999s	user 0.007s	sys 0.007s
I20260709 13:06:18.723774 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:18.557636 24894 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.250s
I20260709 13:06:18.675741 24911 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:18.680150 22556 connection.cc:256] RPC call timeout handler was delayed by -1.2638467330000367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 13:06:18.729174 22556 connection.cc:256] RPC call timeout handler was delayed by -1.246746733000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 13:06:18.729557 22556 connection.cc:256] RPC call timeout handler was delayed by -1.2422467330000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 13:06:18.683300 21860 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:18.716245 24800 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:18.557153 24893 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:18.633911 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 105225us
I20260709 13:06:18.685690 22006 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:18.752171 21926 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:18.646629 21825 leader_election.cc:400] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8.
I20260709 13:06:18.689097 21857 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:18.646653 22585 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:06:18.682643 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.712543 21899 connection.cc:256] RPC call timeout handler was delayed by -1.5862807900000462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
W20260709 13:06:18.753439 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 106939us
W20260709 13:06:18.753647 21899 connection.cc:256] RPC call timeout handler was delayed by -1.483680789999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
I20260709 13:06:18.753973 22009 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:18.756477 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:18.757329 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:18.759047 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:18.727073 22584 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.713203 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:18.617688 24861 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:18.683398 24904 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:18.579008 24897 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:18.717419 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 57245.1us
I20260709 13:06:18.654829 24753 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.722128 21854 raft_consensus.cc:2376] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8. Re-sending same reply.
I20260709 13:06:18.718904 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.821946 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:18.659612 24876 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:18.724624 22590 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:06:18.639643 24766 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.553304 24756 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.660972 24917 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:18.541337 24860 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.630939 24684 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:18.687989 24783 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.693800 24916 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:18.750465 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:18.680733 24888 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.663618 24864 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:06:18.752761 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.761859 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:18.729964 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1648467329999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:06:18.729708 24833 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 2.125s	user 0.014s	sys 0.000s
I20260709 13:06:18.712100 24523 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:18.766705 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.767396 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2050) took 1415 ms. Trace:
I20260709 13:06:18.702883 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:18.767894 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54772.5us
I20260709 13:06:18.716655 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.770889 24794 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.104s
I20260709 13:06:18.779965 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:18.684167 24784 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:18.803206 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 153200us
I20260709 13:06:18.723600 24795 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:18.816962 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:18.785156 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:18.721060 24844 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.714640 24870 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:18.749253 24495 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:18.734369 24755 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:18.748749 24800 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:18.828341 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 328288us
W20260709 13:06:19.906213 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.870466 24928 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.358s	user 0.001s	sys 0.000s
I20260709 13:06:18.761343 24759 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.908145 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2023) took 1237 ms. Trace:
I20260709 13:06:19.907804 24759 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:19.908563 22006 rpcz_store.cc:276] 0709 13:06:18.670763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.752509 (+ 81746us) service_pool.cc:224] Handling call
0709 13:06:19.908081 (+1155572us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:19.232077 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.770143 24894 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 1.142s
W20260709 13:06:19.910168 24506 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 2.507s	user 0.018s	sys 0.001s
I20260709 13:06:19.185654 24883 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:19.911331 24495 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:19.877344 21935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.015s	sys 0.022s
W20260709 13:06:19.912918 21935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.016s	sys 0.022s
I20260709 13:06:19.912671 24495 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:19.914817 22591 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:19.915482 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 742) took 1155 ms. Trace:
I20260709 13:06:19.916009 22591 rpcz_store.cc:276] 0709 13:06:18.760304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.760737 (+   433us) service_pool.cc:224] Handling call
0709 13:06:19.915405 (+1154668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:19.472427 21929 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:19.305632 22584 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:19.919193 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 732) took 1339 ms. Trace:
I20260709 13:06:19.919975 24756 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:18.950619 22589 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:19.739672 24877 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:19.758958 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:18.997575 24879 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:19.922972 21933 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:19.927129 24684 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:19.366982 21855 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.001s
I20260709 13:06:19.928712 21855 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:19.931167 24876 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:19.029383 24772 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:19.910776 24506 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:19.971434 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2078) took 1214 ms (client timeout 1536 ms). Trace:
W20260709 13:06:19.972183 21928 rpcz_store.cc:269] 0709 13:06:18.756741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.779600 (+ 22859us) service_pool.cc:224] Handling call
0709 13:06:19.971247 (+1191647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:19.973961 24506 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:18.852218 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.974534 24777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.002s	sys 0.010s
W20260709 13:06:19.975013 24777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.002s	sys 0.010s
I20260709 13:06:19.508213 21931 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:19.976234 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:19.975868 21933 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:19.977118 21931 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:06:18.992576 24924 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:06:19.978605 21931 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:19.978216 24833 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:19.919703 22584 rpcz_store.cc:276] 0709 13:06:18.579752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.586019 (+  6267us) service_pool.cc:224] Handling call
0709 13:06:19.919110 (+1333091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:19.979547 21853 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:19.032351 24847 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 2.245s	user 0.028s	sys 0.000s
W20260709 13:06:19.579663 21927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.002s	sys 0.006s
I20260709 13:06:19.505838 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.033221 21930 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:19.905798 24785 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:20.016649 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:20.027910 21927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.004s	sys 0.007s
W20260709 13:06:20.028748 24874 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.635s	user 0.013s	sys 0.000s
W20260709 13:06:19.988085 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 1.20294e+06us
I20260709 13:06:19.988821 24783 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:19.001900 21932 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:20.084880 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2080) took 1204 ms. Trace:
I20260709 13:06:20.085445 21932 rpcz_store.cc:276] 0709 13:06:18.883392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.883591 (+   199us) service_pool.cc:224] Handling call
0709 13:06:19.001585 (+117994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.084539 (+1082954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.084793 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:19.022372 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.088292 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:19.991407 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 583) took 1348 ms. Trace:
I20260709 13:06:20.089607 22009 rpcz_store.cc:276] 0709 13:06:18.642882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.766396 (+123514us) service_pool.cc:224] Handling call
0709 13:06:19.991349 (+1224953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:18.777084 24735 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:20.091609 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75149.8us
I20260709 13:06:20.092829 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:19.994982 24844 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:19.022881 24926 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:19.998355 24800 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:19.035086 24897 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:19.579921 22585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.002s	sys 0.006s
I20260709 13:06:20.003253 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 734) took 1307 ms. Trace:
I20260709 13:06:20.012046 24506 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:20.013372 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.26209e+06us
W20260709 13:06:19.211050 24854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.002s	sys 0.028s
W20260709 13:06:19.003647 24930 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:06:20.016788 24721 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.029865 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2027) took 1358 ms. Trace:
I20260709 13:06:20.196120 22008 rpcz_store.cc:276] 0709 13:06:18.671816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.702589 (+ 30773us) service_pool.cc:224] Handling call
0709 13:06:20.018373 (+1315784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:20.073061 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2079) took 1251 ms. Trace:
I20260709 13:06:20.099694 24735 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:20.019184 24917 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:19.993582 24888 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:19.967749 24784 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:20.204651 24786 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:20.205693 24786 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.209869 24917 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.211553 24766 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:20.212435 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.157932 24883 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:20.215443 24883 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.027621 24523 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.102262 21929 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:06:20.216739 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.484328 24758 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.218065 24844 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:20.217877 24758 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:20.006629 21823 connection.cc:256] RPC call timeout handler was delayed by -1.3024637700000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:06:20.223210 21823 connection.cc:256] RPC call timeout handler was delayed by -1.2772637700001042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:06:20.216537 24523 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:20.212929 24864 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:06:20.145153 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:19.011833 24471 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:20.149070 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:20.234228 22592 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:20.149834 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:20.151015 22585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.120s	user 0.007s	sys 0.007s
I20260709 13:06:20.151265 22590 rpcz_store.cc:276] 0709 13:06:18.695551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.695915 (+   364us) service_pool.cc:224] Handling call
0709 13:06:20.003175 (+1307260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:20.153267 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:20.155421 24854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.494s	user 0.002s	sys 0.036s
W20260709 13:06:20.238125 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 84925.8us
I20260709 13:06:20.239022 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.239889 24721 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:20.240567 24721 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:20.081593 24753 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:20.113556 24877 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:20.149118 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:20.154232 24800 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:20.197849 21927 rpcz_store.cc:276] 0709 13:06:18.821654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.821840 (+   186us) service_pool.cc:224] Handling call
0709 13:06:18.881573 (+ 59733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.881910 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:19.045154 (+163244us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.072253 (+1027099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:20.072334 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:20.072402 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.072683 (+   281us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.072891 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":293551,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":1411,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":22002,"spinlock_wait_cycles":54098176,"thread_start_us":992321,"threads_started":1,"wal-append.queue_time_us":1053850}
I20260709 13:06:20.145843 24847 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:20.215540 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.247645 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.215929 24883 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:19.968402 22005 rpcz_store.cc:276] 0709 13:06:17.351737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.351877 (+   140us) service_pool.cc:224] Handling call
0709 13:06:17.364423 (+ 12546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.449116 (+ 84693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.449151 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.449675 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.589365 (+139690us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:17.589409 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.589611 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:18.767160 (+1177549us) raft_consensus.cc:1685] finished
0709 13:06:18.767184 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.767322 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47969,"raft.queue_time_us":111655,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":3000,"thread_start_us":156410,"threads_started":2,"wal-append.queue_time_us":581150,"wal-append.run_cpu_time_us":3747,"wal-append.run_wall_time_us":1776823}
I20260709 13:06:20.218587 24844 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:20.220224 24758 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:20.143663 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50908.5us
W20260709 13:06:20.221129 24934 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.656s	user 0.004s	sys 0.005s
I20260709 13:06:20.222410 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:20.240448 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.078747 24777 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:20.128859 24874 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:20.262224 24783 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:20.259963 21972 connection.cc:256] RPC call timeout handler was delayed by -3.3970264079999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
I20260709 13:06:20.282686 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:20.282832 21972 connection.cc:256] RPC call timeout handler was delayed by -3.3223264079999808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
I20260709 13:06:20.283411 22005 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
W20260709 13:06:20.283767 21972 connection.cc:256] RPC call timeout handler was delayed by -3.3025264079999488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 13:06:20.284332 21972 connection.cc:256] RPC call timeout handler was delayed by -3.1765264079999724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 13:06:20.284740 21972 connection.cc:256] RPC call timeout handler was delayed by -3.1468264079999244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
I20260709 13:06:20.199056 24897 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.282982 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 732) took 3516 ms. Trace:
I20260709 13:06:20.199326 24735 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.198217 24506 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:20.288265 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.262755 24847 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.289070 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:20.289644 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 387) took 2591 ms. Trace:
I20260709 13:06:20.315420 21858 rpcz_store.cc:276] 0709 13:06:17.698667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.698794 (+   127us) service_pool.cc:224] Handling call
0709 13:06:17.821520 (+122726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.421482 (+599962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.289382 (+1867900us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000596d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:20.289574 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417477504}
I20260709 13:06:20.291483 22593 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:06:20.292135 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 488) took 4676 ms (client timeout 3395 ms). Trace:
W20260709 13:06:20.317389 21926 rpcz_store.cc:269] 0709 13:06:15.615428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.911051 (+295623us) service_pool.cc:224] Handling call
0709 13:06:20.291909 (+4380858us) spinlock_profiling.cc:243] Waited 2.77 s on lock 0x7b540003cd54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:20.292069 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6373892992}
I20260709 13:06:20.292627 24844 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:20.319715 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 412310us
I20260709 13:06:20.294627 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 738) took 1649 ms. Trace:
I20260709 13:06:20.325846 22589 rpcz_store.cc:276] 0709 13:06:18.645699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.646014 (+   315us) service_pool.cc:224] Handling call
0709 13:06:18.668699 (+ 22685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:19.920916 (+1252217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.294397 (+373481us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:20.294567 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647534976}
I20260709 13:06:20.294817 22591 raft_consensus.cc:2410] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:06:20.295630 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 394) took 2593 ms (client timeout 1764 ms). Trace:
I20260709 13:06:20.286204 22587 rpcz_store.cc:276] 0709 13:06:16.766691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.767051 (+   360us) service_pool.cc:224] Handling call
0709 13:06:18.120096 (+1353045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.123466 (+  3370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.282786 (+2159320us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54007e0250. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:20.282923 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3110659456}
I20260709 13:06:20.296366 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2062) took 2243 ms. Trace:
I20260709 13:06:20.299036 24833 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.302826 24866 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:20.262745 24800 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:20.308243 24784 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:20.286190 21972 connection.cc:256] RPC call timeout handler was delayed by -2.9488264079999453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 13:06:20.313898 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:20.315152 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:20.343704 21972 connection.cc:256] RPC call timeout handler was delayed by -2.9416264079999337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260709 13:06:20.344082 22592 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEARNER]: Advancing to term 6
W20260709 13:06:20.344228 21972 connection.cc:256] RPC call timeout handler was delayed by -2.8671544329999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:06:20.345238 21972 connection.cc:256] RPC call timeout handler was delayed by -2.8606264079999164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2295s for a response
W20260709 13:06:20.345602 21972 connection.cc:256] RPC call timeout handler was delayed by -2.8311544330000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
W20260709 13:06:20.345875 21972 connection.cc:256] RPC call timeout handler was delayed by -2.7105544329999702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1755s for a response
W20260709 13:06:20.346346 21972 connection.cc:256] RPC call timeout handler was delayed by -2.4099544330000526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
W20260709 13:06:20.293116 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.327759 21855 rpcz_store.cc:269] 0709 13:06:17.702126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.930052 (+227926us) service_pool.cc:224] Handling call
0709 13:06:20.295435 (+2365383us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54002fbcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:20.295538 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412709632}
I20260709 13:06:20.273211 24758 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:20.271031 24777 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.287081 24925 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.328873 22002 rpcz_store.cc:276] 0709 13:06:18.053525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.053685 (+   160us) service_pool.cc:224] Handling call
0709 13:06:18.054167 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.149150 (+ 94983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.149197 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.149763 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.729723 (+579960us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:18.750867 (+ 21144us) log.cc:844] Serialized 14 byte log entry
0709 13:06:18.751104 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.751181 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.220468 (+1469287us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:20.296076 (+ 75608us) raft_consensus.cc:1685] finished
0709 13:06:20.296185 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.296308 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2109,"prepare.queue_time_us":47283,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":100147,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":198765,"spinlock_wait_cycles":8971520,"thread_start_us":394903,"threads_started":2,"wal-append.queue_time_us":912086}
I20260709 13:06:20.285784 24756 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.340456 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:20.346195 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:20.367362 22005 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:20.315524 22590 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
W20260709 13:06:20.348572 21972 connection.cc:256] RPC call timeout handler was delayed by -1.691703265000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:06:20.369021 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.369957 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:20.370573 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:20.373998 22002 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:20.374972 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2022) took 1704 ms. Trace:
I20260709 13:06:20.375425 22002 rpcz_store.cc:276] 0709 13:06:18.670458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.373177 (+1702719us) service_pool.cc:224] Handling call
0709 13:06:20.373772 (+   595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.374606 (+   834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.374890 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:20.339509 24866 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:20.355567 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.359519 21934 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:20.362502 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:20.329301 24862 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.348914 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2075) took 1593 ms. Trace:
I20260709 13:06:20.343308 24844 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:20.360625 24751 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:20.401724 24946 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:20.402251 24751 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:20.402753 24751 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.285502 24916 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.402302 24844 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.381815 24644 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.405464 24644 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.401417 24862 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.363687 24833 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:06:20.366346 24854 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:06:20.401379 21935 rpcz_store.cc:276] 0709 13:06:18.755793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.758190 (+  2397us) service_pool.cc:224] Handling call
0709 13:06:18.758443 (+   253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:18.768031 (+  9588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.768063 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:18.787668 (+ 19605us) log.cc:844] Serialized 52 byte log entry
0709 13:06:19.913732 (+1126064us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:20.340758 (+427026us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.342361 (+  1603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.342465 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.342526 (+    61us) raft_consensus.cc:1685] finished
0709 13:06:20.342545 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.348854 (+  6309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9234,"raft.queue_time_us":9854,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":47573,"thread_start_us":1341921,"threads_started":2,"wal-append.queue_time_us":1385981}
I20260709 13:06:20.385353 24506 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:20.380084 24756 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.366526 24471 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:06:20.386735 24909 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:20.343060 24785 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:06:20.365248 24777 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.387742 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.315573 24897 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.393757 24523 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:20.398110 24941 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:06:20.402421 24946 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.380177 24894 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:20.325742 24721 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:06:20.365340 24758 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:20.435626 24844 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.339800 24874 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
W20260709 13:06:20.422783 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 495229us
I20260709 13:06:20.341624 24839 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:06:20.385430 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:20.485626 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:20.424204 24939 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:06:20.420806 24784 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:20.474045 24832 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.474721 24888 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:20.459388 24894 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.479889 24848 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.480420 24523 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:20.477500 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.494984 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.372843 24925 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.477389 24785 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:20.419355 22005 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:20.464563 22592 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
W20260709 13:06:20.533484 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47991.5us
I20260709 13:06:20.499378 24844 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:20.554396 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:20.559965 24883 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:20.560971 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 472739us
W20260709 13:06:20.559973 24720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.006s	sys 0.013s
W20260709 13:06:20.564551 24720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.013s
W20260709 13:06:20.565790 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 178311us
I20260709 13:06:20.531878 24839 consensus_queue.cc:1243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:20.485028 24864 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:06:20.499894 24888 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:20.564759 24883 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.580602 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:20.535760 24894 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:20.584441 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 213929us
I20260709 13:06:20.639595 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.639542 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.600175 24721 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.465454 24800 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.605432 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:20.605473 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 317243us
W20260709 13:06:20.610380 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 381667us
I20260709 13:06:20.577956 24644 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.532171 24832 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.465368 24866 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.584447 24523 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.583181 24784 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:20.615986 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.618472 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.601145 24909 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:20.621985 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:20.629375 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.665031 21928 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:20.630198 21933 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:06:20.634068 21931 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:06:20.636138 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.666936 21927 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:20.636188 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 743) took 1875 ms. Trace:
I20260709 13:06:20.553025 24848 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:20.668212 24848 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:20.668928 24848 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:20.670958 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 585) took 2023 ms. Trace:
I20260709 13:06:20.671419 22009 rpcz_store.cc:276] 0709 13:06:18.647897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.111742 (+1463845us) service_pool.cc:224] Handling call
0709 13:06:20.112173 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.112642 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.112678 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.116190 (+  3512us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.535866 (+419676us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:20.566353 (+ 30487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.566459 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.670607 (+104148us) raft_consensus.cc:1685] finished
0709 13:06:20.670630 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.670795 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":10356,"thread_start_us":336400,"threads_started":1,"wal-append.queue_time_us":480654}
I20260709 13:06:20.673276 22003 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:20.673946 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:20.674197 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:20.639578 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.639510 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:20.643606 22592 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:20.644388 21929 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:20.647099 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.646505 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:20.647822 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:20.623631 24795 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.005s
I20260709 13:06:20.638726 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:20.650677 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:20.660392 21935 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:06:20.662709 22005 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:06:20.638363 24847 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.666272 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.616963 24917 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:20.758821 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.118258 21930 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:21.118880 21927 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:06:20.822327 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.036569 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1754) took 4323 ms (client timeout 1782 ms). Trace:
I20260709 13:06:20.675165 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:21.123591 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:20.975895 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.916239 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:20.862896 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2021) took 2203 ms. Trace:
I20260709 13:06:20.675050 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:21.127090 22009 raft_consensus.cc:2410] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:21.126243 22008 rpcz_store.cc:276] 0709 13:06:18.659993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.197051 (+1537058us) service_pool.cc:224] Handling call
0709 13:06:20.197394 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.197600 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:20.641072 (+443472us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.862392 (+221320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:20.862481 (+    89us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:20.862547 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.862649 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.862815 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":185579,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":5573,"raft.queue_time_us":2386,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":7164,"thread_start_us":629880,"threads_started":2}
W20260709 13:06:20.676182 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57837.1us
I20260709 13:06:21.128125 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:18.404127 22586 rpcz_store.cc:276] 0709 13:06:16.709041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.709173 (+   132us) service_pool.cc:224] Handling call
0709 13:06:16.709492 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.727949 (+ 18457us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:17.123889 (+395940us) log.cc:844] Serialized 14 byte log entry
0709 13:06:18.119675 (+995786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:18.119726 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:18.119768 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.119834 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.120012 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":534159,"prepare.run_cpu_time_us":1952,"prepare.run_wall_time_us":28081,"raft.queue_time_us":33365,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":1338224,"threads_started":3,"wal-append.queue_time_us":1266273}
W20260709 13:06:21.047242 22000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 587) took 2388 ms (client timeout 2128 ms). Trace:
W20260709 13:06:20.861629 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 305028us
W20260709 13:06:21.129271 22000 rpcz_store.cc:269] 0709 13:06:18.658682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.716351 (+ 57669us) service_pool.cc:224] Handling call
0709 13:06:21.046901 (+2330550us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54002152d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:21.047111 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521703168}
I20260709 13:06:21.130221 21926 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:20.982265 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 401) took 2925 ms (client timeout 1694 ms). Trace:
W20260709 13:06:21.131109 21851 rpcz_store.cc:269] 0709 13:06:18.057056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.068217 (+ 11161us) service_pool.cc:224] Handling call
0709 13:06:20.981809 (+2913592us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b540025a154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:20.982162 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189736832}
W20260709 13:06:20.983563 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 400) took 2981 ms (client timeout 1602 ms). Trace:
W20260709 13:06:21.132356 21854 rpcz_store.cc:269] 0709 13:06:18.002727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.042263 (+ 39536us) service_pool.cc:224] Handling call
0709 13:06:20.983360 (+2941097us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400259754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:20.983488 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936973056}
W20260709 13:06:21.132812 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 482208us
I20260709 13:06:20.713029 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:20.981329 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 584) took 2337 ms. Trace:
I20260709 13:06:21.134013 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.134034 22006 rpcz_store.cc:276] 0709 13:06:18.643848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.909603 (+1265755us) service_pool.cc:224] Handling call
0709 13:06:19.909950 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.156076 (+246126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.156124 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.263423 (+107299us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.634833 (+371410us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:20.634885 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.634987 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.981071 (+346084us) raft_consensus.cc:1685] finished
0709 13:06:20.981092 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.981260 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3732,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":576269,"threads_started":2,"wal-append.queue_time_us":681389}
I20260709 13:06:21.136090 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:20.707244 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.666709 21931 pending_rounds.cc:85] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 3
I20260709 13:06:21.134116 21928 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:21.137290 21931 pending_rounds.cc:107] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:06:21.137667 21931 raft_consensus.cc:2889] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:21.138432 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 491971us
I20260709 13:06:21.139525 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:20.982867 22001 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:21.121702 21853 rpcz_store.cc:269] 0709 13:06:16.693314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.707764 (+ 14450us) service_pool.cc:224] Handling call
0709 13:06:21.016623 (+4308859us) spinlock_profiling.cc:243] Waited 3.24 s on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:21.016816 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7455870848}
I20260709 13:06:21.128780 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:21.145838 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 229671us
W20260709 13:06:21.150698 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 338987us
I20260709 13:06:21.151197 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.155278 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:21.156491 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2059) took 3174 ms. Trace:
I20260709 13:06:21.158244 22001 rpcz_store.cc:276] 0709 13:06:17.981858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.983271 (+  1413us) service_pool.cc:224] Handling call
0709 13:06:18.043303 (+ 60032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.982670 (+2939367us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b54002d6a20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:06:21.156214 (+173544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.156425 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15441664}
I20260709 13:06:21.158280 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.160405 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.160835 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:21.161052 21851 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:21.162108 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.165151 22003 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:21.167250 21935 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:06:21.166051 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.121433 21932 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:21.169312 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:21.170110 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.172350 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:21.176759 22006 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:20.658727 24883 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:20.751252 24950 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:06:21.058010 24937 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.819s	user 0.003s	sys 0.000s
I20260709 13:06:20.668052 22585 rpcz_store.cc:276] 0709 13:06:18.760633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.819067 (+ 58434us) service_pool.cc:224] Handling call
0709 13:06:18.819420 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.819864 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.819911 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.880830 (+ 60919us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.236964 (+1356134us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:20.241484 (+  4520us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.241718 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.241800 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.635887 (+394087us) raft_consensus.cc:1685] finished
0709 13:06:20.635976 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.636124 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2974,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":36041,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":3384,"thread_start_us":1205716,"threads_started":1,"wal-append.queue_time_us":1124553}
W20260709 13:06:20.970047 24735 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.001s
W20260709 13:06:20.907235 24882 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 3.292s	user 0.021s	sys 0.042s
I20260709 13:06:21.007314 24758 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.185288 21851 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:20.750270 24963 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:21.143981 24872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.075s	user 0.033s	sys 0.069s
I20260709 13:06:21.174752 24795 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.180593 21932 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:21.055687 24886 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:20.756695 24751 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.182425 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.188500 24720 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:21.184266 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:21.053740 24889 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.912142 24786 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:20.813146 24909 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:21.185258 22592 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:21.186371 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:21.232457 21854 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:20.654181 24721 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:20.750478 24951 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:06:21.140861 24870 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:21.129058 24756 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:20.974428 24946 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.140651 24943 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:21.141233 24839 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:21.166138 22009 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:21.093014 24925 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.205540 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.154609 24961 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:21.166783 24766 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:21.208272 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:21.266150 21851 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:21.266732 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1765) took 1001 ms. Trace:
I20260709 13:06:21.174661 24784 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:20.674922 24897 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:21.267314 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.165494 24847 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:21.268347 24847 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.268757 24847 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:21.269152 24721 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.269586 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1749) took 4586 ms. Trace:
I20260709 13:06:21.270118 21859 rpcz_store.cc:276] 0709 13:06:16.682649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.792579 (+109930us) service_pool.cc:224] Handling call
0709 13:06:16.792994 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.039894 (+246900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.039942 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.040523 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.330092 (+289569us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:17.498654 (+168562us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.608587 (+109933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.608691 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.906018 (+3297327us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:21.269079 (+363061us) raft_consensus.cc:1685] finished
0709 13:06:21.269188 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.269323 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":352798,"prepare.queue_time_us":593755,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":516648,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":27463,"spinlock_wait_cycles":5143808,"thread_start_us":334239,"threads_started":2,"wal-append.queue_time_us":386313}
I20260709 13:06:21.271479 24751 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.174434 24952 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.180075 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.216022 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:21.301975 21926 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:06:21.180663 24865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.702s	user 0.027s	sys 0.051s
I20260709 13:06:21.173389 24874 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:20.663921 24523 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.218209 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.267364 21851 rpcz_store.cc:276] 0709 13:06:20.265182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.208005 (+942823us) service_pool.cc:224] Handling call
0709 13:06:21.266660 (+ 58655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.224080 24795 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.224436 21927 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:06:21.181556 21935 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:06:21.216935 24872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.163s	user 0.034s	sys 0.069s
I20260709 13:06:21.049988 24888 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.245405 21928 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 8. Ops: []
I20260709 13:06:21.255482 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 608) took 2593 ms. Trace:
I20260709 13:06:21.256501 21853 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:21.188344 24735 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.220897 24886 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:21.226017 24720 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:21.261709 21930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:06:21.319541 21930 raft_consensus.cc:606] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.319895 21930 raft_consensus.cc:614] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:21.304044 24909 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:20.812603 24866 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.322575 22002 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:21.307876 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 127925us
I20260709 13:06:21.308333 21932 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 4: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:21.218958 24963 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.323318 21851 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:21.324062 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 403) took 2827 ms. Trace:
I20260709 13:06:21.324440 21854 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:21.324357 21851 rpcz_store.cc:276] 0709 13:06:18.496375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.317875 (+2821500us) service_pool.cc:224] Handling call
0709 13:06:21.318469 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.323843 (+  5374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.324010 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.324975 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1774) took 1057 ms. Trace:
I20260709 13:06:21.325253 21933 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:21.325385 21854 rpcz_store.cc:276] 0709 13:06:20.268023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.157968 (+889945us) service_pool.cc:224] Handling call
0709 13:06:21.324930 (+166962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.272977 24961 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:21.307703 24865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.837s	user 0.027s	sys 0.052s
I20260709 13:06:21.309710 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:21.250770 24786 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:21.272518 24848 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.000s
I20260709 13:06:21.315878 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 398) took 3467 ms. Trace:
I20260709 13:06:21.315091 22588 rpcz_store.cc:276] 0709 13:06:18.661701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.662689 (+   988us) service_pool.cc:224] Handling call
0709 13:06:18.690201 (+ 27512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.700233 (+ 10032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.700278 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.722945 (+ 22667us) log.cc:844] Serialized 26 byte log entry
0709 13:06:19.047881 (+324936us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:19.047936 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.048028 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.057286 (+2009258us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:21.255155 (+197869us) raft_consensus.cc:1685] finished
0709 13:06:21.255176 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.255420 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8341,"raft.queue_time_us":15395,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2049,"spinlock_wait_cycles":10695040,"thread_start_us":287737,"threads_started":1,"wal-append.queue_time_us":1466879}
W20260709 13:06:21.310111 24894 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.008s
I20260709 13:06:21.326376 22006 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.327286 24735 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.327984 24886 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.329708 22000 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.331332 21853 rpcz_store.cc:276] 0709 13:06:17.848137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.229798 (+3381661us) service_pool.cc:224] Handling call
0709 13:06:21.256073 (+ 26275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.315557 (+ 59484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.315813 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.338171 21851 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:21.370110 21851 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:21.310866 24889 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.339766 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.305235 24952 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.342155 24888 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:21.315546 24872 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:21.321426 24720 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:21.362684 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 182997us
I20260709 13:06:21.315833 24874 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.330704 24961 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.331007 24865 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:21.386059 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:21.364761 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 188947us
I20260709 13:06:21.359580 24926 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:21.342310 24870 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:06:21.399894 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:21.395978 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:21.379545 21851 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.391628 24886 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.399181 24926 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.399096 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:21.400048 24959 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:06:21.397822 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.399122 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.401337 21854 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.405472 21932 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:21.340040 24848 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.401973 21851 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:21.402710 24879 consensus_queue.cc:1175] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" responder_term: 5 status { last_received { term: 5 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 5 index: 4 } }
I20260709 13:06:21.407150 24971 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:21.363631 24946 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:06:21.407730 24971 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:21.408672 24753 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:21.409189 24753 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:21.408615 24971 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.409986 24981 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.410642 24981 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:21.413297 21972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.782s (SENT) [suppressed 10 similar messages]
W20260709 13:06:21.414530 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 263382us
I20260709 13:06:21.415809 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.403198 21926 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:21.382653 24975 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:21.423389 22006 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:21.341904 24894 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.400903 24917 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:21.434595 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:21.434702 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:21.453189 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:21.434959 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:21.454100 21930 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.457990 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:21.394129 24978 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:21.460480 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:21.436550 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:21.400501 24844 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.827s	user 0.008s	sys 0.004s
I20260709 13:06:21.451088 21854 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:21.390002 24847 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.410024 24753 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.442188 24758 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:21.442099 24786 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:21.465745 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73734.4us
W20260709 13:06:21.461663 24966 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 13:06:21.469590 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:21.461158 24925 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:21.480127 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.476758 21930 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:21.463752 24644 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:21.465368 24756 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:21.465514 24844 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:21.485662 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.467581 24979 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:21.468753 24506 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:21.536876 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.538827 21854 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:21.498008 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.454779 24963 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:21.503274 24794 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:21.479976 24980 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.543967 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.544173 24980 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.545285 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.545377 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.546063 22585 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:21.546274 24980 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.488066 24751 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:21.546757 24794 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:21.546926 24751 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:21.547473 24751 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.547782 24794 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.549839 24874 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.550559 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.551167 22592 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:21.546064 22588 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:21.552799 24980 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:21.507113 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 750) took 1076 ms. Trace:
W20260709 13:06:21.553834 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55909.2us
I20260709 13:06:21.553732 22590 rpcz_store.cc:276] 0709 13:06:20.430412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.430564 (+   152us) service_pool.cc:224] Handling call
0709 13:06:20.431975 (+  1411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.433999 (+  2024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.434030 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.438886 (+  4856us) log.cc:844] Serialized 564 byte log entry
0709 13:06:20.637167 (+198281us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:20.637216 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.637285 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.506906 (+869621us) raft_consensus.cc:1685] finished
0709 13:06:21.506927 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.507054 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":193908,"threads_started":1,"wal-append.queue_time_us":736508}
I20260709 13:06:21.507545 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1974) took 3567 ms. Trace:
I20260709 13:06:21.554800 24980 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:21.555938 24948 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:21.556417 24948 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:21.465267 24978 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.557438 24981 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.557220 24948 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.559185 24978 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:21.559640 24940 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.346s	user 0.000s	sys 0.003s
I20260709 13:06:21.508092 24847 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:21.509130 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 389) took 3809 ms (client timeout 1862 ms). Trace:
I20260709 13:06:21.555233 21859 rpcz_store.cc:276] 0709 13:06:17.940586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.271929 (+3331343us) service_pool.cc:224] Handling call
0709 13:06:21.272265 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:21.301715 (+ 29450us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:21.313274 (+ 11559us) log.cc:844] Serialized 14 byte log entry
0709 13:06:21.366035 (+ 52761us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:21.366535 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:06:21.366662 (+   127us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:21.377661 (+ 10999us) log.cc:844] Serialized 14 byte log entry
0709 13:06:21.378187 (+   526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.378295 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.507318 (+129023us) raft_consensus.cc:1685] finished
0709 13:06:21.507337 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.507478 (+   141us) inbound_call.cc:177] Queueing success response
0709 13:06:21.510696 (+  3218us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b1800398620. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
Metrics: {"mutex_wait_us":29192,"prepare.queue_time_us":11546,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":31283,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":15732,"spinlock_wait_cycles":2478336,"thread_start_us":22760,"threads_started":1,"wal-append.queue_time_us":70452}
W20260709 13:06:21.562569 21852 rpcz_store.cc:269] 0709 13:06:17.699866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.700006 (+   140us) service_pool.cc:224] Handling call
0709 13:06:21.508929 (+3808923us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b540025c454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:21.509057 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253372800}
I20260709 13:06:21.509383 21860 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:21.563495 24495 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.548905 24794 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:21.565119 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 396) took 3862 ms. Trace:
W20260709 13:06:21.565320 24957 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:06:21.565552 21860 rpcz_store.cc:276] 0709 13:06:17.702663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.730755 (+1028092us) service_pool.cc:224] Handling call
0709 13:06:21.509058 (+2778303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.563890 (+ 54832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.564852 (+   962us) spinlock_profiling.cc:243] Waited 2.78 s on lock 0x7b540025c450. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:21.565042 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6388252544}
I20260709 13:06:21.509572 22008 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:21.510118 24848 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:21.513548 24973 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.185s	user 0.000s	sys 0.003s
I20260709 13:06:21.514554 24895 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.570462 24895 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:21.574246 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.575138 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.525207 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.525688 24988 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:21.526876 24952 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.522105 24926 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.592579 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:21.593219 22585 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:21.530355 24777 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.079s	user 0.006s	sys 0.006s
I20260709 13:06:21.531029 24753 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:21.592604 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.532845 24925 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.576089 24783 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.516907 21930 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:21.578258 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.578866 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.579547 21852 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:06:21.587321 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.586091 24864 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.626416 22587 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:21.479046 24758 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.642268 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:21.596801 22592 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:21.595809 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:21.532179 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.643486 21930 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:21.599550 22593 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 8. Ops: []
W20260709 13:06:21.643733 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.487001 24872 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.587989 24979 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:21.604985 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:21.608101 21851 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:06:21.652128 22589 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.613780 24988 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.614727 24952 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:21.616842 22000 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:21.617576 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.618335 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:21.659238 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 222801us
W20260709 13:06:21.531673 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.660099 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.661412 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.577392 24795 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:21.646281 24997 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:06:21.638032 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.642446 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:21.643428 22586 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:21.588397 24784 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.635551 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.644807 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.675503 22593 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 4.
I20260709 13:06:21.599867 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.676436 22588 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 3.
I20260709 13:06:21.602118 24925 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.648339 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:21.648753 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.649794 24926 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:21.649331 24864 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.650835 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.652411 24979 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.653831 24720 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.589711 24883 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:21.656173 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.656327 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.700228 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.588198 24963 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.656652 22004 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.657248 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.657311 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1768) took 1391 ms. Trace:
I20260709 13:06:21.659206 21926 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:21.659474 21931 raft_consensus.cc:2410] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:21.664031 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:21.631279 22585 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:21.636322 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.669188 22006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:34918
I20260709 13:06:21.672858 21860 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:21.703137 22006 raft_consensus.cc:606] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.703625 22006 raft_consensus.cc:614] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:21.703657 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:21.703935 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 67738.8us
I20260709 13:06:21.704344 22588 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.704706 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.705354 22586 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:21.598820 24777 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.678112 21930 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:21.667928 21974 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:21.683902 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:21.477990 24865 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.705785 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.669103 24894 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.692876 21854 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:21.667538 24958 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:06:21.696833 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.717492 22587 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 4: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:21.697800 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:21.699867 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 286575us
I20260709 13:06:21.701999 21851 rpcz_store.cc:276] 0709 13:06:20.266390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.337149 (+1070759us) service_pool.cc:224] Handling call
0709 13:06:21.338052 (+   903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.657069 (+319017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.657259 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.673540 21852 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.710592 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.692771 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.712548 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:21.685175 24784 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:21.644321 24783 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.668658 24952 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.755255 22584 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:06:21.790558 24972 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.486s	user 0.003s	sys 0.002s
I20260709 13:06:21.714854 24926 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:21.712545 24847 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:21.670779 24766 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:06:21.754905 22587 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:06:21.719796 22585 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.761404 24874 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:21.756644 21927 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:21.757082 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 46547.1us
W20260709 13:06:21.762020 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 94150.1us
W20260709 13:06:21.716423 22556 connection.cc:256] RPC call timeout handler was delayed by -3.088495433999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3395s for a response
I20260709 13:06:21.716640 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:21.662140 24988 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.699179 24866 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:06:21.782783 24720 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:21.792146 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:21.859994 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:21.717696 24978 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:21.791301 24844 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:21.887128 22593 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:21.725528 24772 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:21.807061 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.706646 24961 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.717161 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.807507 24847 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.755963 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.838322 21852 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:21.903688 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.904763 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.905488 21852 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:21.906252 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.806042 24917 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:21.717161 24776 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:06:21.884294 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.710556 24883 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:21.840287 24926 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:21.888003 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:21.808331 22590 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:21.913147 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.914026 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.916128 22586 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:06:21.916603 22586 pending_rounds.cc:85] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 6
I20260709 13:06:21.916911 22586 pending_rounds.cc:107] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:06:21.917210 22586 raft_consensus.cc:2889] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:21.919128 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:21.920504 22585 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:21.719672 24967 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:06:21.908569 24847 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.930887 24926 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.759168 24758 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.896862 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:21.910310 24478 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:21.907227 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:21.962390 21852 raft_consensus.cc:2376] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4. Re-sending same reply.
I20260709 13:06:21.721329 24786 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:06:21.860208 24975 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.888429 24874 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.713374 24963 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:21.893708 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 177189us
W20260709 13:06:21.903692 24936 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.689s	user 0.002s	sys 0.002s
I20260709 13:06:21.894802 24735 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:22.019250 24735 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:22.019950 24735 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:21.924724 22556 connection.cc:256] RPC call timeout handler was delayed by -1.4727487010000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 13:06:21.890184 24978 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.026800 24978 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:21.901345 24986 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:21.895777 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 178679us
I20260709 13:06:21.928481 21935 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:21.890609 24844 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:22.033438 24844 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.033555 24795 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.034140 24795 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.036635 24795 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.901854 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.937346 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:21.925803 24993 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:22.037572 24993 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:22.037997 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.038746 21927 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:22.038417 24993 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.039705 24993 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:21.966964 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:22.040555 21852 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:22.025995 22556 connection.cc:256] RPC call timeout handler was delayed by -1.3107487009999659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260709 13:06:21.925879 24720 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:22.043932 24720 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.044508 24720 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:21.926246 24971 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:21.969853 24783 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:21.999097 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1722) took 1155 ms. Trace:
I20260709 13:06:22.045321 24971 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:22.045446 21928 rpcz_store.cc:276] 0709 13:06:20.843357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.137651 (+294294us) service_pool.cc:224] Handling call
0709 13:06:21.137927 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.315888 (+177961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.998841 (+682953us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54003539d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:21.999036 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246307200}
I20260709 13:06:22.046387 24971 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:21.923614 24870 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:22.046983 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.047443 24870 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:22.047892 24870 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:22.048163 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.967823 24751 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.051919 24751 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:22.001760 24788 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:21.972635 24874 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:21.960456 24992 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:22.004731 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1720) took 1350 ms (client timeout 1682 ms). Trace:
I20260709 13:06:21.926378 24753 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.061281 24992 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.061837 24992 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:22.061585 24753 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.063930 24753 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:21.907241 24964 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.075s
W20260709 13:06:22.061317 21933 rpcz_store.cc:269] 0709 13:06:20.654306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.666019 (+ 11713us) service_pool.cc:224] Handling call
0709 13:06:22.004559 (+1338540us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540003b954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.004670 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446502272}
I20260709 13:06:22.066399 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.067560 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.068682 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.926230 24766 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.069500 24766 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.071666 24766 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:22.016299 24961 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.073335 22587 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:22.039669 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:21.923550 24872 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:22.074813 24872 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:22.075237 24872 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.001034 22592 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:22.042557 24844 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.049515 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.078563 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.079236 24844 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.079569 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.079835 24844 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:22.080216 24844 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:22.015556 24895 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:22.080626 24844 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:22.077852 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.042667 24970 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.082787 24970 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.080334 21928 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:22.084965 24970 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.092648 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.093961 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.087339 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:22.096872 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:22.089713 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.099304 21928 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.090739 24753 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:06:22.090915 24751 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:06:22.101279 24795 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:22.101923 24795 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.102327 24795 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:22.089828 24983 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:06:22.091274 24766 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:06:22.106781 22004 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:22.107426 24848 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.107970 24848 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.107419 24970 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:22.109457 24970 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.109879 24970 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.119822 24872 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.125644 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:22.126788 24961 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:22.090566 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:22.130844 24983 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.132900 21928 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 4.
I20260709 13:06:22.147256 21928 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:06:22.137895 24872 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:22.148834 24872 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:22.149443 24872 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.150213 21852 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:21.920320 24917 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.151556 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:22.167641 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.001772 24777 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:22.179687 25000 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:06:21.973181 24975 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:22.216228 21857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 406) took 3680 ms (client timeout 1948 ms). Trace:
I20260709 13:06:22.212957 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:22.215413 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 300681us
I20260709 13:06:22.151892 24872 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.222497 24988 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:22.223434 21856 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:22.224212 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.201262 24925 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:22.216655 21857 rpcz_store.cc:269] 0709 13:06:18.535767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.563524 (+ 27757us) service_pool.cc:224] Handling call
0709 13:06:22.216036 (+3652512us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b540010b654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.216153 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93953920}
I20260709 13:06:22.222782 24917 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:22.228883 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:22.230728 24917 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:22.231274 24917 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:22.231402 24952 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:22.231709 24917 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:22.232697 24963 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.153575 24970 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.234673 24848 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.234748 24970 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.157140 24795 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.237241 24795 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:22.230846 25004 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:06:22.240902 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:22.208849 24766 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:22.239538 22588 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.534s	user 0.005s	sys 0.004s
I20260709 13:06:22.244433 22588 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:22.240077 21928 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:22.228195 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.241727 21852 raft_consensus.cc:2410] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:22.169971 24870 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.128611 24874 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.152848 24784 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.148850 24992 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.227097 24975 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.237972 25012 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:22.229430 24872 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:22.273108 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.247534 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:22.223279 24993 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:06:22.275712 21852 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:06:22.212342 24720 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:22.277412 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.279273 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:22.229202 24847 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.260247 24964 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:22.238849 24978 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.291478 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:22.272979 25012 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:22.277343 24766 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:22.298434 21933 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:22.289742 24848 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:22.288815 24478 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:22.300576 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.302652 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.302954 22006 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:22.289983 24883 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:22.303234 21933 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:22.292165 24864 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:22.305429 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.393085 24943 consensus_queue.cc:1175] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8f3b35a35257416db3eaaf364cf6b52b" responder_term: 8 status { last_received { term: 4 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:22.286387 24872 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:22.307047 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.285775 24975 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:06:22.311713 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.312953 21933 raft_consensus.cc:2376] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8. Re-sending same reply.
I20260709 13:06:22.313800 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.314340 21933 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:22.314677 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.293532 25007 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:22.315411 22006 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:22.315814 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:22.316699 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.297962 24964 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:22.318841 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.292253 25011 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:06:22.321656 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 102410us
I20260709 13:06:22.322324 24860 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:22.322762 24874 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.322573 24766 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.324223 24992 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:22.327364 24889 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.947s	user 0.006s	sys 0.002s
I20260709 13:06:22.325193 24870 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:22.327253 21933 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:06:22.328181 24977 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.999s	user 0.000s	sys 0.003s
I20260709 13:06:22.301908 24735 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.330896 24978 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:22.332129 24495 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.005s
I20260709 13:06:22.327587 24898 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.334225 24832 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:22.336649 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.338534 24872 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.339140 24970 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:22.338781 24684 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:06:22.354751 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 735) took 2267 ms. Trace:
I20260709 13:06:22.318933 25007 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:22.359519 22006 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:22.361145 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.320673 24848 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:22.369905 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:22.371040 22591 rpcz_store.cc:276] 0709 13:06:20.087566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.087741 (+   175us) service_pool.cc:224] Handling call
0709 13:06:22.344032 (+2256291us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b54006a04d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.354678 (+ 10646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146263808}
I20260709 13:06:22.372110 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:22.339946 24964 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:22.389448 22006 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:22.364387 24889 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:22.393875 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.397976 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.404243 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:22.406203 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.406437 24784 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:22.407145 24735 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.389840 24759 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.408046 24983 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:22.410741 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.414881 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.390606 24897 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.417877 22006 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:22.389456 25007 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.421994 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:22.423161 21852 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.419649 24495 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:22.419059 24783 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:22.440790 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:22.441167 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 544354us
I20260709 13:06:22.372282 24898 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.426827 24848 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:22.447775 22008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2081) took 1522 ms (client timeout 1706 ms). Trace:
I20260709 13:06:22.449414 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2068) took 1847 ms. Trace:
I20260709 13:06:22.455586 22005 rpcz_store.cc:276] 0709 13:06:20.604766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.605236 (+   470us) service_pool.cc:224] Handling call
0709 13:06:20.617758 (+ 12522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:21.129248 (+511490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.449126 (+1319878us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b540038a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:22.449355 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101706624}
I20260709 13:06:22.449525 21928 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:22.451329 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:22.448182 22008 rpcz_store.cc:269] 0709 13:06:20.925105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.127838 (+202733us) service_pool.cc:224] Handling call
0709 13:06:22.447602 (+1319764us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540042bdd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.447715 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875822976}
I20260709 13:06:22.460366 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.369338 25012 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:22.460937 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2073) took 3693 ms (client timeout 2143 ms). Trace:
W20260709 13:06:22.449360 22000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2078) took 1690 ms (client timeout 2073 ms). Trace:
W20260709 13:06:22.465920 22000 rpcz_store.cc:269] 0709 13:06:20.758560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.155007 (+396447us) service_pool.cc:224] Handling call
0709 13:06:22.449219 (+1294212us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000ca8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.449307 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398483840}
W20260709 13:06:22.450189 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:22.468258 21934 rpcz_store.cc:269] 0709 13:06:18.755160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.756891 (+  1731us) service_pool.cc:224] Handling call
0709 13:06:22.448664 (+3691773us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.448820 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929841024}
I20260709 13:06:22.469679 21934 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:06:22.465160 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.472025 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:22.472952 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.425876 24897 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.476624 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:22.478379 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.478977 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.482728 22588 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:22.483505 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:22.484063 22589 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:22.491328 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:22.492031 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 588) took 3790 ms. Trace:
I20260709 13:06:22.492249 22591 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:22.492431 22002 rpcz_store.cc:276] 0709 13:06:18.701606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.378229 (+1676623us) service_pool.cc:224] Handling call
0709 13:06:20.379151 (+   922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.323175 (+944024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.491782 (+1168607us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54003890d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:22.491973 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169117440}
I20260709 13:06:22.488547 21852 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:22.494341 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:22.496387 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:22.497150 25012 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:22.499215 21852 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.448401 24844 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.501713 22003 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:22.502765 24844 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.503484 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:22.503661 22003 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:22.504908 22003 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:22.493685 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.439376 24917 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.506305 22002 raft_consensus.cc:2410] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:22.506961 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.516220 22589 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:22.473774 24889 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:22.518170 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:22.507994 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.526399 22000 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:22.516340 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.520545 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:22.528973 21855 raft_consensus.cc:2368] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:06:22.529275 24777 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:22.529999 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:22.530485 24776 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:22.530681 21852 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:22.530957 24776 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:22.531311 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:22.531736 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:22.531950 21855 raft_consensus.cc:2376] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8. Re-sending same reply.
I20260709 13:06:22.531877 24776 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.532650 21852 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:22.521523 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:22.533607 22588 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:22.533746 24776 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:22.521956 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74378.1us
I20260709 13:06:22.525386 21934 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:06:22.536837 21934 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:22.537249 21934 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:22.537503 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:22.537920 21934 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.538664 21855 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:22.538861 21934 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.540302 21855 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:22.542547 22589 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:22.545806 22000 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:22.519928 24889 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.549633 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.554203 22000 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:22.551285 21934 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:22.551940 22556 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:22.555915 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:22.556483 22592 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:22.557668 22556 leader_election.cc:400] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:06:22.558339 22556 leader_election.cc:403] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:06:22.561106 22588 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:22.542019 24720 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.562359 22009 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:22.561375 22589 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:22.566669 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:22.554769 24889 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.555495 24978 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:06:22.564352 25023 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:22.636988 25017 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 13:06:22.643126 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76726.7us
W20260709 13:06:22.645032 24989 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.158s	user 0.002s	sys 0.000s
I20260709 13:06:22.635771 24983 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 3. Current term is 4.
I20260709 13:06:22.669291 25024 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:22.670408 25024 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:22.671581 25024 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.678701 25023 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:22.718144 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.721763 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.723456 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:22.725871 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.727721 21933 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:22.730051 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.730762 22008 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:22.731040 22000 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:22.734090 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:22.736209 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.725601 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.738284 22008 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:22.739200 22009 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:22.737238 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:22.739797 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.740334 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:22.743352 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.743156 24495 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.744361 24495 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.745527 24684 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:22.746241 24684 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:22.747706 24684 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.748483 24495 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.749488 24721 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:22.749713 24784 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:22.750223 24721 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:22.750444 24784 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:22.752092 24784 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.756240 24478 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:22.757377 24478 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.752010 24721 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.762251 24495 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:22.763800 24495 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:22.764312 24495 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:22.758896 24874 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.760632 24478 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.766283 24495 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.768127 24495 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.763381 24879 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:22.794508 24979 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:22.800809 21934 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:06:22.835305 25034 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:06:22.837509 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:22.815119 24979 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:22.853829 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:22.854414 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.853029 21933 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:06:22.870039 25032 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:06:22.872370 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2106) took 1252 ms. Trace:
W20260709 13:06:22.879536 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1726) took 1481 ms (client timeout 1734 ms). Trace:
W20260709 13:06:22.880108 21935 rpcz_store.cc:269] 0709 13:06:21.397986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.399674 (+  1688us) service_pool.cc:224] Handling call
0709 13:06:22.879037 (+1479363us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54002e81d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.879307 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204815744}
I20260709 13:06:22.881649 22006 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:22.889405 24993 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:22.890172 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 752) took 1273 ms. Trace:
I20260709 13:06:22.890672 22586 rpcz_store.cc:276] 0709 13:06:21.621545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.759037 (+137492us) service_pool.cc:224] Handling call
0709 13:06:21.790403 (+ 31366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.962393 (+171990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.889581 (+927188us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54006266d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:22.889947 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288172928}
I20260709 13:06:22.891388 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:22.893641 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 740) took 1319 ms. Trace:
I20260709 13:06:22.894091 25023 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.894543 24979 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.896422 24979 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.898021 24925 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:22.898537 24925 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:22.898671 24917 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:22.898017 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49899.4us
I20260709 13:06:22.903806 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:22.908761 24684 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:22.909571 21851 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:22.898012 24506 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:22.913621 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.914713 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:22.915560 24993 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:22.915993 24993 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:22.916488 24993 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:22.918542 24860 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.005s
I20260709 13:06:22.919492 21858 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:06:22.920091 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.872871 21928 rpcz_store.cc:276] 0709 13:06:21.619419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.245866 (+626447us) service_pool.cc:224] Handling call
0709 13:06:22.449055 (+203189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.457766 (+  8711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.871930 (+414164us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540003be50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.872164 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412232064}
I20260709 13:06:22.894122 22590 rpcz_store.cc:276] 0709 13:06:21.574598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.588750 (+ 14152us) service_pool.cc:224] Handling call
0709 13:06:21.589254 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.912594 (+323340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.893090 (+980496us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400668fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:22.893413 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503265536}
W20260709 13:06:22.934198 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.936240 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:22.943440 24945 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.944216 24945 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.900635 24925 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.903808 25007 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.961346 24523 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.963277 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:22.973135 25028 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.425s	user 0.002s	sys 0.000s
I20260709 13:06:22.940116 24862 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:22.966782 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:22.974767 24933 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:22.975230 24933 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.975615 24933 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:22.976253 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 393) took 5274 ms. Trace:
I20260709 13:06:22.966711 24772 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:22.953953 24874 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:22.967770 24948 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.940394 24506 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:22.989500 24506 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.990931 24925 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:22.993227 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.993901 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:06:22.996160 24772 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:22.968694 24860 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:22.996742 24772 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:22.997306 24860 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:22.997808 24860 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:22.998008 24772 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.999078 24860 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:22.969271 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.000471 22006 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.002126 24943 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.002808 24943 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:22.996182 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 1993) took 1367 ms (client timeout 1696 ms). Trace:
W20260709 13:06:23.004529 21851 rpcz_store.cc:269] 0709 13:06:21.628806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.755735 (+126929us) service_pool.cc:224] Handling call
0709 13:06:22.995870 (+1240135us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002fbcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.996005 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314154752}
I20260709 13:06:23.007217 24847 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:22.971402 24889 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.008901 22002 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:22.976681 21856 rpcz_store.cc:276] 0709 13:06:17.701842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.716348 (+1014506us) service_pool.cc:224] Handling call
0709 13:06:22.219353 (+3503005us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.227500 (+  8147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.975795 (+748295us) spinlock_profiling.cc:243] Waited 3.49 s on lock 0x7b540010b650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:22.976040 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8017375104}
W20260709 13:06:22.967949 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.978448 24495 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 13:06:23.014892 25013 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.658s	user 0.000s	sys 0.002s
I20260709 13:06:23.016891 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2093) took 1415 ms. Trace:
W20260709 13:06:22.984864 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.017369 24917 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:06:23.018085 25019 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.616s	user 0.002s	sys 0.000s
I20260709 13:06:23.018836 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.019975 22000 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:23.022763 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.024514 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 1987) took 1428 ms. Trace:
W20260709 13:06:23.024654 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.024925 21857 rpcz_store.cc:276] 0709 13:06:21.595568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.243445 (+647877us) service_pool.cc:224] Handling call
0709 13:06:22.256060 (+ 12615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.256498 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.256526 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.262479 (+  5953us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.395730 (+133251us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:22.418469 (+ 22739us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.418739 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.418844 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.017135 (+598291us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:23.024018 (+  6883us) raft_consensus.cc:1685] finished
0709 13:06:23.024135 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.024323 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5482,"prepare.queue_time_us":25645,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":2150,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":21967,"spinlock_wait_cycles":14430336,"thread_start_us":101371,"threads_started":1,"wal-append.queue_time_us":123841}
W20260709 13:06:23.030545 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.031149 24897 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.017412 21927 rpcz_store.cc:276] 0709 13:06:21.601606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.081959 (+480353us) service_pool.cc:224] Handling call
0709 13:06:22.082278 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.085929 (+  3651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.085963 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.090249 (+  4286us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.252259 (+162010us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:22.252318 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.252409 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.013668 (+761259us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:23.016346 (+  2678us) raft_consensus.cc:1685] finished
0709 13:06:23.016367 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.016723 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3112,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":26601344,"thread_start_us":161535,"threads_started":1,"wal-append.queue_time_us":212036}
W20260709 13:06:23.031934 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1775) took 1637 ms (client timeout 2099 ms). Trace:
W20260709 13:06:23.032441 21853 rpcz_store.cc:269] 0709 13:06:21.394395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.398828 (+  4433us) service_pool.cc:224] Handling call
0709 13:06:23.031446 (+1632618us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b540008a7d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:23.031759 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217276800}
I20260709 13:06:23.035681 24889 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:23.036757 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50645.1us
W20260709 13:06:23.044687 22007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 591) took 2823 ms (client timeout 1978 ms). Trace:
W20260709 13:06:23.045209 22007 rpcz_store.cc:269] 0709 13:06:20.220685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.222134 (+  1449us) service_pool.cc:224] Handling call
0709 13:06:23.044147 (+2822013us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540045b0d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:23.044437 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454846336}
W20260709 13:06:23.047060 22004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2028) took 4374 ms (client timeout 3811 ms). Trace:
I20260709 13:06:22.968246 24800 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:06:23.049368 22003 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:06:23.047523 22004 rpcz_store.cc:269] 0709 13:06:18.672096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.821716 (+149620us) service_pool.cc:224] Handling call
0709 13:06:23.046714 (+4224998us) spinlock_profiling.cc:243] Waited 2.83 s on lock 0x7b540040c254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:23.046854 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6517741312}
I20260709 13:06:23.053109 24800 consensus_queue.cc:1243] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:06:23.054412 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.054776 24967 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:23.055222 22008 raft_consensus.cc:2376] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6. Re-sending same reply.
I20260709 13:06:23.055065 24847 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:23.050530 21928 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:23.056682 24800 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:23.057080 21853 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:23.057134 24800 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:23.058735 25009 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.789s	user 0.004s	sys 0.006s
I20260709 13:06:23.058566 24800 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.059859 24800 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:23.062021 22005 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:23.063586 24945 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.064620 24777 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:23.065676 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:23.065291 24777 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.068157 24777 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.047744 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:23.052189 24772 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.079334 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2076) took 2423 ms. Trace:
I20260709 13:06:23.079890 22001 rpcz_store.cc:276] 0709 13:06:20.655753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.187737 (+531984us) service_pool.cc:224] Handling call
0709 13:06:21.219859 (+ 32122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.302121 (+ 82262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:21.302233 (+   112us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:21.302288 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:21.302323 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.078772 (+1776449us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b5400426154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:23.078883 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.079133 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188723584}
I20260709 13:06:23.065235 25007 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.087803 21929 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:23.080350 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.074960 24860 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:23.090917 21858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1770) took 2821 ms (client timeout 1964 ms). Trace:
W20260709 13:06:23.091337 21858 rpcz_store.cc:269] 0709 13:06:20.267040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.385125 (+118085us) service_pool.cc:224] Handling call
0709 13:06:23.076571 (+2691446us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:23.088321 (+ 11750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5595967872}
I20260709 13:06:23.088408 24993 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.093830 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2091) took 1492 ms. Trace:
I20260709 13:06:23.094228 21929 rpcz_store.cc:276] 0709 13:06:21.601101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.601204 (+   103us) service_pool.cc:224] Handling call
0709 13:06:23.093623 (+1492419us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400098dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:23.093775 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3186066688}
I20260709 13:06:23.095978 24993 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.097244 24971 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:23.104159 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.097266 24948 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.117933 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:23.151611 24788 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.151950 22004 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:23.152467 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.152374 24788 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.154168 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:23.155611 24945 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:23.156122 24945 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:23.156553 24945 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.158746 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.159541 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.159929 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:23.160630 21743 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7d07aa0dcdd1484ab87925c9a4604b81 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:23.161279 24993 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:23.161943 21852 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:23.162415 24993 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:23.162947 24993 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:23.163043 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.163431 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.164038 21853 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:23.165679 21926 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:23.166797 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.161837 21935 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:23.168462 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.168596 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.169245 22007 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:23.170222 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.171550 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.172420 21851 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:23.173903 21852 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:06:23.174074 21854 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:23.175554 21853 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:23.176026 21855 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:06:23.176435 21855 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 4
I20260709 13:06:23.176714 21855 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.5
I20260709 13:06:23.102898 24943 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:23.162485 24756 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:23.164772 22001 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
W20260709 13:06:23.180540 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76808.9us
I20260709 13:06:23.187098 21855 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Aborting config change with OpId 5.5: Aborted: Op aborted by new leader
W20260709 13:06:23.191290 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 143768us
I20260709 13:06:23.193608 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.147538 24495 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.213099 24860 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:23.213747 22008 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:23.217630 24788 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.143121 24948 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.228870 21930 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:23.230154 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.231104 24948 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.221602 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.222918 22002 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:23.225394 24860 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:23.231528 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.164305 24993 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:23.233901 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45431us
I20260709 13:06:23.237974 22003 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.226122 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.251626 21855 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:23.236835 24860 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.253039 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.253288 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.233069 24756 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:23.242610 24898 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.271775 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.273375 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:23.267131 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.262984 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.276105 24897 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:23.276451 24948 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:23.293630 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57416.6us
I20260709 13:06:23.276742 24897 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:23.240770 24945 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.299146 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.300923 22006 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:23.302577 21935 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:23.276952 25007 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.309760 21851 raft_consensus.cc:2368] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8.
I20260709 13:06:23.276700 24495 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:23.310067 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.313488 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.305373 24993 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:23.322077 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.320262 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:23.327914 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 133545us
I20260709 13:06:23.328926 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1778) took 1870 ms. Trace:
I20260709 13:06:23.324541 24847 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:06:23.338290 24847 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:23.338840 24847 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:23.333426 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.351841 24945 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.348505 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.356889 22003 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:23.359688 22001 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:23.360087 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.333243 21859 rpcz_store.cc:276] 0709 13:06:21.395467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.563089 (+167622us) service_pool.cc:224] Handling call
0709 13:06:21.563385 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.450405 (+887020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.450432 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.450471 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:22.450499 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.210704 (+760205us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b540008acd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:23.225280 (+ 14576us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.266189 (+ 40909us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6459,"raft.queue_time_us":6839,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10720,"spinlock_wait_cycles":2024264064}
W20260709 13:06:23.363746 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.364400 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.322927 25007 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.366811 21851 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:23.367697 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.364460 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.371289 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.382243 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.382978 21851 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.379001 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.379839 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:23.386611 25044 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:23.392967 24876 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:23.395705 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.396592 24909 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.397202 24909 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.378432 25023 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:06:23.398370 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:23.399498 24909 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:23.399618 24751 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:23.400033 24978 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:23.400578 24978 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.401170 24978 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.400610 21753 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 7, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 7 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:23.405206 24897 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.383729 24721 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:23.409205 24795 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.368610 24756 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.410099 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 413) took 1061 ms. Trace:
I20260709 13:06:23.340091 24847 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.378790 24945 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:23.417377 24945 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.417599 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.417857 24945 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.399878 22557 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.410521 21860 rpcz_store.cc:276] 0709 13:06:22.321586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.889818 (+568232us) service_pool.cc:224] Handling call
0709 13:06:22.890252 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.893183 (+  2931us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:22.893276 (+    93us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.956299 (+ 63023us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.294951 (+338652us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:23.295008 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.295103 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.383130 (+ 88027us) raft_consensus.cc:1685] finished
0709 13:06:23.383150 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.383302 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9332,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":148,"thread_start_us":254472,"threads_started":1,"wal-append.queue_time_us":335124}
W20260709 13:06:23.429301 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53620.1us
I20260709 13:06:23.430418 24898 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:23.430992 24898 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.427960 21859 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:23.368554 24933 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:23.443348 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 92684.7us
I20260709 13:06:23.432782 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:23.447749 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.448369 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.446280 21855 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
W20260709 13:06:23.449345 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 119067us
I20260709 13:06:23.429220 24847 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:23.431414 24898 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:23.445101 24967 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:23.468253 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.462594 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.455026 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.464764 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.478639 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.466220 24751 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:23.486831 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:23.487787 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 88028.3us
I20260709 13:06:23.488818 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:23.489750 24751 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:23.490084 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.490476 24751 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:23.493130 24721 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:23.494771 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.496364 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.496608 24788 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:23.452574 24925 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:06:23.501807 24788 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:23.502339 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.503052 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.502990 24788 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.504019 21930 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:23.507207 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.507923 21857 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:23.508391 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:23.508780 24979 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.612s	user 0.004s	sys 0.012s
I20260709 13:06:23.467095 24967 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:23.499235 25037 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:06:23.510193 21926 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:23.510571 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45920.1us
I20260709 13:06:23.511313 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:23.509979 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47614.6us
I20260709 13:06:23.511651 24933 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.510670 24979 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:23.496919 21858 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:23.508889 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.509974 25007 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.497413 22590 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:23.463605 21753 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 6, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 6 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:23.519609 21928 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:06:23.524561 21857 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.527652 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.530417 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.535935 21932 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.536420 21934 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:23.499423 24495 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.544085 21929 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:23.544301 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.510100 24967 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.547525 21753 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.550261 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.550144 24948 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:23.551265 24971 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:23.552050 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.553956 21753 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 5 to 6, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 6 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:06:23.557518 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45276us
I20260709 13:06:23.559165 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.559710 22002 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:23.515988 24979 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:06:23.559039 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46027us
I20260709 13:06:23.562827 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:23.563462 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.564520 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50460.2us
I20260709 13:06:23.569875 21932 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:23.570868 24948 raft_consensus.cc:2764] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:23.573752 21934 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:06:23.575634 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.577941 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.574816 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.582551 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:23.582434 21858 raft_consensus.cc:2410] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:23.586473 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:23.575702 21823 leader_election.cc:336] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.982s (SENT)
I20260709 13:06:23.576660 24979 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.606384 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.607275 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.607299 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:23.624436 21860 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:06:23.628453 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65698.4us
I20260709 13:06:23.636516 21851 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:23.638099 24759 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.640004 21851 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:23.643549 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:23.627981 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:23.646768 24772 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.006s
I20260709 13:06:23.626996 24874 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:06:23.676476 24874 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:23.678570 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 102882us
W20260709 13:06:23.677603 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 112185us
I20260709 13:06:23.629918 24948 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:06:23.695061 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.669894 21753 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 6, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 6 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:23.700498 22588 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:23.701082 24866 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.701261 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:23.701691 24866 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.702416 24943 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:23.702553 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 114111us
I20260709 13:06:23.704214 24866 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:23.705643 21758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 282) took 11973 ms (client timeout 15000 ms). Trace:
I20260709 13:06:23.632661 24860 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:23.717398 21823 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.143s (SENT)
I20260709 13:06:23.719693 21852 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:23.724074 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.730890 21753 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 7, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:23.706115 21758 rpcz_store.cc:269] 0709 13:06:11.743181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.801370 (+ 58189us) service_pool.cc:224] Handling call
0709 13:06:23.705387 (+11904017us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:15.812188 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:15.868651 (+ 56463us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:15.868714 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:06:15.868735 (+    21us) tablet.cc:662] Decoding operations
0709 13:06:16.279679 (+410944us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:16.279764 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:16.279803 (+    39us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:06:16.433873 (+154070us) tablet.cc:701] Row locks acquired
0709 13:06:16.458121 (+ 24248us) write_op.cc:260] PREPARE: finished
0709 13:06:16.538429 (+ 80308us) write_op.cc:270] Start()
0709 13:06:16.566222 (+ 27793us) write_op.cc:276] Timestamp: P: 1783602376478375 usec, L: 0
0709 13:06:16.566250 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:06:16.595568 (+ 29318us) log.cc:844] Serialized 8001 byte log entry
0709 13:06:17.578777 (+983209us) op_driver.cc:464] REPLICATION: finished
0709 13:06:18.151703 (+572926us) write_op.cc:301] APPLY: starting
0709 13:06:18.151829 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:18.152305 (+   476us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:18.152463 (+   158us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:18.315607 (+163144us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:18.316014 (+   407us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 13:06:18.316073 (+    59us) write_op.cc:312] APPLY: finished
0709 13:06:18.316588 (+   515us) log.cc:844] Serialized 139 byte log entry
0709 13:06:18.316802 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.318466 (+  1664us) write_op.cc:454] Released schema lock
0709 13:06:18.400556 (+ 82090us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:18.441483 (+ 40927us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.441516 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":40350,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":531015,"apply.run_cpu_time_us":6550,"apply.run_wall_time_us":303541,"num_ops":20,"prepare.queue_time_us":185572,"prepare.run_cpu_time_us":14521,"prepare.run_wall_time_us":861096,"raft.queue_time_us":321589,"raft.run_cpu_time_us":14453,"raft.run_wall_time_us":468769,"replication_time_us":911625,"thread_start_us":507284,"threads_started":4,"wal-append.queue_time_us":34054,"wal-append.run_cpu_time_us":12163,"wal-append.run_wall_time_us":3104887}]]}
I20260709 13:06:23.683267 24721 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:06:23.737054 21856 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:23.744131 24860 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.747825 21856 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [4.7-7.9]
I20260709 13:06:23.751710 21856 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:06:23.752281 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69662.1us
I20260709 13:06:23.752401 24777 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:23.766266 24777 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:23.766687 24777 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:23.767733 24777 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.769070 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.769744 24777 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.769846 22586 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:23.770900 24751 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:23.771066 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.771322 24751 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:23.771905 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.772800 22591 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:23.772632 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.769846 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:23.775112 24898 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:23.753751 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44289.1us
I20260709 13:06:23.777522 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:23.754647 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49783.4us
I20260709 13:06:23.753464 21856 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:06:23.758284 24933 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:23.764062 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.776104 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.756520 24967 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.783846 21857 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:23.748864 21859 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:23.748864 22002 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:23.733067 25037 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:23.796097 22006 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
W20260709 13:06:23.741653 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 114659us
I20260709 13:06:23.778841 21856 pending_rounds.cc:85] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 4
I20260709 13:06:23.797763 21856 pending_rounds.cc:107] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:06:23.802819 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.803536 22592 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:23.804442 21856 raft_consensus.cc:2917] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:06:23.804768 24862 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.805107 21856 raft_consensus.cc:2889] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:23.805323 24862 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.805827 24800 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:23.806509 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:23.807534 24862 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.772403 24751 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.814962 22003 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:23.815514 22003 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:23.800683 21753 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 8, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.818598 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.819017 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.725677 24874 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.821211 21853 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:23.821578 22008 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:23.792727 24933 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:23.831872 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 120219us
I20260709 13:06:23.837456 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:23.799145 25037 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.816848 25023 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.837036 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:23.853752 22591 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:23.854182 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.822993 24495 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:23.860903 21852 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:23.862525 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:23.851291 24993 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.004s
I20260709 13:06:23.804494 24945 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.872840 22003 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.874694 24978 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.865235 25023 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:23.877347 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57035.8us
I20260709 13:06:23.877861 25007 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:23.847433 25037 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:23.878301 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:23.880354 24495 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:23.880836 24879 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:23.882882 21852 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:06:23.925838 21852 pending_rounds.cc:85] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 9
I20260709 13:06:23.926076 21852 pending_rounds.cc:107] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:06:23.926223 22003 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.926299 21852 raft_consensus.cc:2889] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:23.883319 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:23.926954 21853 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:23.883738 21855 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:23.894093 24933 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.895411 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.930787 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.899196 21929 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:23.921765 24909 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:06:23.881662 24795 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:23.902462 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:23.902894 22587 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:23.904788 24874 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:23.907796 24751 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.910356 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.944070 22586 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:06:23.944442 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.911365 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.912778 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:23.913164 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:23.919196 21854 raft_consensus.cc:2410] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:06:23.924903 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.928822 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:23.931349 24993 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:23.930475 21753 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 6, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 6 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:23.929564 24847 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:23.896795 24776 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:23.929688 25024 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:06:23.900396 24898 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.921207 25007 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:23.936690 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:23.946174 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.935923 24879 consensus_queue.cc:1243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:23.958019 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:23.958837 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:23.958250 21927 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:23.945166 25066 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:23.961525 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48408us
I20260709 13:06:23.962155 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.985841 22005 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:06:23.962494 24776 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:23.963541 21931 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:23.939270 24943 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.940315 24795 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:23.939543 24979 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.996510 24979 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.004289 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.005224 22001 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:23.970116 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 102770us
I20260709 13:06:23.984686 25007 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:24.009225 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.970748 25024 consensus_queue.cc:1243] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:24.010514 24967 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:24.010926 24967 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:23.960359 24933 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:23.970651 24495 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.972239 24978 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:23.974241 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:23.979198 21851 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:23.986343 22008 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.030725 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:23.988934 22003 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:23.997125 24847 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:23.969126 25067 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:24.037457 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.037326 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:23.998927 24945 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.037351 25067 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:24.039412 24847 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.039543 22592 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:24.039352 25067 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:23.988396 25066 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.040341 25066 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.040620 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.041802 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:24.011816 24967 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.043988 24967 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.999074 24795 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.045711 24795 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.000432 22593 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:24.049417 22590 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 4
I20260709 13:06:24.049742 22590 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.5
I20260709 13:06:24.050045 22590 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Aborting config change with OpId 5.5: Aborted: Op aborted by new leader
I20260709 13:06:24.050395 22584 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:24.001668 22004 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.051729 22588 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.050428 22590 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:24.002468 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.053891 22009 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.011658 21753 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 7, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.990813 24776 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:24.060187 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 192121us
I20260709 13:06:24.060631 22589 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:24.033514 25023 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.033771 25037 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.034236 25069 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:24.060621 21753 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 7, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.037993 22006 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.053822 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.014766 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.056282 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:24.061342 21858 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:24.020531 22000 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:24.085438 22001 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:24.091817 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.092725 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.094527 22587 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.095083 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.097342 22588 raft_consensus.cc:1217] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-7.7]   Dedup: 3.5->[5.6-7.7]
I20260709 13:06:24.097239 21753 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 5, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:06:24.098562 22585 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:24.097450 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.100814 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.014374 24943 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.097885 22588 pending_rounds.cc:85] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 5
I20260709 13:06:24.114563 22588 pending_rounds.cc:107] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:06:24.114825 22588 raft_consensus.cc:2889] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:24.110634 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:24.118588 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 191655us
I20260709 13:06:24.098140 24898 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:24.100636 24993 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:24.136479 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:24.125278 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 102808us
I20260709 13:06:24.119822 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.086041 24909 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:06:24.124733 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
W20260709 13:06:24.138844 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57038.8us
I20260709 13:06:24.115687 24759 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:24.157261 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.094565 24777 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:24.162705 22004 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:24.166339 21753 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:24.168113 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:24.169322 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:24.170506 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:24.172139 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.174402 24943 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:24.165822 24866 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.177904 22009 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:24.165580 24776 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.168530 21753 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 5, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 5 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.179962 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:24.180482 21753 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 6, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 6 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.182440 21753 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 6, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 6 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.185357 24847 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:24.185487 24759 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.185931 24847 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:24.186074 24759 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.186503 24759 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.186941 24847 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.189303 24800 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:24.189221 24978 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:24.192358 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:24.192610 21856 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.193059 21852 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:24.193329 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:24.205984 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:24.209142 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.210870 22005 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:24.210971 22006 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:24.208204 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:24.213644 21858 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:24.214401 22592 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:24.136015 24993 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.233855 24909 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:24.235898 24721 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:24.237140 25067 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:24.237960 25067 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:24.238979 25067 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.240509 25067 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:24.242132 25070 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:24.242720 24898 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:24.245409 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:24.245908 25066 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.229100 24759 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.230610 24978 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.231557 22591 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:24.237102 24993 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:24.246655 25068 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:24.208437 25076 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.213851 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:24.217808 21935 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:24.263667 22592 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:06:24.158427 25081 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:24.264063 22592 pending_rounds.cc:85] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 7
I20260709 13:06:24.264303 22592 pending_rounds.cc:107] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:06:24.218595 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.264537 22592 raft_consensus.cc:2889] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:24.264752 21931 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
W20260709 13:06:24.267539 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53789.2us
I20260709 13:06:24.268261 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.259617 24879 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:24.280220 25037 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.261379 25068 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.280907 25068 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.281208 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:24.281832 21927 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:24.282135 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 115515us
I20260709 13:06:24.284940 24879 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:24.285606 24978 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.289889 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.291821 21927 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:24.292938 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.293515 22005 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.293829 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:24.294607 21856 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.295372 25068 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.297495 24759 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:24.299971 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 87783.8us
I20260709 13:06:24.301136 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:24.301678 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:24.302457 22590 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:24.303086 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:24.303690 24926 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:24.304183 24926 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.304582 24926 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.306001 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:24.304777 24879 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:24.312914 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.313956 21856 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:24.315001 22005 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:24.317449 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.336091 22003 pending_rounds.cc:85] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 2
I20260709 13:06:24.350944 22003 pending_rounds.cc:107] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 7.3
I20260709 13:06:24.351310 22003 raft_consensus.cc:2889] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:24.339349 24860 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:24.340423 24788 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:24.341354 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.341652 24943 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:24.342341 25081 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:24.345360 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.345263 25083 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:24.345746 24978 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.346858 25066 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.275314 24909 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.385571 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:24.385987 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.347657 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.386785 21929 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:24.351073 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:24.352365 22007 raft_consensus.cc:2376] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8. Re-sending same reply.
I20260709 13:06:24.318956 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:24.388221 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69524.7us
I20260709 13:06:24.347687 24879 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:24.353615 24866 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:24.370517 24860 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.369740 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.371342 22001 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.398725 24993 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.371160 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:24.373796 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.351145 25023 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:24.394280 22009 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:24.403131 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:24.394796 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43797.7us
I20260709 13:06:24.406428 22589 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:24.404313 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:24.360136 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.402032 24860 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.397222 25024 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:24.393365 24945 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.270176 25076 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.410287 21931 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:24.421815 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:24.422165 22005 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:24.393992 24495 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:24.402374 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.413672 21927 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:24.385927 24909 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:24.417407 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.397689 25068 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:24.407269 25023 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.428874 21929 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:24.353832 24898 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:06:24.404338 25066 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:24.435120 22008 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:24.437417 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.414187 25099 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:24.442792 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:24.441354 22008 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:24.438171 24866 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:24.445719 21928 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:24.438382 25023 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.449348 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.449198 24879 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.450024 21929 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:24.452970 21928 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:24.453852 21931 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:24.433367 24909 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.454969 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:24.455731 21932 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:24.434785 24495 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.458001 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:24.400295 25007 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:24.490535 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.497962 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:24.432842 25024 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:24.495087 21930 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:24.500485 21927 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:24.434563 25097 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 13:06:24.508751 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48094.4us
I20260709 13:06:24.510694 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:24.511495 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 386969us
I20260709 13:06:24.512768 22001 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:06:24.514427 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 143392us
W20260709 13:06:24.516129 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:24.516096 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 136283us
I20260709 13:06:24.465534 24866 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:24.459426 25066 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:24.518317 22002 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.518949 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:24.519677 22008 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.507146 24978 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.440037 24945 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.516216 24847 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:24.517930 24993 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:24.562494 25068 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.573431 25068 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.575692 25068 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.576320 24788 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:24.576876 24788 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.577178 24866 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.578922 24788 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:24.583617 25089 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:06:24.583434 24926 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.585701 24788 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:24.586354 24788 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.588339 24788 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:24.590771 21927 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:24.593170 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:24.594295 25066 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.595709 24945 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:24.598188 24926 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.599412 25023 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.605461 22008 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.613476 25024 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:24.617694 25082 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:24.542120 25098 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:24.566131 24978 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.621518 25097 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:24.622464 25097 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:24.623566 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.624563 22009 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:24.623819 25097 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:24.630913 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 295709us
W20260709 13:06:24.633024 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:24.633612 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 114706us
I20260709 13:06:24.637243 22001 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:24.634440 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.639000 22590 raft_consensus.cc:2376] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8. Re-sending same reply.
W20260709 13:06:24.633126 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62355.6us
I20260709 13:06:24.644773 22591 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:06:24.646277 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 242057us
W20260709 13:06:24.572926 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55844.5us
I20260709 13:06:24.649313 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:24.650638 22586 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:24.654035 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.654630 22591 raft_consensus.cc:2376] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9. Re-sending same reply.
I20260709 13:06:24.655742 22592 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:06:24.710979 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.711138 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.724732 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:24.709977 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:24.752852 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 106944us
I20260709 13:06:24.765897 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.766314 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:24.766158 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 56126.5us
I20260709 13:06:24.767585 22589 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:24.769078 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:24.770524 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59176.2us
I20260709 13:06:24.773255 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:24.776393 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.768689 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:24.777143 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.779474 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:24.780323 21929 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.777455 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:24.786224 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 75155.1us
I20260709 13:06:24.787223 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.783674 22592 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:24.789049 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:24.772620 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.793437 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:24.794623 21858 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:24.794461 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:24.798087 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:24.798849 21851 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:24.800437 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.801995 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:24.807024 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:24.809609 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:24.810413 22009 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:06:24.813232 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.793422 24847 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:24.819093 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46963.1us
I20260709 13:06:24.784920 25099 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:24.787281 25067 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:24.791145 25007 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:24.823383 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.795539 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.830338 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:24.774132 24948 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:06:24.806414 24933 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:24.847121 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46759.3us
I20260709 13:06:24.816035 24897 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.863343 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:24.823261 24721 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:24.872828 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:24.874701 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.816483 24795 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:06:24.864952 25098 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:24.825755 24847 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.870645 24978 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.875978 25107 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.905483 22009 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:24.878882 25007 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:24.924098 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.884393 25113 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:06:24.877750 24800 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:06:24.893172 25098 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:24.935236 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 72096.4us
I20260709 13:06:24.932196 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:24.936916 22009 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:24.882668 25068 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:24.909757 24866 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:24.909754 25067 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:24.946441 22001 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:06:24.877254 24897 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:24.947898 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 302556us
I20260709 13:06:24.884865 25103 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:06:24.939453 25007 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:24.963848 22009 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:24.966202 25106 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:06:24.966711 25106 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:24.934630 25023 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:24.885511 25099 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:24.926350 24945 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:06:24.949677 25098 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.951174 24721 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:24.952363 24866 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.945539 25119 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.965284 24898 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:24.926570 25107 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.951409 24909 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:24.974649 24788 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:25.016968 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:24.982800 24866 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:25.033694 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:25.049264 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.050488 22584 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.049137 24948 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:25.052181 25110 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:06:25.054065 24897 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:25.049456 24721 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.056262 24948 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.056804 24948 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.058952 24948 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.060256 25067 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.060861 25067 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:25.064806 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61700us
I20260709 13:06:24.981585 25066 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:06:25.068063 24800 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:25.069173 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.014928 25102 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:25.081892 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:25.082720 22587 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:25.084615 22008 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:06:25.087044 21929 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:25.085234 22008 pending_rounds.cc:85] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
I20260709 13:06:25.087967 22008 pending_rounds.cc:107] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:06:25.088289 22008 raft_consensus.cc:2889] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:24.996882 25119 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.023856 24788 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:25.090139 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75194.6us
I20260709 13:06:25.081475 24897 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.094746 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.094960 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.101372 22589 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:06:25.102629 25111 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.127s	user 0.001s	sys 0.001s
I20260709 13:06:25.149801 25099 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.094426 25102 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:25.153318 25102 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.154237 25107 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:25.156059 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:25.089507 25007 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:25.165912 25122 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:06:25.167138 25114 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:06:25.169113 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76590.2us
I20260709 13:06:25.169857 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.170447 24788 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.170962 25024 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.171643 25024 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.091656 24800 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.092121 25117 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 13:06:25.177253 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 84692.7us
I20260709 13:06:25.178375 22589 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:25.179600 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.173789 25024 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.169902 24909 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:25.183779 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 358794us
I20260709 13:06:25.174819 25103 raft_consensus.cc:2764] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:25.188148 21972 leader_election.cc:400] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8.
I20260709 13:06:25.188654 21972 leader_election.cc:403] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:25.181128 25106 raft_consensus.cc:2764] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:25.190138 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 614) took 1130 ms. Trace:
I20260709 13:06:25.190721 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.183270 24898 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.190979 22002 rpcz_store.cc:276] 0709 13:06:24.059216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.099638 (+ 40422us) service_pool.cc:224] Handling call
0709 13:06:24.099950 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.518983 (+419033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.189929 (+670946us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54004380d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:25.190093 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961612032}
I20260709 13:06:25.193285 24788 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.199752 24800 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:25.236232 25128 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:25.216022 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.227699 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:25.225594 25106 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:06:25.225740 25113 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.187654 25117 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:06:25.200963 25107 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8.
I20260709 13:06:25.203830 24874 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:25.240563 24860 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.807s	user 0.008s	sys 0.007s
I20260709 13:06:25.245751 24860 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:25.247956 24860 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.230470 25067 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.257169 25083 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:25.243021 25081 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.260721 25083 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:25.253605 24874 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.262974 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.266058 25099 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:25.267297 25099 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.267797 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.269630 25099 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:25.263530 24866 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.270715 25081 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:25.271121 25081 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:25.271485 25081 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.272303 24866 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.273183 25117 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:25.273730 25117 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:25.274704 25117 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.276604 24909 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.264181 25083 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.277148 24909 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.278131 25083 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.279215 24909 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.268680 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.269641 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:25.282835 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:25.283315 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81163.5us
I20260709 13:06:25.261222 24788 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:25.265576 25119 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:25.265965 24495 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:25.294737 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:25.294251 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:25.295022 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.295910 25107 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:25.299070 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:25.291941 24860 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:06:25.292874 22007 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:25.293030 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.288585 25113 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:25.301715 22007 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:25.303326 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:25.307432 22003 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:25.307924 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:25.313230 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:25.310099 24860 consensus_queue.cc:1243] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:25.292389 25106 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:06:25.309563 25107 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:25.312842 22007 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.296607 24909 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:06:25.319713 24759 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.320968 22008 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:25.322728 22007 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:25.319130 24788 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:25.325409 24788 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:25.326244 24788 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.327456 24788 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.329023 24776 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:25.330427 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46009.4us
I20260709 13:06:25.334127 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.303568 25098 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.335731 24860 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:25.320369 24759 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.351161 25098 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:25.343878 25091 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.988s	user 0.015s	sys 0.008s
I20260709 13:06:25.336691 25119 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:06:25.347286 25108 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:25.358502 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:25.365217 25108 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:25.365873 24839 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.366151 25108 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.366417 24839 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.367614 25108 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.321600 24926 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:06:25.369148 24909 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:25.369678 24909 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.363517 25068 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.370601 25068 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.371702 24909 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.372561 25068 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.326251 25105 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:25.373428 25105 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:25.374543 24788 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:25.374715 25024 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.374852 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:25.375330 25024 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.376283 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 617) took 1197 ms. Trace:
W20260709 13:06:25.376942 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:25.377509 25024 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.376637 22004 rpcz_store.cc:276] 0709 13:06:24.178479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.225195 (+ 46716us) service_pool.cc:224] Handling call
0709 13:06:24.225527 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:24.226142 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.226172 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:24.226640 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:06:24.315897 (+ 89257us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:24.315963 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.316067 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:25.343330 (+1027263us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:25.376015 (+ 32685us) raft_consensus.cc:1685] finished
0709 13:06:25.376036 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.376227 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":77,"spinlock_wait_cycles":33346304,"thread_start_us":88901,"threads_started":1,"wal-append.queue_time_us":128687}
I20260709 13:06:25.374372 25105 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.378049 24866 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:25.378651 24866 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:25.368707 24839 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:25.379586 24839 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:25.379623 24866 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.386866 22584 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:25.387014 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:25.388398 22004 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:25.387400 22007 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:06:25.388849 22007 pending_rounds.cc:85] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:06:25.388597 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.389375 22007 pending_rounds.cc:107] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:06:25.389848 22007 raft_consensus.cc:2889] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:25.394043 22000 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:25.396965 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.397544 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.397948 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.398319 21928 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:25.398648 21932 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:25.397648 21852 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:25.399403 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.399487 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.399988 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:25.400615 22009 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:25.329468 25137 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:25.393713 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.400218 21928 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:25.408953 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.409360 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2065) took 1074 ms. Trace:
I20260709 13:06:25.397595 21926 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:25.400629 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:25.424346 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:25.425154 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.427397 21927 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.427773 21933 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:25.422084 22007 rpcz_store.cc:276] 0709 13:06:24.335165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.398012 (+ 62847us) service_pool.cc:224] Handling call
0709 13:06:24.398332 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.292789 (+894457us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002d4720. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:06:25.408744 (+115955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.409280 (+   536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264035712}
I20260709 13:06:25.429793 21932 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:25.404623 22008 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:25.435242 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:25.436013 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:25.436765 21930 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.438961 24926 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:25.439427 24926 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.439815 24926 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.442669 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:25.443547 21931 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:25.445312 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:25.447075 24978 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:25.447494 24978 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:25.379837 25107 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.400653 24898 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:25.452405 21930 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
W20260709 13:06:25.428823 24943 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:06:25.453903 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.435964 24874 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.478442 21851 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [4.7-7.9]
I20260709 13:06:25.479545 21858 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:25.480401 21930 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:25.433125 25137 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:25.488593 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.489717 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:25.493939 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.502364 21932 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:25.503793 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:25.504703 21930 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.470690 24898 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:25.484257 25069 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:25.522662 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.523701 21853 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:25.524734 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.525590 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.525862 21857 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:06:25.526439 21855 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:25.520943 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.530900 21851 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:25.531772 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:25.519537 21927 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:06:25.535187 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 227335us
I20260709 13:06:25.535514 21859 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:25.506469 24839 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:25.538695 21927 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:25.494153 24874 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:25.548748 21930 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:25.555094 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.555326 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.555979 21930 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:25.521371 25069 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.562410 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.557359 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.564977 21854 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:25.569955 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.478994 24978 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.580930 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.583973 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:25.580956 21972 connection.cc:256] RPC call timeout handler was delayed by -2.5991187189998755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 13:06:25.592650 21972 connection.cc:256] RPC call timeout handler was delayed by -1.8907087779999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 13:06:25.571620 25069 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:25.593250 21972 connection.cc:256] RPC call timeout handler was delayed by -1.8565087779999203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
W20260709 13:06:25.595471 21972 connection.cc:256] RPC call timeout handler was delayed by -1.8565087779999203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
W20260709 13:06:25.595932 21972 connection.cc:256] RPC call timeout handler was delayed by -1.842108777999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 13:06:25.596338 21972 connection.cc:256] RPC call timeout handler was delayed by -1.7377087779999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 13:06:25.596712 21972 connection.cc:256] RPC call timeout handler was delayed by -1.7143087779999178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 13:06:25.597168 21972 connection.cc:256] RPC call timeout handler was delayed by -1.6810087779999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260709 13:06:25.597571 21972 connection.cc:256] RPC call timeout handler was delayed by -1.6351087779999034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:06:25.598078 21972 connection.cc:256] RPC call timeout handler was delayed by -1.5982087779999574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 13:06:25.598482 21972 connection.cc:256] RPC call timeout handler was delayed by -1.5712087779999138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:06:25.598896 21972 connection.cc:256] RPC call timeout handler was delayed by -1.5154087779999372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:06:25.599391 21972 connection.cc:256] RPC call timeout handler was delayed by -1.464108777999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
W20260709 13:06:25.599778 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4056087779998734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
W20260709 13:06:25.600247 21972 connection.cc:256] RPC call timeout handler was delayed by -1.346208777999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 13:06:25.600627 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2976087779999261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 13:06:25.573032 24839 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:25.605633 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:25.606434 21927 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:25.583249 24847 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:25.605232 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:25.592377 25134 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:06:25.614301 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:25.615167 21853 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:25.629379 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65192.3us
W20260709 13:06:25.627635 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53971.6us
I20260709 13:06:25.579269 24874 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.630177 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.589745 25024 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:06:25.633105 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:25.633463 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:25.611377 24847 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.635985 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:25.638823 24847 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:25.639498 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:25.640540 25136 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:25.641186 25117 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:25.640901 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:25.643270 25136 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.644073 25136 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:25.645466 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.648479 25117 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.650249 25117 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:25.653057 24721 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:25.653489 24721 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:25.653878 24721 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.652220 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:25.649384 25140 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:06:25.666538 22002 raft_consensus.cc:2410] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:06:25.645401 24839 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.667418 25120 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:06:25.667755 25144 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:06:25.669580 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.670267 21854 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:25.672200 24874 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.672865 25076 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:25.674335 24926 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:25.676705 25143 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:06:25.681521 25098 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:25.682039 25098 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:25.683008 25098 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.684484 25098 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.687235 24978 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:25.689936 24926 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.692126 22588 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:06:25.631587 25137 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.633606 24898 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.702772 24993 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.703341 24993 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:25.692898 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:25.699090 22588 pending_rounds.cc:85] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 8
I20260709 13:06:25.705399 22588 pending_rounds.cc:107] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:06:25.705734 22588 raft_consensus.cc:2889] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:25.659385 24776 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:25.656837 25081 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:25.714004 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.722620 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.691864 25097 raft_consensus.cc:2764] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:06:25.727149 21857 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:25.731804 21855 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:06:25.727931 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:25.728777 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.732921 22003 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.733450 21860 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:25.731192 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:25.741106 25097 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:25.741825 25097 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:25.743989 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.744628 21856 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:25.745442 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2138) took 1093 ms. Trace:
I20260709 13:06:25.745819 22001 rpcz_store.cc:276] 0709 13:06:24.651622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.954544 (+302922us) service_pool.cc:224] Handling call
0709 13:06:24.954944 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:25.042037 (+ 87093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.042083 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:25.086765 (+ 44682us) log.cc:844] Serialized 26 byte log entry
0709 13:06:25.240913 (+154148us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:25.240973 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.350219 (+109246us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b540042bdd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:25.350264 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:25.729700 (+379436us) raft_consensus.cc:1685] finished
0709 13:06:25.729729 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.745376 (+ 15647us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8393,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":1004,"spinlock_wait_cycles":183186304,"thread_start_us":101137,"threads_started":1,"wal-append.queue_time_us":196653}
I20260709 13:06:25.713297 24751 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:25.711463 24776 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:25.769173 21853 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:25.770936 21853 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:06:25.768419 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:25.777578 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.780777 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:25.782021 21860 raft_consensus.cc:2410] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260709 13:06:25.756808 25102 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.000s	sys 0.015s
W20260709 13:06:25.786801 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 134673us
I20260709 13:06:25.799670 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:25.795526 22003 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:25.803923 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:25.713305 25076 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:06:25.819334 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:25.760304 24776 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:25.806128 25102 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.001s	sys 0.015s
I20260709 13:06:25.779968 24788 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:25.817826 25007 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:06:25.829824 25141 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:06:25.830715 25106 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.838907 25081 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:25.824958 25097 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:25.852250 25145 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:06:25.851164 25023 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:25.824069 24926 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:25.887216 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 162386us
I20260709 13:06:25.886621 24993 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:25.952821 25097 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.963598 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:25.970348 22585 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:25.977447 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:25.946616 25023 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:25.997303 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 195731us
I20260709 13:06:26.004748 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:25.954483 24751 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:25.955372 24926 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.028690 25101 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:26.031302 25159 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:26.046356 25141 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:06:26.047868 25109 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.071s	user 0.004s	sys 0.000s
I20260709 13:06:26.053830 25101 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:26.063479 25094 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.698s	user 0.016s	sys 0.013s
I20260709 13:06:26.064141 25101 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.034425 24847 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:26.070632 25142 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
I20260709 13:06:26.071139 24847 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.073997 25168 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:26.074676 25168 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:26.075702 25168 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:26.081540 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.048149 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 228799us
I20260709 13:06:26.089725 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.098013 22585 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:26.059321 25159 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:26.116747 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.118952 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.121388 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:26.125118 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:26.209987 25163 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:06:26.117548 25159 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:26.240361 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.106091 25069 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.240808 24776 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.241602 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.242615 25069 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.242748 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.242937 24776 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.243721 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.243965 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.244577 22000 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:26.245573 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.245810 22009 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:26.246778 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:26.247337 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.247428 22009 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:26.231125 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109979us
I20260709 13:06:26.231431 25170 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.244907 22006 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:26.249300 25170 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.250458 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:26.251205 25149 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.614s	user 0.001s	sys 0.009s
I20260709 13:06:26.251621 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.251720 25170 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:26.249342 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.253731 22009 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:26.253971 25146 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.606s	user 0.005s	sys 0.011s
I20260709 13:06:26.254261 25165 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:26.255177 25165 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:26.255492 25165 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:26.256709 24721 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.000s
I20260709 13:06:26.254302 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.231896 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.239945 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.201116 25177 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.237136 25179 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:26.237920 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.173763 24751 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:26.238353 25105 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:26.238898 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.198575 25173 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:26.120462 25023 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:26.232595 25171 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:06:26.249173 25007 raft_consensus.cc:1068] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:06:26.265440 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 259813us
I20260709 13:06:26.266234 22007 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:26.256400 25165 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.268034 22000 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:26.269683 22005 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:26.306376 22004 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:26.284624 22001 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:26.297269 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:26.263192 24721 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.317833 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.319779 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:26.302910 24776 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:26.324271 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:26.327306 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.331346 25154 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:26.332095 25154 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.268622 25177 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.322705 25173 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.321797 25165 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:26.337709 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:26.343708 25157 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.380s	user 0.014s	sys 0.000s
I20260709 13:06:26.340001 24751 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.273036 25179 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.350900 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:26.337738 25081 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.353324 24839 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:26.353925 24839 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:26.330729 25184 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:26.354964 24839 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:26.364063 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 671175us
W20260709 13:06:26.367380 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.343043 25170 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:26.329231 24776 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.355424 25179 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:26.364775 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 346033us
W20260709 13:06:26.374941 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.376586 21852 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:26.352183 25169 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:26.364079 25108 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.004s
I20260709 13:06:26.382747 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.384341 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:26.387245 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:26.376319 24800 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:26.404395 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 621) took 1108 ms. Trace:
I20260709 13:06:26.404938 22002 rpcz_store.cc:276] 0709 13:06:25.294340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.294502 (+   162us) service_pool.cc:224] Handling call
0709 13:06:26.402293 (+1107791us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54003890d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:26.402422 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850379520}
I20260709 13:06:26.378278 25154 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.382299 25169 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.382678 25120 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:26.402696 24978 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.000s
I20260709 13:06:26.404785 24800 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.405381 24948 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.417543 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:26.418735 21859 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:26.420790 24776 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.422783 25177 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.417164 25097 raft_consensus.cc:2764] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:26.425290 24721 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.416411 24800 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:26.427476 25159 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.435045 25170 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:26.411792 25169 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:26.383460 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 56215.3us
I20260709 13:06:26.446380 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.415275 25120 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.448424 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 786) took 1089 ms. Trace:
I20260709 13:06:26.455370 21852 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:26.449043 22591 rpcz_store.cc:276] 0709 13:06:25.359404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.359591 (+   187us) service_pool.cc:224] Handling call
0709 13:06:25.359947 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.360198 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:25.586886 (+226688us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.447518 (+860632us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b1800453400. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf 00007f3b707e9038
0709 13:06:26.447789 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.447999 (+   210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.448060 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:26.448120 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.448192 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.448345 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":975693,"prepare.queue_time_us":210101,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":9179904,"thread_start_us":78491,"threads_started":1}
I20260709 13:06:26.466378 24800 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.471002 24800 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.487141 25159 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:26.450989 25024 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:26.488210 25024 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.490550 25024 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.461032 25170 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:26.492899 25120 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:26.493302 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.499962 22587 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:06:26.501407 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 99684.4us
I20260709 13:06:26.508890 21859 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:26.510983 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 785) took 1879 ms. Trace:
I20260709 13:06:26.511420 22588 rpcz_store.cc:276] 0709 13:06:24.631317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.631468 (+   151us) service_pool.cc:224] Handling call
0709 13:06:24.710017 (+ 78549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.706215 (+996198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.510802 (+804587us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b54004cca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:26.510938 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183548672}
I20260709 13:06:26.513185 25101 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:26.527704 25131 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.530745 24759 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.537042 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:26.537292 25068 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:26.537375 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:26.537833 25068 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.538360 21932 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:26.539433 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:26.541158 24721 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:26.528354 25131 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.549464 25068 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:26.551964 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 91803.8us
I20260709 13:06:26.552592 25190 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:26.552834 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:26.553435 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:26.549214 24721 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.553763 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:26.555534 24721 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.557785 25173 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:26.556594 25174 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.450s	user 0.010s	sys 0.000s
W20260709 13:06:26.558208 25176 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:06:26.557332 25180 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:26.559278 25180 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.559463 25131 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:26.559679 25180 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:26.540525 25107 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:26.560164 25185 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.240s	user 0.001s	sys 0.001s
I20260709 13:06:26.561610 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:26.562317 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 543939us
I20260709 13:06:26.553242 25190 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.564152 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:26.551831 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49439.6us
I20260709 13:06:26.565820 21926 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:26.566103 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:26.575541 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:26.584052 21856 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:26.585172 21852 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:26.585670 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.583843 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:26.583086 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:26.589394 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:26.588937 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:26.590291 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:26.590641 21852 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:26.591020 21857 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:26.576582 25099 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:26.575580 25165 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:26.593890 21856 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:26.591003 21926 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:26.599009 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:26.599766 21852 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:26.602562 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:26.563208 25169 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.611971 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2148) took 1217 ms. Trace:
I20260709 13:06:26.603086 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:26.627347 22590 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:26.612357 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.591287 25136 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:06:26.612433 24948 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:06:26.601994 25165 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.631680 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:26.648041 21930 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:06:26.594076 24495 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.618914 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.620621 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54627.4us
I20260709 13:06:26.621883 21926 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:26.592583 25117 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.598250 25113 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.596462 25099 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:26.598474 24898 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:26.651854 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.722512 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 90902.3us
I20260709 13:06:26.783955 25195 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:26.631724 21899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 1.553s (SENT) [suppressed 19 similar messages]
I20260709 13:06:26.720763 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:26.721621 21933 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:26.614557 21929 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:26.724248 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:26.633383 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:26.632221 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:26.725345 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 70727.4us
I20260709 13:06:26.734455 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:26.703027 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.814904 22588 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:26.722960 25188 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:06:26.752386 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:26.643409 25165 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:26.793630 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:26.816407 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 92230.7us
I20260709 13:06:26.855659 21930 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:26.632750 25190 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:26.629406 25202 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.857496 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.722849 24993 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:26.818218 21927 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
W20260709 13:06:26.823705 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 71510us
I20260709 13:06:26.856223 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:26.823333 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88973.1us
I20260709 13:06:26.824672 25076 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.824822 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:26.868762 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.869596 21933 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:26.869979 22005 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:26.825866 21932 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:26.825629 25173 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:26.828413 24978 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:26.816411 25206 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:26.644008 24847 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.841738 21926 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:26.879403 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.631914 25007 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:26.637612 25136 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:26.844857 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.846840 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.848364 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:26.881310 22007 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:26.881443 22006 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:26.726263 24495 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.860508 25112 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:06:26.869742 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.649286 25108 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.786338 25099 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:26.733152 25197 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:26.898370 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79995.1us
I20260709 13:06:26.644930 25159 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 13:06:26.862063 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:26.909090 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:26.877307 25104 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.793545 25113 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:26.721735 25097 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.205s
I20260709 13:06:26.878438 24898 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:26.918412 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56436.5us
I20260709 13:06:26.919554 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:26.924504 21927 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:26.924610 22007 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:26.886813 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:26.903829 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 320802us
I20260709 13:06:26.904088 22009 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:26.864228 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:26.736045 25117 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.909221 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:26.929395 21742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65206.6us
I20260709 13:06:26.929639 22005 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:26.902974 25136 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:26.913255 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 281537us
I20260709 13:06:26.902537 22002 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:26.902570 25206 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:26.945312 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:26.945976 21926 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:26.946199 25206 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.947108 21926 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.947028 25137 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:26.947827 25137 consensus_queue.cc:1243] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:06:26.948575 25175 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:26.948984 25137 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:26.949823 25205 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:06:26.951836 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:26.953572 25162 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.907s	user 0.004s	sys 0.000s
I20260709 13:06:26.955915 25103 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:06:26.956413 25103 raft_consensus.cc:1068] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:26.921976 25007 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:06:26.957790 25098 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:26.906721 25108 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.961690 25108 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:26.964268 25180 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.907631 25197 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.931903 25194 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:26.965205 25197 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:26.965548 25194 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:26.966542 25194 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.973078 25155 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.973711 25155 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.977372 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:26.931535 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44822.3us
I20260709 13:06:26.931612 22002 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:26.930807 25190 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.933622 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.934122 25131 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:26.934541 24948 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.904989 24978 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:26.941538 25169 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.941897 24800 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:26.901625 24847 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:26.942860 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:26.922549 25195 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.922695 25104 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:26.945215 24751 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:26.998531 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:27.001873 21926 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:27.013430 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:27.029382 22005 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:27.015084 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:27.015407 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:27.018186 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:27.019218 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.034884 21860 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:27.006737 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 447) took 1246 ms. Trace:
I20260709 13:06:27.022629 21851 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:27.023957 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:26.984650 21927 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:27.025149 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:26.980211 22002 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.019959 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:27.048722 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:26.983009 24898 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:27.048611 25210 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:27.068193 22007 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:27.068362 24898 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.072139 25104 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.006876 24948 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.076296 24948 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:26.987139 25190 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.022950 24751 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:27.077883 24751 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.078508 25190 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.081565 21927 pending_rounds.cc:85] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 3
I20260709 13:06:27.081923 21927 pending_rounds.cc:107] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 13:06:27.082238 21927 raft_consensus.cc:2917] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:06:26.987457 25098 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:26.998226 25068 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:26.944792 25107 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:06:26.976112 25155 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:26.987979 25101 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:27.037360 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.098563 21853 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:26.979054 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52733.7us
I20260709 13:06:27.050024 22003 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:27.049402 22005 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.055860 22006 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:27.104070 22006 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:27.056517 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:26.980449 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:26.996582 25131 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.026068 25099 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.026216 25195 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.023921 24800 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.001171 24721 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:06:27.067090 24847 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:26.996388 21858 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:27.039516 21854 rpcz_store.cc:276] 0709 13:06:25.760764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.852325 (+ 91561us) service_pool.cc:224] Handling call
0709 13:06:25.852782 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:25.889136 (+ 36354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.889180 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:25.889749 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:06:26.095999 (+206250us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:26.119288 (+ 23289us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.624294 (+505006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.006331 (+382037us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:27.006379 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:27.006423 (+    44us) raft_consensus.cc:1685] finished
0709 13:06:27.006527 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.006678 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":503620,"prepare.queue_time_us":24828,"prepare.run_cpu_time_us":5061,"prepare.run_wall_time_us":96893,"raft.queue_time_us":637179,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":62083,"spinlock_wait_cycles":82513792,"thread_start_us":165152,"threads_started":2,"wal-append.queue_time_us":291777}
I20260709 13:06:27.082742 21927 raft_consensus.cc:2889] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.083099 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:27.094581 22008 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:27.054808 24495 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:27.105563 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49155.9us
I20260709 13:06:27.057780 25197 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.146718 25197 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.109889 25101 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:27.147434 25101 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:27.154402 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:27.155102 22006 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.112301 25190 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:06:27.124003 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:27.111373 24866 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.004469 25113 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:27.176345 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72066.1us
I20260709 13:06:27.012251 24926 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:27.007901 25180 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.076886 24948 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.021837 25209 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.142037 21857 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.129806 25155 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:27.164767 25076 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:27.169929 21932 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:27.173985 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.181679 22005 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:27.120004 25104 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:27.305968 25104 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.308419 25104 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.159992 21926 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.091427 25007 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.090s
W20260709 13:06:27.204802 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 271234us
I20260709 13:06:27.156893 22006 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.009104 25136 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.188009 24495 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.324523 21926 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:27.127758 25211 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:06:27.186736 24721 consensus_queue.cc:1243] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:27.329496 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:27.141996 25107 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.161s
I20260709 13:06:27.175441 25205 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.303586 25076 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.167361 25103 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.157s
I20260709 13:06:27.174813 25190 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.156s
I20260709 13:06:27.342110 21926 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:27.332990 24926 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:27.339908 25169 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:27.192910 24948 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.358894 25099 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:27.317723 25155 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.364907 22006 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:06:27.376204 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 652) took 1290 ms. Trace:
I20260709 13:06:27.378203 25183 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.378348 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 795) took 1144 ms. Trace:
I20260709 13:06:27.390225 25136 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:27.353616 21857 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:27.190444 24866 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:27.474941 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.502933 22006 pending_rounds.cc:85] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:27.504594 22006 pending_rounds.cc:107] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:06:27.503609 22584 rpcz_store.cc:276] 0709 13:06:26.233990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.234419 (+   429us) service_pool.cc:224] Handling call
0709 13:06:26.234755 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.377753 (+1142998us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b2c001f0f78. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6ea7f291 00007f3b6ea9a0ca 00007f3b707c84ad 00007f3b707cee1b 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:06:27.377893 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.377938 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.378002 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:27.378048 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.378154 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.378290 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3490560}
I20260709 13:06:27.504963 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:27.507840 22006 raft_consensus.cc:2889] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.394249 25076 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:27.507674 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:27.502965 22585 rpcz_store.cc:276] 0709 13:06:26.085800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.085946 (+   146us) service_pool.cc:224] Handling call
0709 13:06:26.091833 (+  5887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.098645 (+  6812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.375699 (+1277054us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b5400530cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:27.376091 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13595008}
I20260709 13:06:27.511703 22588 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:06:27.396587 25113 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.315696 24978 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.342128 24495 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.335703 25203 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.173s
I20260709 13:06:27.468080 25169 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.409417 25007 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.395709 25097 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.468344 24926 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.315344 25209 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.391881 25205 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.523424 25169 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.533262 25007 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.552364 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2081) took 1312 ms. Trace:
I20260709 13:06:27.552791 22004 rpcz_store.cc:276] 0709 13:06:26.239693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.246789 (+  7096us) service_pool.cc:224] Handling call
0709 13:06:26.247049 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.306797 (+ 59748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.552115 (+1245318us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b5400166054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:27.552302 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135639296}
W20260709 13:06:27.537395 25194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.013s	sys 0.025s
I20260709 13:06:27.546156 25155 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:27.546723 24800 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:27.532977 24926 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:27.581995 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 426143us
W20260709 13:06:27.537578 25179 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.034s
I20260709 13:06:27.520720 25113 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.516234 25183 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.534655 25097 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.584663 25131 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:27.573978 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 593514us
I20260709 13:06:27.736678 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:27.392941 25098 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:06:27.580328 25194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.013s	sys 0.026s
I20260709 13:06:27.610356 25209 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:27.585194 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 407935us
W20260709 13:06:27.608528 24751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.007s	sys 0.008s
I20260709 13:06:27.754951 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:27.755874 25218 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:27.756788 24751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.010s	sys 0.010s
I20260709 13:06:27.757526 21933 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:27.758042 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.021s	sys 0.030s
W20260709 13:06:27.758989 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.021s	sys 0.030s
W20260709 13:06:27.758834 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 149659us
I20260709 13:06:27.760695 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:27.608371 25179 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.006s	sys 0.035s
I20260709 13:06:27.757557 25218 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:27.782095 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:27.782716 21857 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.781302 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:27.791133 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.794668 25225 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:27.789572 25180 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:27.795547 25225 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:27.795667 25180 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.770241 25102 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.797547 25102 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.798475 25137 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.798995 25137 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.800284 25102 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.801877 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.802089 21854 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:27.803649 25204 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:27.803843 25179 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:27.804735 25137 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.798015 25180 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.805572 25211 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:27.805784 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:27.806093 25211 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.806969 25023 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.808269 25211 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.777751 24948 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:27.809191 25137 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:27.764360 24751 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:27.781579 25101 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.000s
I20260709 13:06:27.811223 25180 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.811972 25180 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.811801 25101 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.815163 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.806387 25097 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.819648 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.826293 21858 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:27.827809 24495 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.827862 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:27.775141 25218 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.829152 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.808841 25211 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:27.829244 24866 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.830940 22002 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:27.846447 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.836817 25102 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.829912 25137 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:27.834967 25218 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.850662 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.854655 21857 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:27.860226 21852 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:27.849603 25211 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:27.865970 25211 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.810983 24751 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.849156 25097 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:27.868052 25183 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:27.872057 21860 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.020s	sys 0.051s
W20260709 13:06:27.874022 21860 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.020s	sys 0.051s
I20260709 13:06:27.860165 25137 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.895771 25137 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:27.897673 25007 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.855742 25102 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.903679 22588 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:27.870916 25205 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:27.796905 25225 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:27.922366 22002 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:06:27.922892 22002 pending_rounds.cc:85] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 4
I20260709 13:06:27.923210 22002 pending_rounds.cc:107] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:06:27.923758 22002 raft_consensus.cc:2889] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.925420 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2089) took 1076 ms. Trace:
I20260709 13:06:27.927438 22002 rpcz_store.cc:276] 0709 13:06:26.849092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.902106 (+ 53014us) service_pool.cc:224] Handling call
0709 13:06:26.902419 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.828677 (+926258us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b580058c4d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7072b0aa 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4
0709 13:06:27.925149 (+ 96472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.925365 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765554560}
W20260709 13:06:27.933851 25222 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:27.934334 25107 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:27.939584 25173 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:27.943225 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.981101 21858 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:27.900410 25097 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:27.945030 24943 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.945555 25180 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:27.930353 25113 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:27.988715 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 224444us
I20260709 13:06:27.949717 25007 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.991174 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.995468 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.017184 22004 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:28.019326 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:26.626078 21934 rpcz_store.cc:276] 0709 13:06:25.383692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.383805 (+   113us) service_pool.cc:224] Handling call
0709 13:06:25.397775 (+ 13970us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:25.435387 (+ 37612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.435421 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:25.436068 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:06:26.258841 (+822773us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b18004a2300. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:06:26.415731 (+156890us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:26.475006 (+ 59275us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.550192 (+ 75186us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.596926 (+ 46734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.597040 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:26.601086 (+  4046us) raft_consensus.cc:1685] finished
0709 13:06:26.601110 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.601237 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142308,"prepare.queue_time_us":60351,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":99,"raft.queue_time_us":146001,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":47015,"spinlock_wait_cycles":40820096,"thread_start_us":212072,"threads_started":2,"wal-append.queue_time_us":992967}
I20260709 13:06:27.989497 25205 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.980463 25107 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:28.023429 22588 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:28.025239 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:27.983302 25173 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:28.028829 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:28.029279 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:28.028396 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1820) took 1008 ms. Trace:
I20260709 13:06:28.031755 22588 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:28.034916 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.036337 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:28.038615 22588 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:28.040297 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:28.040477 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 285606us
I20260709 13:06:28.040948 21858 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.041731 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:28.031687 21860 rpcz_store.cc:276] 0709 13:06:27.020138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.050552 (+ 30414us) service_pool.cc:224] Handling call
0709 13:06:27.050951 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.051200 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:27.874565 (+823365us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.028096 (+153531us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:28.028142 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:28.028180 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.028251 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.028344 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10274,"prepare.queue_time_us":869990,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1645,"raft.queue_time_us":10376,"thread_start_us":918410,"threads_started":2}
I20260709 13:06:28.053815 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:28.058539 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:28.066498 21851 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:06:27.990783 24943 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.050457 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2039) took 1680 ms. Trace:
I20260709 13:06:28.065280 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.080189 25219 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.050438 24898 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:28.080806 25219 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:28.081992 25230 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:28.082334 25102 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:28.082923 25112 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:28.083735 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 799) took 1266 ms. Trace:
I20260709 13:06:28.084937 24943 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.050635 24866 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:28.085829 24866 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.026834 25007 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.029803 25205 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.023262 25169 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:28.344072 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 322381us
W20260709 13:06:28.259656 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.332289 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.159584 21858 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:28.469794 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.084113 22590 rpcz_store.cc:276] 0709 13:06:26.817497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.817622 (+   125us) service_pool.cc:224] Handling call
0709 13:06:26.817947 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.818859 (+   912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.818886 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:26.819369 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:06:27.372483 (+553114us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b1800364cc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:06:27.833846 (+461363us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:27.833914 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.834007 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.083433 (+249426us) raft_consensus.cc:1685] finished
0709 13:06:28.083452 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.083674 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":529,"raft.queue_time_us":4194,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":197,"spinlock_wait_cycles":33010560,"thread_start_us":460627,"threads_started":1,"wal-append.queue_time_us":1125049}
W20260709 13:06:28.471798 24978 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.012s	sys 0.008s
I20260709 13:06:28.478653 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:28.479437 21858 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:28.487759 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:28.499261 25226 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:28.502898 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 373228us
W20260709 13:06:28.103168 25076 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.000s
W20260709 13:06:28.093266 25136 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.584s	user 0.005s	sys 0.005s
I20260709 13:06:28.504973 22591 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:06:28.493144 24978 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.012s	sys 0.008s
W20260709 13:06:28.442374 25218 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.001s
I20260709 13:06:28.091661 25159 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:28.506601 25232 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:06:28.506675 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.637068 22587 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:28.494524 25190 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:28.507364 25183 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:28.508301 22585 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:28.086243 25097 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:28.095661 25236 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:28.511297 21859 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:28.532119 25112 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:28.512773 25007 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:28.514199 25226 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:28.514370 25205 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:28.516542 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1765) took 1594 ms (client timeout 1646 ms). Trace:
I20260709 13:06:28.665315 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.525245 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.666133 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:28.666386 22589 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:28.504856 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:28.665228 21932 rpcz_store.cc:269] 0709 13:06:26.922619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.923879 (+  1260us) service_pool.cc:224] Handling call
0709 13:06:28.516383 (+1592504us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54004295d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:28.516485 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499944832}
I20260709 13:06:28.656252 25236 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.669601 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:28.670411 21932 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:28.528990 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:28.380685 24898 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:28.534132 24788 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:28.672082 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 143144us
I20260709 13:06:28.672194 25103 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:28.545217 25131 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:28.672803 25103 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.672475 24788 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.673324 25103 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:28.548220 24759 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.674078 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:28.674015 24759 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:28.675196 21932 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:28.675241 25106 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:28.676194 24759 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.675961 25106 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.676204 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:28.553115 22593 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:28.553622 25194 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.679992 25194 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.682727 25194 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.553767 25229 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.558346 25216 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:28.683686 25229 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.685138 25236 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.686381 25229 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.686503 25103 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.687400 21932 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:28.688665 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:28.568384 21929 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:28.690251 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:28.579185 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2160) took 1951 ms. Trace:
I20260709 13:06:28.602412 25069 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.691649 25103 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:28.691339 25069 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.693913 25103 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:28.622712 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 548) took 1971 ms. Trace:
I20260709 13:06:28.606873 24721 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.694067 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:28.695034 24721 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.695983 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.696780 25102 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:28.697547 25102 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:28.425499 25173 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.698542 25173 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:28.697141 21928 rpcz_store.cc:276] 0709 13:06:26.628211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.628327 (+   116us) service_pool.cc:224] Handling call
0709 13:06:26.651024 (+ 22697us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:27.066041 (+415017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.066075 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:27.066527 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:06:27.536358 (+469831us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:27.536404 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.578650 (+1042246us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54000979d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:28.578718 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.578778 (+    60us) raft_consensus.cc:1685] finished
0709 13:06:28.578798 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.579105 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16573,"raft.queue_time_us":51512,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":39302,"spinlock_wait_cycles":953575808,"thread_start_us":469462,"threads_started":1,"wal-append.queue_time_us":698438}
I20260709 13:06:28.561435 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:28.086611 24751 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.343364 25239 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:28.700652 25239 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:28.701184 21929 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:28.067692 21855 rpcz_store.cc:276] 0709 13:06:26.370503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.372578 (+  2075us) service_pool.cc:224] Handling call
0709 13:06:26.379354 (+  6776us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:26.877951 (+498597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.877999 (+    48us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:26.881570 (+  3571us) log.cc:844] Serialized 78 byte log entry
0709 13:06:27.177306 (+295736us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:27.776573 (+599267us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.038328 (+261755us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.039294 (+   966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.039363 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.039412 (+    49us) raft_consensus.cc:1685] finished
0709 13:06:28.039424 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.050397 (+ 10973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203734,"prepare.queue_time_us":773043,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":7185,"raft.queue_time_us":253412,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":65051,"thread_start_us":1263408,"threads_started":4,"wal-append.queue_time_us":394130}
I20260709 13:06:28.702258 24866 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.633816 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.702836 25239 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.703027 24721 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:28.703348 25137 consensus_queue.cc:1175] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 5 status { last_received { term: 5 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 5 index: 8 } }
I20260709 13:06:28.703851 24721 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:28.704523 24721 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.530443 25076 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.705133 24721 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.694684 21930 rpcz_store.cc:276] 0709 13:06:26.650036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.866654 (+216618us) service_pool.cc:224] Handling call
0709 13:06:26.883058 (+ 16404us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:26.902883 (+ 19825us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:26.912292 (+  9409us) log.cc:844] Serialized 14 byte log entry
0709 13:06:27.063199 (+150907us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b4800241ed8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:06:27.188621 (+125422us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:27.195552 (+  6931us) log.cc:844] Serialized 78 byte log entry
0709 13:06:27.195726 (+   174us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:27.195992 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 13:06:27.314836 (+118844us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.509204 (+1194368us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b1800397f00. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:28.538379 (+ 29175us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b1800397f00. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:06:28.620962 (+ 82583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.621230 (+   268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.621372 (+   142us) raft_consensus.cc:1685] finished
0709 13:06:28.621393 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.621789 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1359099,"prepare.queue_time_us":172924,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":1381283,"raft.run_cpu_time_us":2939,"raft.run_wall_time_us":278899,"spinlock_wait_cycles":98472960,"thread_start_us":222603,"threads_started":1,"wal-append.queue_time_us":589806,"wal-append.run_cpu_time_us":2735,"wal-append.run_wall_time_us":1009063}
I20260709 13:06:28.707176 25076 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.707944 21930 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:28.638497 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:28.703693 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 142421us
I20260709 13:06:28.709470 21932 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:28.710873 21930 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:28.636034 25218 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.714392 25218 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.716593 24721 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.729866 21932 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:06:28.566664 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:28.734743 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 168230us
I20260709 13:06:28.630577 22588 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:28.512333 25202 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:06:28.090402 24926 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.629240 25159 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:28.541129 25108 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:28.673082 25131 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.546258 24800 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.548964 24948 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:28.761541 24948 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.763368 24721 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.764375 24948 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.677970 24788 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.678256 25106 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.549755 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:28.765225 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:28.765823 21934 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:28.678471 25007 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:28.767160 25007 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.769891 25007 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.694007 25069 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.771148 24800 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.774034 24800 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.381265 25107 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.775054 25107 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:28.755486 24978 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.779217 24978 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.785281 24926 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.786381 22584 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:28.755220 24759 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:06:28.790575 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.742969 25229 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:28.793984 25159 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:28.665859 25112 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.798491 25197 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:28.638087 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.800299 22586 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:28.652105 21859 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:06:28.722364 25101 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:28.657244 25179 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:28.765903 22591 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.740370 25232 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.808691 25112 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:28.809260 25232 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.810135 25131 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.810282 25103 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.810783 25103 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.811885 25232 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:28.812340 25106 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.812846 25103 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:28.814093 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1766) took 1891 ms. Trace:
I20260709 13:06:28.814752 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.814462 21933 rpcz_store.cc:276] 0709 13:06:26.922974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.923083 (+   109us) service_pool.cc:224] Handling call
0709 13:06:26.957360 (+ 34277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.758010 (+800650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.813879 (+1055869us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b540017eb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:28.814040 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839533440}
I20260709 13:06:28.815414 22001 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:28.817549 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:28.649212 25190 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:28.803165 25197 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:28.758282 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.819983 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.822559 22002 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:28.826581 22588 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:28.826884 25197 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.828121 25197 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:28.805903 25159 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:28.850835 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 141393us
I20260709 13:06:28.816138 25202 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:28.812783 25106 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.833225 25229 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:28.642976 22558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.004s	sys 0.009s
I20260709 13:06:28.865588 22588 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:28.865955 22558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.004s	sys 0.009s
I20260709 13:06:28.871328 25202 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:28.879427 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:28.880317 22587 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:28.884013 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.885465 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:28.816843 25179 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:28.900692 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 428391us
W20260709 13:06:28.902602 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 414902us
I20260709 13:06:28.903084 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.904235 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.887581 24978 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.918601 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.919553 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:28.932535 22005 raft_consensus.cc:2368] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:28.921381 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.933769 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:28.919287 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.923838 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.933815 22003 raft_consensus.cc:2410] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:28.921104 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.936403 22008 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:28.938036 21824 leader_election.cc:400] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:28.938462 21824 leader_election.cc:403] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:06:28.942965 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:28.943293 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.946364 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:28.948210 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 1.62777e+06us
I20260709 13:06:28.944813 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.915441 25202 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:28.897912 25120 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:28.961196 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.951761 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:28.969539 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.970484 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:28.971807 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.887002 25131 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:06:28.905268 25235 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:29.015411 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45921.6us
W20260709 13:06:29.015467 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45145.6us
I20260709 13:06:28.969882 22008 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:29.017266 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.905470 25182 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:28.983242 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.928059 25098 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.949990 25239 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:28.928725 25226 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:28.913880 25232 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:06:28.974004 24721 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:29.058296 21927 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.023s	sys 0.071s
I20260709 13:06:29.018518 25120 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.059473 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:28.913389 25119 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
W20260709 13:06:29.092561 21927 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.024s	sys 0.073s
W20260709 13:06:29.092896 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 416259us
I20260709 13:06:29.026666 24847 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:28.975832 25076 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:06:28.973752 25069 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:06:29.060091 24751 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:29.099677 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:29.131559 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 229888us
I20260709 13:06:29.131987 21855 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:28.968366 25248 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.145s
W20260709 13:06:29.095908 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.007s	sys 0.029s
W20260709 13:06:29.135098 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.007s	sys 0.029s
I20260709 13:06:29.097856 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:29.057567 25206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.689s	user 0.023s	sys 0.023s
I20260709 13:06:29.136155 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.138965 21929 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.095948 25120 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:29.147241 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49464.6us
I20260709 13:06:29.064555 25256 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:29.061884 25098 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.067502 24800 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:29.068392 25226 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.168366 24788 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:29.184149 21853 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:06:29.062439 25165 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.677s	user 0.032s	sys 0.049s
I20260709 13:06:29.174186 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:29.100018 25131 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:29.025111 25229 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.207482 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.168263 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:29.146500 25206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.778s	user 0.024s	sys 0.023s
I20260709 13:06:29.190965 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:29.216514 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.130998 25097 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.130652 25139 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.142637 24751 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.230996 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:29.175272 24866 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:29.207047 25165 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.822s	user 0.033s	sys 0.049s
I20260709 13:06:29.238808 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.239152 21931 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:06:29.179204 24800 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:29.239694 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.264592 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:29.187752 25190 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:29.263032 21933 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:29.184075 25101 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:29.240350 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.149395 25218 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:06:29.229683 24751 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.274230 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.274868 21933 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:29.277715 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.278465 21933 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:29.289727 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.216238 24788 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.299347 24788 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:29.300625 21932 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:29.301152 25106 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.301734 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.302345 21932 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:29.306011 25098 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.309172 21930 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:06:29.309027 25238 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:06:29.309746 25179 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:29.311340 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2050) took 1526 ms. Trace:
I20260709 13:06:29.311714 21854 rpcz_store.cc:276] 0709 13:06:27.785348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.803169 (+ 17821us) service_pool.cc:224] Handling call
0709 13:06:27.803595 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:27.803995 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.804034 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:27.804737 (+   703us) log.cc:844] Serialized 52 byte log entry
0709 13:06:28.040128 (+235391us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:28.478961 (+438833us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.479177 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.479245 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.967762 (+488517us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b180020e6c0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:29.311055 (+343293us) raft_consensus.cc:1685] finished
0709 13:06:29.311151 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.311283 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":432888,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":7859,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":12099,"spinlock_wait_cycles":233330560,"thread_start_us":219470,"threads_started":1,"wal-append.queue_time_us":721837}
I20260709 13:06:29.313654 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.314044 25226 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.315393 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.239554 25097 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.322013 25097 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:29.309027 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.322806 21926 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:29.277087 25101 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.325109 25101 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.232868 25139 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.268021 25131 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:29.326876 25253 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:06:29.265779 24866 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:29.329730 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.331573 24866 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:29.332993 24948 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:29.333086 25256 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:29.333666 24948 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:29.334064 25256 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:29.334156 24948 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:29.195281 25239 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:06:29.355659 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.365701 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.365067 21929 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:29.366408 21933 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
W20260709 13:06:29.277335 25173 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.000s
I20260709 13:06:29.386528 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.388504 22000 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:29.376533 21932 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
W20260709 13:06:29.395223 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.404240 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1826) took 1395 ms. Trace:
I20260709 13:06:29.404865 21852 rpcz_store.cc:276] 0709 13:06:28.009186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.574551 (+565365us) service_pool.cc:224] Handling call
0709 13:06:28.575029 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:28.575526 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:28.575566 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:28.576325 (+   759us) log.cc:844] Serialized 52 byte log entry
0709 13:06:29.060132 (+483807us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:29.060198 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.060311 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:29.403923 (+343612us) raft_consensus.cc:1685] finished
0709 13:06:29.403952 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.404139 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13574,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":483338,"threads_started":1,"wal-append.queue_time_us":819546}
I20260709 13:06:29.367556 25256 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:29.407835 25256 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.326938 25139 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:29.374397 24800 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.413022 24800 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:29.419519 25219 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:29.427631 25269 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:29.385125 25117 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:29.441659 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 210729us
I20260709 13:06:29.443295 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:29.446841 25279 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:29.450544 25279 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:29.450919 25279 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:29.447418 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.455027 25195 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:29.450054 25255 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:06:29.455704 25273 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:29.457285 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.459425 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:29.460263 25266 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.185s	user 0.003s	sys 0.000s
I20260709 13:06:29.466483 24948 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.473067 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1827) took 1461 ms. Trace:
I20260709 13:06:29.473533 21857 rpcz_store.cc:276] 0709 13:06:28.011370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.611108 (+599738us) service_pool.cc:224] Handling call
0709 13:06:28.611572 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:28.737113 (+125541us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:28.752925 (+ 15812us) log.cc:844] Serialized 14 byte log entry
0709 13:06:29.083272 (+330347us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:29.083326 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:29.083372 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.471564 (+388192us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b18003722e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419
0709 13:06:29.472806 (+  1242us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:29.472854 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.472975 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2052,"prepare.queue_time_us":15507,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":1555,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5976,"spinlock_wait_cycles":310924544,"thread_start_us":328130,"threads_started":1,"wal-append.queue_time_us":672598}
I20260709 13:06:29.475713 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:29.476284 21857 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:29.477082 25182 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:29.481143 25107 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.005s
I20260709 13:06:29.482982 25108 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.483525 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1821) took 2463 ms. Trace:
I20260709 13:06:29.468712 24948 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:29.484055 21851 rpcz_store.cc:276] 0709 13:06:27.020414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.084878 (+ 64464us) service_pool.cc:224] Handling call
0709 13:06:27.097786 (+ 12908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:28.075990 (+978204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.483189 (+1407199us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b5400259c54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:29.483418 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223027072}
I20260709 13:06:29.489487 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.490654 21854 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:06:29.498993 21858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.012s	sys 0.020s
W20260709 13:06:29.499369 21858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.012s	sys 0.020s
I20260709 13:06:29.503376 22585 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:29.503391 21932 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:29.506341 21933 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:29.507431 21851 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:06:29.508128 21857 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:29.483552 25108 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.490643 25216 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.514202 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.519193 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.520231 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.517891 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.527257 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.528369 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.530512 25108 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.530902 24751 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:29.543007 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.543469 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.543918 25120 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:29.545982 25159 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.705s	user 0.010s	sys 0.000s
I20260709 13:06:29.552053 22006 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:29.552776 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:29.546905 25282 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.553799 25282 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.554502 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:29.555907 25282 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:29.548040 25273 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.549621 25069 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:29.554112 24800 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:29.556602 25273 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.562426 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.516417 25216 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:29.595005 25257 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:06:29.596901 21933 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:29.606992 25120 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.606835 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:29.608603 22589 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.609586 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.610682 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.614053 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 553) took 1067 ms. Trace:
I20260709 13:06:29.614646 21851 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:29.614430 21928 rpcz_store.cc:276] 0709 13:06:28.546484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.700676 (+154192us) service_pool.cc:224] Handling call
0709 13:06:28.700991 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:28.762967 (+ 61976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:28.763012 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:28.763073 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:28.763116 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.613775 (+850659us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b5400228b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:29.613837 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.613993 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123466112}
I20260709 13:06:29.618310 22589 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:29.619433 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:29.622663 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.621734 21930 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:29.625379 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.625505 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.626021 21930 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:29.621810 25279 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.634461 24751 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:29.635480 25206 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:29.635705 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.636001 25206 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.636363 25206 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.636534 22585 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:06:29.630635 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1773) took 2314 ms. Trace:
I20260709 13:06:29.631163 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.632666 25216 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.628928 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.642642 21935 rpcz_store.cc:276] 0709 13:06:27.316480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.351846 (+ 35366us) service_pool.cc:224] Handling call
0709 13:06:27.352200 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:28.542681 (+1190481us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:28.542726 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:28.543360 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:06:28.952510 (+409150us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:28.952551 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.952625 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:29.594281 (+641656us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:29.626644 (+ 32363us) raft_consensus.cc:1685] finished
0709 13:06:29.626664 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.630554 (+  3890us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1147394,"raft.queue_time_us":1149227,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":120,"spinlock_wait_cycles":7470464,"thread_start_us":407545,"threads_started":1,"wal-append.queue_time_us":605600}
I20260709 13:06:29.647419 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.648424 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.622280 25068 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:29.649729 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.650314 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:29.651090 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.655592 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:29.645627 25275 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:06:29.658841 25169 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:29.662025 21931 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:29.663173 21931 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:29.666260 25194 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:29.666747 25194 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:29.667565 25194 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.669440 21929 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:06:29.669811 21929 pending_rounds.cc:85] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 7
I20260709 13:06:29.670118 21929 pending_rounds.cc:107] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:06:29.670357 21929 raft_consensus.cc:2889] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:29.675843 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:29.679324 21935 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:29.680411 21935 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:29.685178 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:29.685519 21931 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:29.687463 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:29.683161 25068 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:29.688535 24751 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:29.689041 24751 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:29.689500 24751 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.694828 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41260 (request call id 87) took 1711 ms. Trace:
I20260709 13:06:29.695293 21860 rpcz_store.cc:276] 0709 13:06:27.983399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.515804 (+532405us) service_pool.cc:224] Handling call
0709 13:06:29.693379 (+1177575us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d5735 00007f3b707cd9fa 00007f3b757954c9 00007f3b6ed11c54 00007f3b6ed11be2 00007f3b6ed11b11 00007f3b6ed11a8c 00007f3b6ed10d42 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:29.694725 (+  1346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307843328}
I20260709 13:06:29.695827 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:29.701661 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.703161 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.705390 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.706826 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.704326 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.714671 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:29.715262 22001 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:29.715931 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.716495 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.717118 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.717742 22585 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:29.718389 22584 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:29.722003 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.723886 22588 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [6.6-8.7]
I20260709 13:06:29.729406 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:29.730235 21931 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.737828 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:29.738245 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:29.740226 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.683086 25206 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.742115 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:29.698611 25155 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:29.744403 22592 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:29.744834 22002 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:06:29.745190 22002 pending_rounds.cc:85] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
I20260709 13:06:29.745429 22002 pending_rounds.cc:107] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:06:29.745641 22002 raft_consensus.cc:2889] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:29.745857 22004 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:29.737015 25223 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:29.747985 25294 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:29.748741 25223 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:29.749325 25182 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:29.749725 25182 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:29.749826 25223 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.750643 25182 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.743831 25068 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:29.752928 21931 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:29.764889 22001 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
W20260709 13:06:29.766507 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48809.3us
I20260709 13:06:29.770392 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:29.699863 25248 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:06:29.893349 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:29.931003 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.934299 24909 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:29.935935 25206 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.937853 22008 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:29.781406 25076 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 13:06:29.757414 25155 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:29.938571 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:29.950883 21855 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:29.950986 22001 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:29.952760 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.953909 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.955318 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.956776 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.957546 21852 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:29.958555 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.959760 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.960788 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.961740 21860 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:29.962255 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:29.962896 21852 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 4.
I20260709 13:06:29.963670 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:29.965251 22008 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:29.966425 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2153) took 1049 ms. Trace:
I20260709 13:06:29.967286 22003 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:06:29.968343 22003 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 4
I20260709 13:06:29.968627 22003 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:06:29.968942 22003 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:29.942413 25279 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:29.966996 22006 rpcz_store.cc:276] 0709 13:06:28.916854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.923602 (+  6748us) service_pool.cc:224] Handling call
0709 13:06:29.966187 (+1042585us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b540038a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:29.966360 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418132352}
I20260709 13:06:29.973793 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:29.975996 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1769) took 2998 ms. Trace:
I20260709 13:06:29.976408 21927 rpcz_store.cc:276] 0709 13:06:26.977838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.044316 (+ 66478us) service_pool.cc:224] Handling call
0709 13:06:27.044660 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:29.094892 (+2050232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.094919 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:29.132849 (+ 37930us) log.cc:844] Serialized 52 byte log entry
0709 13:06:29.246754 (+113905us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:29.247142 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 13:06:29.255061 (+  7919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.966263 (+711202us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b5400229054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:29.966328 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:29.966388 (+    60us) raft_consensus.cc:1685] finished
0709 13:06:29.975782 (+  9394us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.975935 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1237638,"prepare.queue_time_us":1042,"prepare.run_cpu_time_us":4131,"prepare.run_wall_time_us":22277,"raft.queue_time_us":1358630,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":10925,"spinlock_wait_cycles":84312832,"thread_start_us":602511,"threads_started":3,"wal-append.queue_time_us":247654}
I20260709 13:06:29.986471 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 462) took 1052 ms. Trace:
I20260709 13:06:29.953624 25155 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:29.986910 21858 rpcz_store.cc:276] 0709 13:06:28.934462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.934603 (+   141us) service_pool.cc:224] Handling call
0709 13:06:28.934906 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:28.935104 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:28.973902 (+ 38798us) log.cc:844] Serialized 14 byte log entry
0709 13:06:29.957591 (+983689us) log.cc:844] Serialized 14 byte log entry
0709 13:06:29.986144 (+ 28553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:29.986198 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:29.986242 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.986306 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:29.986415 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":501835,"prepare.queue_time_us":91079,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":437038,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":548019,"threads_started":2,"wal-append.queue_time_us":560432}
I20260709 13:06:29.961320 24909 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:29.995276 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:29.995196 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:29.996085 21855 raft_consensus.cc:2376] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8. Re-sending same reply.
I20260709 13:06:29.997496 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:29.998308 22006 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:29.941591 25184 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:06:30.017467 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:30.002666 25182 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:30.006699 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:06:30.007416 21823 connection.cc:256] RPC call timeout handler was delayed by -1.0322849689998748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
I20260709 13:06:30.009881 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.028817 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:30.012714 21935 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:30.033030 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:30.033174 21927 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:30.035156 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:30.036266 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:30.038053 21931 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:30.055729 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.023861 25289 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:30.060016 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.062448 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.033843 25295 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:30.067015 25076 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:06:30.085458 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 88807.7us
I20260709 13:06:30.088429 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2162) took 1162 ms. Trace:
I20260709 13:06:30.088837 22000 rpcz_store.cc:276] 0709 13:06:28.926341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.944518 (+ 18177us) service_pool.cc:224] Handling call
0709 13:06:30.088242 (+1143724us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400168854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:30.088373 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999471488}
W20260709 13:06:30.092763 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63423.3us
I20260709 13:06:30.094179 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2168) took 1165 ms. Trace:
I20260709 13:06:30.065508 25289 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:30.098904 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:30.094594 22005 rpcz_store.cc:276] 0709 13:06:28.928355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.061213 (+132858us) service_pool.cc:224] Handling call
0709 13:06:29.092865 (+ 31652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:29.093638 (+   773us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:29.131703 (+ 38065us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.093747 (+962044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:30.093803 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:30.093919 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.094002 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.094119 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":859158,"prepare.queue_time_us":37902,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":860643,"spinlock_wait_cycles":1598848,"thread_start_us":102510,"threads_started":1,"wal-append.queue_time_us":305078}
I20260709 13:06:30.099750 21858 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:30.105752 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:30.107136 21858 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:30.105664 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.109897 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:30.110486 21855 raft_consensus.cc:2376] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7. Re-sending same reply.
I20260709 13:06:30.114213 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:30.114845 21858 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.115904 25239 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:30.097968 25023 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:30.125407 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.126319 22007 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:30.128086 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:30.129433 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.103371 25289 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:30.096912 25248 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:30.067507 25219 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:06:30.120042 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 565547us
I20260709 13:06:30.135035 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.135872 22007 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:30.131559 21858 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:30.131489 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.137283 22009 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:06:30.138106 22009 pending_rounds.cc:85] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 3
I20260709 13:06:30.138038 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:30.137611 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.133608 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.140468 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.141068 22008 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.141309 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.138568 22009 pending_rounds.cc:107] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:06:30.142057 22009 raft_consensus.cc:2889] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.142705 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.137202 21858 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:30.143497 22002 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.130530 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:30.132893 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.145413 22001 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.139339 22004 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.088102 24909 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:30.151762 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46217.4us
I20260709 13:06:30.133360 25101 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:30.152984 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.155524 22008 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:30.156739 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.148819 25068 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:30.158572 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:30.109331 25206 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:06:30.144340 21858 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.172001 21858 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.129621 25098 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:06:30.160221 25226 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:30.109133 24866 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:06:30.133275 25107 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:30.133054 25159 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.140784 25290 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.140949 25210 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.169641 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:30.158341 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 454363us
W20260709 13:06:30.210816 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.211236 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.168522 25279 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:30.166309 25301 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:30.140719 25197 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 13:06:30.289240 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 226907us
I20260709 13:06:30.174088 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:30.211690 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.290491 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:30.157811 25119 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.214706 25273 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:30.222896 25210 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.223940 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2161) took 1298 ms. Trace:
I20260709 13:06:30.224315 25159 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.228827 25194 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.233950 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:30.287492 25209 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:30.254763 25297 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
W20260709 13:06:30.270583 25291 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.305s	user 0.000s	sys 0.002s
I20260709 13:06:30.274242 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:30.277303 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 120665us
I20260709 13:06:30.310086 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.311324 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.164036 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:30.279176 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.314113 22005 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:06:30.286096 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 148122us
I20260709 13:06:30.167793 25248 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:06:30.165720 25165 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:06:30.210834 25137 raft_consensus.cc:1064] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:06:30.166018 25101 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:30.295039 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 120996us
I20260709 13:06:30.300915 24866 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:30.181592 25203 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:30.187278 25226 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:30.209882 25107 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.300895 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.296816 25304 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:30.240579 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.275952 25279 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:30.306384 22001 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:30.275211 25290 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:30.306676 25023 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:30.316599 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 152678us
I20260709 13:06:30.314806 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:30.319664 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:30.322230 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:30.323194 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:30.325367 22589 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEARNER]: Advancing to term 7
I20260709 13:06:30.330272 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:30.296190 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:30.360486 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64357.1us
I20260709 13:06:30.361440 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.332221 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:30.362814 22591 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:30.334169 22008 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.299149 22003 rpcz_store.cc:276] 0709 13:06:28.926018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.945774 (+ 19756us) service_pool.cc:224] Handling call
0709 13:06:29.966919 (+1021145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:29.973729 (+  6810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.223561 (+249832us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540038a4d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:30.223858 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347681664}
I20260709 13:06:30.336602 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:30.306210 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 203042us
I20260709 13:06:30.297156 25173 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:06:30.355151 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.377626 22001 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:30.347354 25203 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.318619 25098 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:30.347616 25226 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:30.357669 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:30.328378 25299 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:30.332446 25106 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:30.383747 25106 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.384148 25106 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.384219 25101 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:30.380383 25203 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.351797 25107 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.389597 25023 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.392321 25106 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:30.172461 25208 consensus_queue.cc:1175] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 8 status { last_received { term: 6 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:30.333644 24866 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:30.348227 25304 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:30.413909 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94273.8us
I20260709 13:06:30.395355 25106 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:30.417131 24866 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:30.365262 25097 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:30.352560 25279 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.417685 25139 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:30.402022 25159 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:30.397446 25300 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:06:30.456079 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:30.420766 25103 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:30.425148 25304 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.425398 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:30.426002 22000 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:30.466559 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.467549 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:30.468027 22000 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.434437 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:30.464399 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.450863 22589 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:06:30.381084 25287 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:30.452574 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.471143 22587 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
W20260709 13:06:30.420217 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62623.3us
W20260709 13:06:30.410977 24751 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.720s	user 0.006s	sys 0.004s
I20260709 13:06:30.476648 22592 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:30.430279 25106 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:30.497504 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.472308 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.460009 22591 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:30.402357 25290 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:30.397727 25202 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:06:30.517683 25202 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:30.518095 25202 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:30.519032 25202 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.520512 25202 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.477054 25287 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:30.486861 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:30.487156 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 172407us
W20260709 13:06:30.488369 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 158157us
I20260709 13:06:30.492761 25107 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:30.520969 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.493084 25120 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:30.472442 22589 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 4
I20260709 13:06:30.499735 22000 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:30.521886 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.497033 25309 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:30.500914 25117 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:30.496627 25103 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:30.501806 25234 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:30.503422 25173 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:30.493889 25106 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:30.521368 25287 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:30.504746 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:30.496539 24751 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:30.507547 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.507834 24866 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.509913 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:30.581840 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77174.4us
I20260709 13:06:30.510689 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.529862 22003 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:30.582980 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.583828 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.549471 25139 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:30.524861 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:30.525724 21859 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:30.527408 21858 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:06:30.589869 21858 pending_rounds.cc:85] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 6
I20260709 13:06:30.590139 21858 pending_rounds.cc:107] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:06:30.590377 21858 raft_consensus.cc:2889] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.538728 22589 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:06:30.591347 22589 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.522679 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:30.541766 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.549731 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:30.553203 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:30.452164 25155 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.560096 22588 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:30.505177 25190 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:30.539306 25290 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.510677 25210 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:30.525238 25208 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:30.580658 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1840) took 1429 ms. Trace:
I20260709 13:06:30.637297 25208 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.637559 21857 rpcz_store.cc:276] 0709 13:06:29.151570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.510009 (+358439us) service_pool.cc:224] Handling call
0709 13:06:29.510422 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.937276 (+426854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.937304 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.937727 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.100338 (+162611us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:30.100394 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.100486 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.580414 (+479928us) raft_consensus.cc:1685] finished
0709 13:06:30.580433 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.580579 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":445334,"raft.queue_time_us":427491,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6265,"thread_start_us":162166,"threads_started":1,"wal-append.queue_time_us":193141}
W20260709 13:06:30.581998 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72153.7us
I20260709 13:06:30.639434 25208 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:30.510421 25169 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:30.521509 22008 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:30.585958 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 75332us
I20260709 13:06:30.594923 22591 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:30.640022 25208 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:30.643569 25208 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:30.469601 25226 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.555804 25309 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:30.653764 25309 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:30.625680 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 100882us
I20260709 13:06:30.657310 22590 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:30.657186 25203 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.658370 25155 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:30.623126 25139 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.646800 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:30.694482 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47778.9us
I20260709 13:06:30.642601 25103 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:30.623871 25190 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:30.649410 25289 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.000s
I20260709 13:06:30.556229 25113 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:06:30.649226 25290 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.651048 24751 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:30.651160 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.651684 25107 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:30.652071 25159 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:30.622161 25097 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:30.627955 22585 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:30.663285 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:30.604439 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:30.806272 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:30.807104 22003 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:30.702445 25190 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.640115 25194 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:30.579464 25173 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:30.718262 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:30.629128 25234 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:30.649178 24866 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:30.728662 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 77586.4us
I20260709 13:06:30.875638 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.877290 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:30.797537 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.878263 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.878641 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.699168 25117 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:30.722081 25119 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:30.756182 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.880218 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:30.736277 25159 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.746291 25256 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:30.701442 22008 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:30.800428 25097 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:30.921578 22000 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:30.860414 24751 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.698561 25304 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.949494 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:30.718145 25289 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:30.962042 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 411516us
I20260709 13:06:30.889392 25117 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.940300 22000 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:30.893584 25194 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:30.964603 25194 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.965102 25194 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.965266 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.966427 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:30.966686 22008 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:30.894035 24866 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:30.918366 25159 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:30.702804 25106 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:30.973488 22003 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:30.933373 25097 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:30.976336 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:31.034053 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:31.037936 21858 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:30.945964 25119 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:31.013427 21859 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:30.969451 25117 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:30.971361 25306 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.378s	user 0.001s	sys 0.001s
W20260709 13:06:31.040891 25313 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:06:31.041198 24759 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:30.975847 25256 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:31.036257 25291 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:06:30.973965 25203 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:31.076386 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.078382 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.052470 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:31.086247 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:30.962710 24751 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:31.106967 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.011s	sys 0.030s
I20260709 13:06:30.969580 25273 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:06:30.931949 25208 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:31.145807 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.013s	sys 0.030s
W20260709 13:06:31.317529 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 734602us
I20260709 13:06:31.053195 24759 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.318362 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.317597 21853 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:31.319504 21853 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:31.189739 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.323093 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.324378 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:31.320204 21853 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.328258 21853 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.085963 25256 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:31.014671 24866 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.087936 25106 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:31.334151 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.043s	sys 0.049s
W20260709 13:06:31.339169 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.043s	sys 0.049s
I20260709 13:06:31.044375 25119 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:31.324836 25313 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 13:06:31.340687 25317 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.294s	user 0.000s	sys 0.000s
I20260709 13:06:31.342341 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 669) took 1324 ms. Trace:
W20260709 13:06:31.342587 25120 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.803s	user 0.006s	sys 0.002s
I20260709 13:06:31.345103 25325 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:31.345535 25120 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:31.345888 25325 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:31.348269 25325 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:31.342993 22006 rpcz_store.cc:276] 0709 13:06:30.017775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.018594 (+   819us) service_pool.cc:224] Handling call
0709 13:06:30.018997 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.130693 (+111696us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:30.285188 (+154495us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.442621 (+157433us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.453866 (+ 11245us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.454912 (+  1046us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.455119 (+   207us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:30.455166 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.455272 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.512636 (+ 57364us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18002fc4a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:31.342028 (+829392us) raft_consensus.cc:1685] finished
0709 13:06:31.342062 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.342257 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121901,"prepare.queue_time_us":155401,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":4270,"raft.queue_time_us":124586,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":29927,"spinlock_wait_cycles":49509504,"thread_start_us":303668,"threads_started":2,"wal-append.queue_time_us":338348}
I20260709 13:06:31.351106 22006 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:31.352200 25159 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:31.353685 22008 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:31.356050 25159 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:31.354559 25119 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:31.356598 22006 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:06:31.359037 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 719284us
I20260709 13:06:31.359944 25119 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.369570 25159 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:31.370177 25159 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.370618 25159 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:31.371443 21932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.053s	sys 0.089s
W20260709 13:06:31.373203 21932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.053s	sys 0.090s
I20260709 13:06:31.318499 25226 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:31.322786 24759 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.392894 21853 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:06:31.393451 21853 pending_rounds.cc:85] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 7
I20260709 13:06:31.393728 21853 pending_rounds.cc:107] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:06:31.394032 21853 raft_consensus.cc:2889] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:31.376371 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:31.332808 25203 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:31.401598 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.024s	sys 0.043s
I20260709 13:06:31.402873 22006 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:06:31.413627 22006 pending_rounds.cc:85] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 3
I20260709 13:06:31.413895 22006 pending_rounds.cc:107] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:06:31.414122 22006 raft_consensus.cc:2889] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:31.400182 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:31.415720 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.024s	sys 0.043s
W20260709 13:06:31.416357 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 536177us
I20260709 13:06:31.337415 25290 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:31.337975 25256 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.419711 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:31.376056 25309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.024s	sys 0.029s
I20260709 13:06:31.371865 25324 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.437456 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:31.439271 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.413043 25325 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:31.440923 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.445935 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 673) took 1151 ms. Trace:
I20260709 13:06:31.449383 22007 rpcz_store.cc:276] 0709 13:06:30.294787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.294967 (+   180us) service_pool.cc:224] Handling call
0709 13:06:30.295268 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.295645 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:30.296190 (+   545us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.420530 (+124340us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b1800319ae0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:06:30.562468 (+141938us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.563375 (+   907us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.580394 (+ 17019us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:30.580433 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.580505 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.445636 (+865131us) raft_consensus.cc:1685] finished
0709 13:06:31.445749 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.445872 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3359,"prepare.queue_time_us":532,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1768,"raft.queue_time_us":30728,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":39314,"spinlock_wait_cycles":33074048,"thread_start_us":136199,"threads_started":1,"wal-append.queue_time_us":645886}
I20260709 13:06:31.446105 24866 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:31.431540 25309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.024s	sys 0.029s
I20260709 13:06:31.419029 25139 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:31.447039 25304 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:31.472816 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.438675 25120 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:31.466147 25206 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:31.436355 25173 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:31.501915 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.461356 25273 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:31.516741 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2109) took 1198 ms. Trace:
I20260709 13:06:31.517347 22002 rpcz_store.cc:276] 0709 13:06:30.318626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.318815 (+   189us) service_pool.cc:224] Handling call
0709 13:06:30.350010 (+ 31195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.350453 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:30.356786 (+  6333us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.523130 (+166344us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b1800368920. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:06:30.977455 (+454325us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.978003 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.978163 (+   160us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:30.978203 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.978268 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.512205 (+533937us) raft_consensus.cc:1685] finished
0709 13:06:31.516488 (+  4283us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.516676 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23758,"prepare.queue_time_us":13490,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":37464,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":44727,"spinlock_wait_cycles":15367808,"thread_start_us":453135,"threads_started":1,"wal-append.queue_time_us":1071003}
I20260709 13:06:31.486891 25107 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:31.491329 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.489236 25139 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.530902 22589 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:06:31.524132 21931 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.015s	sys 0.016s
I20260709 13:06:31.463773 25232 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 13:06:31.533739 21931 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.015s	sys 0.016s
I20260709 13:06:31.507299 25120 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:31.465071 25325 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:31.535373 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44116.5us
I20260709 13:06:31.515713 25309 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:31.515439 25206 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:31.537089 25068 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.026s	sys 0.017s
I20260709 13:06:31.558523 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 677) took 1055 ms. Trace:
I20260709 13:06:31.561915 22002 rpcz_store.cc:276] 0709 13:06:30.503232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.542533 (+1039301us) service_pool.cc:224] Handling call
0709 13:06:31.557880 (+ 15347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.558107 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.558142 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.558196 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.558236 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.558327 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.558454 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:31.532837 25155 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.536221 25289 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:31.550971 25120 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.534163 25332 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.557494 25206 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.533879 25139 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:31.582577 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.003s	sys 0.007s
W20260709 13:06:31.583325 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.003s	sys 0.007s
W20260709 13:06:31.583952 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.04063e+06us
W20260709 13:06:31.569897 22004 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.006s	sys 0.005s
I20260709 13:06:31.554859 25331 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:31.586230 22004 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.006s	sys 0.005s
W20260709 13:06:31.587957 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 263633us
I20260709 13:06:31.588762 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:31.589334 21858 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.027s	sys 0.035s
W20260709 13:06:31.590745 21858 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.027s	sys 0.035s
I20260709 13:06:31.592442 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2180) took 1086 ms. Trace:
I20260709 13:06:31.592980 22002 rpcz_store.cc:276] 0709 13:06:30.506343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.580873 (+1074530us) service_pool.cc:224] Handling call
0709 13:06:31.591801 (+ 10928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.592012 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.592051 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.592105 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:31.592153 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.592222 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.592374 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:31.569527 25327 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:31.606243 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 512556us
I20260709 13:06:31.617089 25139 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:31.606981 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.014s	sys 0.001s
I20260709 13:06:31.608477 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.570159 25309 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:06:31.620925 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 826) took 1864 ms. Trace:
I20260709 13:06:31.633675 22590 rpcz_store.cc:276] 0709 13:06:29.756948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.757098 (+   150us) service_pool.cc:224] Handling call
0709 13:06:29.757408 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.657813 (+900405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.620668 (+962855us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b5400595454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:31.620853 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2069084032}
I20260709 13:06:31.636060 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 802) took 1314 ms. Trace:
I20260709 13:06:31.637753 25246 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:31.639549 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 671) took 1508 ms. Trace:
I20260709 13:06:31.636701 22588 rpcz_store.cc:276] 0709 13:06:30.321163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.323601 (+  2438us) service_pool.cc:224] Handling call
0709 13:06:30.324000 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.620874 (+296874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.635740 (+1014866us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540076fcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:31.635986 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542040448}
I20260709 13:06:31.622779 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 827) took 1389 ms. Trace:
I20260709 13:06:31.623829 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 804) took 1302 ms. Trace:
I20260709 13:06:31.649849 22585 rpcz_store.cc:276] 0709 13:06:30.321717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.323259 (+  1542us) service_pool.cc:224] Handling call
0709 13:06:30.323533 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.802101 (+478568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.623557 (+821456us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b5400684f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:31.623758 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693150080}
I20260709 13:06:31.569202 25155 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.645386 22587 rpcz_store.cc:276] 0709 13:06:30.233459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.233627 (+   168us) service_pool.cc:224] Handling call
0709 13:06:31.622530 (+1388903us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400668fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:31.622659 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347184512}
W20260709 13:06:31.663375 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:31.640000 22009 rpcz_store.cc:276] 0709 13:06:30.131383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.142841 (+ 11458us) service_pool.cc:224] Handling call
0709 13:06:30.143080 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.363738 (+220658us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:30.559662 (+195924us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.893671 (+334009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:30.893728 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:30.893767 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.639319 (+745552us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54003895d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:31.639380 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.639496 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":559,"prepare.queue_time_us":43117,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":27221,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":18647,"spinlock_wait_cycles":506936704,"thread_start_us":499403,"threads_started":2,"wal-append.queue_time_us":827683}
W20260709 13:06:31.640905 25117 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.000s
I20260709 13:06:31.610948 25173 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:31.675321 21900 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:31.643510 24751 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:31.677386 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 277297us
I20260709 13:06:31.677559 22591 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:06:31.678089 21823 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.671s (SENT) [suppressed 9 similar messages]
W20260709 13:06:31.666930 25322 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:06:31.626659 25309 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:31.596143 25332 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 13:06:31.670929 22009 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:31.673242 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.673960 25323 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:06:31.636194 25137 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.678853 25173 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:31.698683 25173 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.677254 25117 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.694598 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2183) took 1187 ms. Trace:
I20260709 13:06:31.717432 22009 rpcz_store.cc:276] 0709 13:06:30.507197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.668120 (+1160923us) service_pool.cc:224] Handling call
0709 13:06:31.668503 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.693206 (+ 24703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.694512 (+  1306us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:31.718120 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.719918 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 678) took 1131 ms. Trace:
I20260709 13:06:31.687706 25309 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:31.721478 22009 rpcz_store.cc:276] 0709 13:06:30.588222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.718907 (+1130685us) service_pool.cc:224] Handling call
0709 13:06:31.719287 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.719526 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.719562 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.719616 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.719659 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.719728 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.719867 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:31.723205 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 573) took 1182 ms. Trace:
I20260709 13:06:31.723907 21932 rpcz_store.cc:276] 0709 13:06:30.541016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.542221 (+  1205us) service_pool.cc:224] Handling call
0709 13:06:30.542557 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.542926 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.542956 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.554111 (+ 11155us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.377609 (+823498us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.377672 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.377777 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.722942 (+345165us) raft_consensus.cc:1685] finished
0709 13:06:31.722970 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.723135 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15875,"raft.queue_time_us":91,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":5150,"thread_start_us":823055,"threads_started":1,"wal-append.queue_time_us":968015}
W20260709 13:06:31.731885 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 179781us
I20260709 13:06:31.622135 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:31.733919 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 111856us
I20260709 13:06:31.734270 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.735831 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:31.621198 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.014s	sys 0.001s
I20260709 13:06:31.705569 25273 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:06:31.749435 21860 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:31.738479 25348 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:31.738934 25342 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.737974 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.726363 25331 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:06:31.747982 25330 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:06:31.761643 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:31.737974 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:31.628482 25210 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:31.762840 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:31.742676 21856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.214s	user 0.000s	sys 0.017s
I20260709 13:06:31.714649 25344 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:31.725546 25325 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:31.751927 21860 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:31.763514 21855 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:31.763950 21856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.000s	sys 0.018s
I20260709 13:06:31.769392 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:31.760203 25287 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.165s	user 0.008s	sys 0.004s
W20260709 13:06:31.760504 25194 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.795s	user 0.007s	sys 0.004s
I20260709 13:06:31.756215 25348 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:31.786893 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:31.759752 25331 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:31.765858 25139 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:31.765983 25347 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.770941 25342 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:31.805255 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:31.767459 25210 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.809602 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.815181 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:31.823017 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:31.825378 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2083) took 1301 ms. Trace:
W20260709 13:06:31.825695 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.787757 25333 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:31.630113 25310 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.174s
I20260709 13:06:31.825845 21856 rpcz_store.cc:276] 0709 13:06:30.523672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.527520 (+  3848us) service_pool.cc:224] Handling call
0709 13:06:30.527817 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.528038 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:30.528282 (+   244us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.825032 (+1296750us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.825092 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.825135 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.825210 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.825322 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49180,"prepare.queue_time_us":9080,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":1235922,"threads_started":1,"wal-append.queue_time_us":1212488}
I20260709 13:06:31.790200 25155 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:31.787945 25348 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:31.833633 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:31.835559 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 190702us
W20260709 13:06:31.873318 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 119254us
I20260709 13:06:31.875735 22009 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:31.780856 25287 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.803638 25139 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:31.781221 25194 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:31.868960 25310 consensus_queue.cc:1243] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:06:31.879146 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 108791us
I20260709 13:06:31.821782 25210 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:31.871264 25344 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:31.830960 25099 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:31.868566 25103 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:31.879953 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93195.2us
I20260709 13:06:31.806921 25350 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:06:31.832844 25232 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 13:06:31.895076 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 88483.4us
I20260709 13:06:31.888458 25226 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:31.919278 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.820593 25347 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:31.917898 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:31.895076 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.934953 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:31.882256 25290 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:31.935321 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.935371 21932 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:31.888710 25139 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:31.893991 25357 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:31.940874 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.941215 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:31.890444 25068 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:31.941253 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:31.944736 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:31.941506 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.879916 25208 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:06:31.937383 21930 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:31.985011 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.003196 21852 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:31.996896 21860 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:31.952605 22009 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
W20260709 13:06:32.004820 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.985579 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52984.9us
I20260709 13:06:31.941262 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.999593 21856 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:32.005806 21935 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:32.006413 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:32.008597 25232 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:32.008641 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.009274 25232 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:32.010357 25232 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.005122 25344 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.006235 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.010398 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.016175 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:31.996883 21927 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:32.001514 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 84384.8us
I20260709 13:06:31.998351 25357 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:32.021929 25357 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.022867 22009 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:32.031718 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:31.920359 25117 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.037876 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:32.038533 21860 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:32.041054 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.044621 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.023404 25357 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:32.002380 25339 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:06:31.939155 25155 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:32.072438 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.073205 21931 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.888428 24759 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:32.054483 25347 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.055230 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.047554 21935 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:32.059476 21857 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:32.063263 21860 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:31.996774 25068 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:32.079982 25068 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.946905 25372 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:32.092581 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.093685 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.094439 21927 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:31.937132 25343 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:32.099833 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:32.043576 25309 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.108474 21931 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:32.055843 25117 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:32.109512 25117 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:32.110163 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 93781.5us
W20260709 13:06:32.111501 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59336.6us
W20260709 13:06:32.057287 25345 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:06:32.112754 25117 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:32.079516 25190 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.203s	user 0.013s	sys 0.000s
W20260709 13:06:32.091415 25346 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:06:32.090780 25103 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:32.160161 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47024.7us
I20260709 13:06:32.166756 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:32.091744 25279 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.631s	user 0.010s	sys 0.003s
I20260709 13:06:31.946177 25290 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:31.895474 25331 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.048287 25351 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:06:32.175995 21927 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:32.180378 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2170) took 1830 ms. Trace:
I20260709 13:06:32.091267 25155 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.169636 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.171093 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:32.161232 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.186100 22001 rpcz_store.cc:276] 0709 13:06:30.350409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.973153 (+622744us) service_pool.cc:224] Handling call
0709 13:06:31.003103 (+ 29950us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.033696 (+ 30593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.033725 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.042800 (+  9075us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.564771 (+521971us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:31.564810 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.564914 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.180139 (+615225us) raft_consensus.cc:1685] finished
0709 13:06:32.180161 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.180313 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30192,"raft.queue_time_us":39454,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":7855,"thread_start_us":425182,"threads_started":1,"wal-append.queue_time_us":604497}
I20260709 13:06:32.190641 21854 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:32.161674 25341 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 13:06:32.196215 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.197952 22001 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:32.160948 25372 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:32.187039 25362 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
W20260709 13:06:32.203382 25353 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:06:32.165894 25368 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:32.158022 25190 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.116336 25343 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.212947 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:32.211292 25097 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.160s	user 0.013s	sys 0.000s
I20260709 13:06:32.183068 25364 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:06:32.238440 22006 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:32.224604 25194 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:32.264026 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.256785 25347 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:32.271924 25337 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:32.272584 25361 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:06:32.272537 25337 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:32.274246 25358 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:06:32.274842 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 675) took 1915 ms. Trace:
I20260709 13:06:32.275202 22003 rpcz_store.cc:276] 0709 13:06:30.359020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.042893 (+683873us) service_pool.cc:224] Handling call
0709 13:06:31.043352 (+   459us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:31.050208 (+  6856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.050250 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:31.200926 (+150676us) log.cc:844] Serialized 78 byte log entry
0709 13:06:31.566994 (+366068us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.628107 (+ 61113us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4800130158. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:06:31.771251 (+143144us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.919044 (+147793us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.919374 (+   330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.919503 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.274585 (+355082us) raft_consensus.cc:1685] finished
0709 13:06:32.274613 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.274783 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27322,"prepare.queue_time_us":379158,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":203684,"raft.run_cpu_time_us":2793,"raft.run_wall_time_us":618754,"spinlock_wait_cycles":9142912,"thread_start_us":486216,"threads_started":3,"wal-append.queue_time_us":379548}
I20260709 13:06:32.262401 25287 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.271938 25347 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.283397 25347 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.284472 25343 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.231061 25379 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:06:32.284864 25068 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.286441 25239 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:32.286841 25239 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:32.286974 25068 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.287711 25239 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.290498 25190 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:32.291149 25120 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.001s
W20260709 13:06:32.291546 25352 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
I20260709 13:06:32.291960 25347 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.293699 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 574) took 1752 ms. Trace:
I20260709 13:06:32.289230 25387 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:32.294574 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 675) took 1870 ms. Trace:
I20260709 13:06:32.295042 22584 rpcz_store.cc:276] 0709 13:06:30.424224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.429768 (+  5544us) service_pool.cc:224] Handling call
0709 13:06:30.430117 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.430556 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.430599 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.451680 (+ 21081us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.648605 (+1196925us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b18004a4ac0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:06:31.742336 (+ 93731us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.813580 (+ 71244us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.814445 (+   865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.814518 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.292345 (+477827us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b18004a4ac0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:32.294377 (+  2032us) raft_consensus.cc:1685] finished
0709 13:06:32.294396 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.294512 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":313229,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":7717,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":29487,"spinlock_wait_cycles":27457152,"thread_start_us":149913,"threads_started":2,"wal-append.queue_time_us":1362527}
I20260709 13:06:32.298782 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 571) took 1823 ms. Trace:
I20260709 13:06:32.299178 21930 rpcz_store.cc:276] 0709 13:06:30.475383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.475515 (+   132us) service_pool.cc:224] Handling call
0709 13:06:30.480230 (+  4715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.004557 (+1524327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.298542 (+293985us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400429ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:32.298721 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3313844992}
I20260709 13:06:32.300117 21854 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:32.294061 21928 rpcz_store.cc:276] 0709 13:06:30.541355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.541483 (+   128us) service_pool.cc:224] Handling call
0709 13:06:30.554220 (+ 12737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.955816 (+401596us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:31.530350 (+574534us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.826196 (+295846us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.826959 (+   763us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.827091 (+   132us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.827135 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.827197 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.293476 (+466279us) raft_consensus.cc:1685] finished
0709 13:06:32.293496 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.293640 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62253,"prepare.queue_time_us":535579,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":85489,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":44623,"spinlock_wait_cycles":15441536,"thread_start_us":1152225,"threads_started":4,"wal-append.queue_time_us":242084}
I20260709 13:06:32.302197 25155 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:32.291183 25159 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.920s	user 0.008s	sys 0.003s
I20260709 13:06:32.305248 25347 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.305598 25159 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.291576 25120 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.274964 25337 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:32.313303 25106 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.935s	user 0.005s	sys 0.006s
W20260709 13:06:32.314278 25355 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:06:32.315192 24759 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.315588 22007 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:32.288790 25239 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:32.318886 25206 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.001s
I20260709 13:06:32.319278 25206 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:32.319799 25173 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.000s
I20260709 13:06:32.320192 25173 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.296795 25289 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:32.322104 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 472) took 1238 ms. Trace:
I20260709 13:06:32.322415 25289 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:32.322602 25139 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.323282 25289 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.322501 21858 rpcz_store.cc:276] 0709 13:06:31.083809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.084118 (+   309us) service_pool.cc:224] Handling call
0709 13:06:31.084469 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.596698 (+512229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.596731 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.597230 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.779747 (+182517us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:31.885246 (+105499us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.886403 (+  1157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.893203 (+  6800us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.321900 (+428697us) raft_consensus.cc:1685] finished
0709 13:06:32.321922 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.322049 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57673,"prepare.queue_time_us":223369,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":4115,"raft.queue_time_us":354764,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":32420,"thread_start_us":695385,"threads_started":3,"wal-append.queue_time_us":224396}
I20260709 13:06:32.324365 25120 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.324419 25289 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:32.325307 22000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2112) took 2005 ms (client timeout 2582 ms). Trace:
W20260709 13:06:32.325671 22000 rpcz_store.cc:269] 0709 13:06:30.319417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.549433 (+230016us) service_pool.cc:224] Handling call
0709 13:06:32.304971 (+1755538us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540040d654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:32.325249 (+ 20278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751882624}
W20260709 13:06:32.326884 25256 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.887s	user 0.010s	sys 0.000s
I20260709 13:06:32.327407 22002 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:32.327257 25256 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.328467 21929 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:32.330216 25119 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.964s	user 0.010s	sys 0.002s
I20260709 13:06:32.331355 22009 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:32.331506 25139 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:32.332106 25139 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.326790 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.349139 25256 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.349351 22000 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:32.350107 25372 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:32.350539 25372 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:32.350930 25372 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:32.336452 25373 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:32.340474 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:32.352543 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:32.352809 22584 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.353422 25119 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:32.353880 25119 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:32.353587 25139 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.354099 25371 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:32.323838 25159 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:32.341297 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:32.342077 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:32.341096 25324 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.876s	user 0.000s	sys 0.012s
I20260709 13:06:32.225140 25368 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.265799 25103 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.346668 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:32.354367 25119 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.341589 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:32.235549 25097 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.441223 25097 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.356102 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:32.356333 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:32.443214 22590 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:32.356989 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:32.447568 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.448424 21854 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:32.446427 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.450099 21856 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:32.450989 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 283278us
I20260709 13:06:32.443239 22588 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:06:32.453261 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.004s	sys 0.008s
W20260709 13:06:32.453738 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.008s
W20260709 13:06:32.453845 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 96887.6us
I20260709 13:06:32.453840 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.456140 21857 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:32.461508 21975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.005s	sys 0.003s
W20260709 13:06:32.461853 21975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.005s	sys 0.003s
W20260709 13:06:32.462617 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.462970 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:32.468439 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:32.470824 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 357993us
I20260709 13:06:32.479956 22587 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:06:32.483501 22004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.003s	sys 0.007s
W20260709 13:06:32.483897 22004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.003s	sys 0.007s
I20260709 13:06:32.363006 22584 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:32.395332 22589 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:32.187649 25279 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.372565 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:32.410045 22003 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:32.403779 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.420603 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.464308 25299 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.494155 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:32.495622 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.403509 25324 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:32.496977 21855 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.491480 21858 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.491725 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:32.478051 25165 raft_consensus.cc:1027] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: attempt to promote peer 34f3b83da84448bf8aac62b37a1486eb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:06:32.500434 25260 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.502805 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.473934 25337 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:32.509837 22589 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
W20260709 13:06:32.511016 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 59376.4us
I20260709 13:06:32.477492 25206 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.519995 21933 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:32.520604 21858 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:32.490646 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.526126 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:32.526860 22000 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.528921 22003 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:32.531806 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:32.474366 25310 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:06:32.492331 25342 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:32.500483 25101 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.539680 21858 pending_rounds.cc:85] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 5
I20260709 13:06:32.550215 21858 pending_rounds.cc:107] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:06:32.551404 21858 raft_consensus.cc:2889] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:32.553856 22000 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:32.561928 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:32.539069 25210 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.000s
I20260709 13:06:32.565999 22000 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.572754 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.538237 25223 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.582300 21927 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:06:32.564181 25364 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:32.509301 25299 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.571290 25210 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.501047 25260 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.623478 21928 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:06:32.566496 25394 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:32.624414 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.565222 25120 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:32.627702 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.567643 25097 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.630420 21935 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:32.568975 24847 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:32.632422 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.636058 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.656092 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.657894 21928 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.657979 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.659346 22590 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
W20260709 13:06:32.661052 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 166964us
I20260709 13:06:32.663084 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:32.634721 21930 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:32.664420 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.664332 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:32.665676 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:32.665871 22585 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:32.665746 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 123881us
I20260709 13:06:32.587074 25351 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:32.588099 25290 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.671211 22590 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:32.564337 25101 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.674566 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:32.675329 21854 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:32.627777 25139 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.684887 21934 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:32.634575 25375 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:32.632488 25173 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.563596 25342 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.695412 22000 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:32.739923 21934 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:32.745640 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:32.680150 25363 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:06:32.722136 25348 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.825s	user 0.006s	sys 0.006s
I20260709 13:06:32.677994 24847 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:32.674901 25097 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.657402 25120 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.633500 25223 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.759910 25182 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.761368 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:32.781682 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:32.768447 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.790233 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:32.791079 22003 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:32.803062 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 395542us
I20260709 13:06:32.737197 25375 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:32.778918 25392 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:06:32.790294 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:32.770747 25396 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:06:32.810078 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:32.820715 25101 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:32.777545 25388 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 13:06:32.810544 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:32.770037 25395 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:06:32.767417 24847 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.773797 25210 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:32.833119 22000 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:32.774964 25097 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.778350 25223 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.776039 25120 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.839452 22000 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:32.840721 25155 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:32.841854 22000 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:32.636785 25364 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.776410 25260 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.795938 25363 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:32.843860 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 833) took 1316 ms. Trace:
I20260709 13:06:32.844249 25364 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.844364 25363 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.779594 25182 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.657289 25394 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.853600 22000 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.854429 25394 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.856143 22000 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:06:32.857061 21935 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:32.809959 25375 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.813580 25279 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.821983 21928 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:32.882653 22000 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:32.883610 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.682876 25351 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:32.784032 25348 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.829873 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:32.830439 25327 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:32.832095 25119 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.842007 25219 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:32.770745 25299 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.844280 22591 rpcz_store.cc:276] 0709 13:06:31.527624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.527949 (+   325us) service_pool.cc:224] Handling call
0709 13:06:31.551431 (+ 23482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.677922 (+126491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.837730 (+1159808us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54005000d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:32.843794 (+  6064us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289285376}
I20260709 13:06:32.824717 22003 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:32.852188 25182 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:32.857352 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.902658 21928 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:32.903498 22000 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:32.905303 22000 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:32.881608 25368 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.908562 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.909335 21928 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 5: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:32.910153 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.893901 25327 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.910768 21934 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:32.910765 22003 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:32.895880 25219 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:32.916688 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:32.923306 25372 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.007s
I20260709 13:06:32.936142 25119 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:32.937860 25402 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:06:32.941258 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 466456us
I20260709 13:06:32.941356 25208 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:32.943281 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2182) took 2436 ms. Trace:
I20260709 13:06:32.943683 22002 rpcz_store.cc:276] 0709 13:06:30.506964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.594215 (+1087251us) service_pool.cc:224] Handling call
0709 13:06:32.327229 (+733014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.327781 (+   552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.943064 (+615283us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540040d650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:32.943218 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684862336}
W20260709 13:06:32.945127 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 476765us
I20260709 13:06:32.945683 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:32.946600 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.947171 25324 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:32.948122 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:32.948969 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2179) took 2452 ms. Trace:
I20260709 13:06:32.949172 25324 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:32.949846 25324 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:32.949868 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:32.950865 25324 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:32.953146 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:32.953362 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:32.954528 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:32.954620 25206 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:32.955384 25206 consensus_queue.cc:1243] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:32.955817 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:32.956344 25342 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:32.956990 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:32.957245 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.955039 25406 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:06:32.922639 25329 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:32.946552 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:32.958298 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:32.958500 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.953660 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2189) took 1142 ms. Trace:
I20260709 13:06:33.017870 22009 rpcz_store.cc:276] 0709 13:06:31.811612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.031995 (+220383us) service_pool.cc:224] Handling call
0709 13:06:32.032555 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.331762 (+299207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.953455 (+621693us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54000ca8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:32.953607 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686738688}
I20260709 13:06:32.958909 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:32.955978 25348 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:32.981909 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:32.978219 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.027285 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.028471 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:32.949536 22007 rpcz_store.cc:276] 0709 13:06:30.497091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.454673 (+957582us) service_pool.cc:224] Handling call
0709 13:06:31.455040 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.316196 (+861156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.948666 (+632470us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b5400426154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:32.948911 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978405632}
W20260709 13:06:32.987493 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.949210 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:32.954613 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:33.029922 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:33.031030 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 81885.2us
I20260709 13:06:33.032044 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:33.083096 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51164.9us
I20260709 13:06:33.083336 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:33.083412 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 53293.4us
W20260709 13:06:33.084491 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 102707us
I20260709 13:06:33.099754 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:33.101439 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 143083us
I20260709 13:06:33.106047 21931 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:33.103860 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:32.958079 25372 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.132790 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:33.156368 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.161594 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:32.988826 25387 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:32.987357 25394 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:33.087474 25223 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:33.015724 25404 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.118379 25333 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:33.117995 24788 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:33.109149 25246 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:33.129417 25239 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:06:33.119484 25206 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:33.119215 25226 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:06:33.164177 25363 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:33.099056 25120 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:06:33.183085 25333 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:33.105381 25329 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:33.176437 25387 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.185153 25246 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:33.177213 25394 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.109515 25219 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.190637 25351 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.199965 25372 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:33.201496 25409 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:06:33.205209 25394 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.206089 25387 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:33.206660 25411 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:06:33.209049 25375 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.211405 25375 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.213578 25404 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:33.118176 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 171023us
I20260709 13:06:33.256116 21855 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:06:33.118649 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 164079us
W20260709 13:06:33.136783 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.260622 21929 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:33.168197 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:33.267611 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:33.271157 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1857) took 1221 ms. Trace:
I20260709 13:06:33.271775 21859 rpcz_store.cc:276] 0709 13:06:32.049724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.049925 (+   201us) service_pool.cc:224] Handling call
0709 13:06:32.050190 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.921442 (+871252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.921471 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:32.921511 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:32.921546 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.270471 (+348925us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:33.270563 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.270903 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003157248}
I20260709 13:06:33.281402 25246 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.188090 25206 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.292281 25391 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:33.199070 25333 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:29.615514 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:33.299468 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 625937us
I20260709 13:06:33.299702 25107 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:33.300979 25107 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:33.301222 25182 consensus_queue.cc:1175] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8f3b35a35257416db3eaaf364cf6b52b" responder_term: 4 status { last_received { term: 4 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 4 index: 7 } }
W20260709 13:06:33.299531 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 636303us
I20260709 13:06:33.312245 25368 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:33.313823 25101 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:33.314600 25101 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:33.317430 25202 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.318488 25202 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.322542 25202 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.327069 25324 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:33.327256 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 68420.9us
I20260709 13:06:33.329267 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:33.329680 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 225807us
I20260709 13:06:33.330667 24788 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.309028 25327 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.333688 25099 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:33.332903 24788 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.334614 25099 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.316356 25101 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.342321 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1856) took 1303 ms. Trace:
I20260709 13:06:33.346638 21852 rpcz_store.cc:276] 0709 13:06:32.039221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.039629 (+   408us) service_pool.cc:224] Handling call
0709 13:06:32.040018 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.040423 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:32.361420 (+320997us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.261914 (+900494us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.329620 (+ 67706us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.330686 (+  1066us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.330936 (+   250us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:33.330984 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.331121 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.341873 (+ 10752us) raft_consensus.cc:1685] finished
0709 13:06:33.341895 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.342139 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":598623,"prepare.queue_time_us":340407,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":83,"raft.queue_time_us":454527,"raft.run_cpu_time_us":489,"raft.run_wall_time_us":10269,"thread_start_us":663564,"threads_started":3,"wal-append.queue_time_us":185317}
I20260709 13:06:33.350242 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2110) took 3031 ms. Trace:
I20260709 13:06:33.355592 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2177) took 2889 ms. Trace:
I20260709 13:06:33.357815 25372 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:33.358718 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.350754 22005 rpcz_store.cc:276] 0709 13:06:30.318912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.319198 (+   286us) service_pool.cc:224] Handling call
0709 13:06:30.466111 (+146913us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.466602 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:30.498634 (+ 32032us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.491162 (+992528us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b1800356bc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:06:31.745518 (+254356us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.746083 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.746258 (+   175us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:31.746305 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.746403 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.831779 (+ 85376us) raft_consensus.cc:1685] finished
0709 13:06:31.868553 (+ 36774us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.349833 (+1481280us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540040d150. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:33.350039 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10764,"prepare.queue_time_us":46086,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":561,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":16945,"spinlock_wait_cycles":276840064,"thread_start_us":254010,"threads_started":1,"wal-append.queue_time_us":1240008,"wal-append.run_cpu_time_us":2776,"wal-append.run_wall_time_us":1142933}
I20260709 13:06:33.362171 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2169) took 3030 ms. Trace:
I20260709 13:06:33.362442 25368 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.362609 22008 rpcz_store.cc:276] 0709 13:06:30.331447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.968823 (+637376us) service_pool.cc:224] Handling call
0709 13:06:30.971201 (+  2378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.354092 (+382891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.361535 (+2007443us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400166f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:33.361941 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878957312}
I20260709 13:06:33.363480 25368 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.367573 25376 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:33.358731 25372 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.369051 25376 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:33.369668 25376 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:33.369140 21754 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 2 to 6, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 6 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:33.370679 25372 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:33.374115 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:33.336549 25099 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.376489 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 572) took 2900 ms. Trace:
I20260709 13:06:33.376952 21933 rpcz_store.cc:276] 0709 13:06:30.475762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.476463 (+   701us) service_pool.cc:224] Handling call
0709 13:06:30.480923 (+  4460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.524493 (+2043570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.325567 (+801074us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b5400228b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:33.376224 (+ 50657us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4688859008}
I20260709 13:06:33.378749 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:33.379513 21933 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:33.383100 22591 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:33.384799 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.385948 22587 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:33.356079 22006 rpcz_store.cc:276] 0709 13:06:30.469318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.418332 (+949014us) service_pool.cc:224] Handling call
0709 13:06:31.418695 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.269313 (+850618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.354937 (+1085624us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b5400080a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:33.355345 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877921536}
I20260709 13:06:33.401520 21856 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:33.407773 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75392.6us
I20260709 13:06:33.356271 25327 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:33.414146 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:33.411692 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2190) took 1599 ms. Trace:
I20260709 13:06:33.415879 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:33.417110 22590 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:33.417371 25329 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:33.415097 22001 rpcz_store.cc:276] 0709 13:06:31.811879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.217966 (+406087us) service_pool.cc:224] Handling call
0709 13:06:32.237188 (+ 19222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.237699 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:32.263380 (+ 25681us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.347977 (+1084597us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b18002ee7c0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:06:33.404682 (+ 56705us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.405660 (+   978us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.405795 (+   135us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:33.405841 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.405970 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.407132 (+  1162us) raft_consensus.cc:1685] finished
0709 13:06:33.411188 (+  4056us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.411438 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26068,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":2528,"raft.run_wall_time_us":10782,"spinlock_wait_cycles":76283264,"thread_start_us":55967,"threads_started":1,"wal-append.queue_time_us":1126003}
I20260709 13:06:33.419799 24751 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:33.412251 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.371119 25376 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.325346 25219 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:33.432628 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2111) took 2978 ms. Trace:
I20260709 13:06:33.433194 22004 rpcz_store.cc:276] 0709 13:06:30.319157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.592950 (+273793us) service_pool.cc:224] Handling call
0709 13:06:30.593350 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.588077 (+994727us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:31.789439 (+201362us) log.cc:844] Serialized 14 byte log entry
0709 13:06:32.566365 (+776926us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.569774 (+  3409us) log.cc:844] Serialized 26 byte log entry
0709 13:06:32.569977 (+   203us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:32.570357 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.295696 (+725339us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b18003e6080. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:06:33.296250 (+   554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.296456 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.296717 (+   261us) raft_consensus.cc:1685] finished
0709 13:06:33.296747 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.297725 (+   978us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154053,"prepare.queue_time_us":330847,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":873332,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":96726272,"thread_start_us":1788703,"threads_started":3,"wal-append.queue_time_us":764920}
I20260709 13:06:33.437501 25372 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:33.441998 25289 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:33.440728 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:33.444937 25329 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:33.446489 25206 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.420372 25325 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:06:33.449401 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:33.450331 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 126196us
I20260709 13:06:33.407868 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:33.451083 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.451699 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:33.449397 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 120455us
I20260709 13:06:33.384974 24759 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:33.420368 25368 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.452630 25113 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:33.469772 25376 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:33.465849 25202 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:33.471063 25202 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:33.460505 24759 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:33.465936 25327 consensus_queue.cc:1175] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8f3b35a35257416db3eaaf364cf6b52b" responder_term: 6 status { last_received { term: 5 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:33.465982 25113 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.486724 21933 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:33.487614 25113 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:33.484511 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.485266 25408 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:33.489809 25408 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.490386 25408 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:33.485739 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:33.485975 21934 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.568s	user 0.014s	sys 0.003s
I20260709 13:06:33.496019 21934 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:33.496085 25310 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:33.498723 21931 raft_consensus.cc:2410] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:33.499428 25103 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:33.500439 25103 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.503654 25113 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.503933 21928 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:33.506114 25103 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.514375 21931 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:33.516682 21931 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:33.517284 21931 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:33.518013 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48890us
I20260709 13:06:33.519479 25405 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:33.520375 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.520519 25405 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:33.521144 25405 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:33.522466 25405 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.523860 25405 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:33.472805 25202 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:33.536866 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61276.9us
I20260709 13:06:33.545269 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:33.564507 21927 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:33.566840 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:33.582625 25219 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:33.583303 25219 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:33.584646 25219 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.573218 25103 raft_consensus.cc:2764] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:33.598397 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:33.618014 22588 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 5.
I20260709 13:06:33.581642 21931 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:33.619143 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89563.2us
I20260709 13:06:33.621474 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:33.621676 25113 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:33.621975 21933 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:33.622682 25113 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:33.623662 25206 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:33.624053 25206 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.619164 21931 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:33.627301 25431 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:33.628242 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2202) took 1043 ms. Trace:
I20260709 13:06:33.629810 22006 raft_consensus.cc:2410] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:33.628648 21930 rpcz_store.cc:276] 0709 13:06:32.584187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.667855 (+ 83668us) service_pool.cc:224] Handling call
0709 13:06:32.668294 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.705848 (+ 37554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.705880 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.280332 (+574452us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:06:33.284402 (+  4070us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.477885 (+193483us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:33.477954 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.478076 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.624859 (+146783us) raft_consensus.cc:1685] finished
0709 13:06:33.624881 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.628035 (+  3154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10238,"raft.queue_time_us":14017,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15122,"spinlock_wait_cycles":8140032,"thread_start_us":91055,"threads_started":1,"wal-append.queue_time_us":261075}
I20260709 13:06:33.626807 25206 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:33.631565 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.629083 25408 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:33.652160 25400 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.872s	user 0.001s	sys 0.000s
W20260709 13:06:33.653438 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63251.5us
I20260709 13:06:33.653515 25113 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:33.655140 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:33.654188 25404 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:33.657128 25404 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:33.661301 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:33.662322 25099 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:33.662873 25099 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:33.663362 25099 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.667303 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:33.655836 22004 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:33.631834 25206 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:33.673314 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:33.676036 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.674693 25372 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.680464 25372 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.686002 25023 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.686604 25023 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.690781 25120 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:33.692137 25120 consensus_queue.cc:1243] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:33.693383 25394 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:33.693992 25394 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:33.694047 24751 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:33.694814 21859 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:33.695250 25394 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.696383 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.697996 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.695662 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.698249 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.698647 21855 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:33.699755 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:33.699950 21859 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:33.700714 25372 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.701269 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:33.701821 21858 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:33.684721 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 484) took 1284 ms. Trace:
I20260709 13:06:33.703471 21854 rpcz_store.cc:276] 0709 13:06:32.399888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.476731 (+ 76843us) service_pool.cc:224] Handling call
0709 13:06:33.684321 (+1207590us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540010cf54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:33.684506 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395087744}
I20260709 13:06:33.706272 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 480) took 1307 ms. Trace:
I20260709 13:06:33.707396 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:33.707968 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.708184 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:33.708658 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.688776 25023 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:33.717238 25372 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:33.718254 25372 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.718938 25372 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.720436 21927 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:33.706678 21860 rpcz_store.cc:276] 0709 13:06:32.398730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.398962 (+   232us) service_pool.cc:224] Handling call
0709 13:06:32.402624 (+  3662us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.403020 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:32.403050 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.411381 (+  8331us) log.cc:844] Serialized 26 byte log entry
0709 13:06:32.548756 (+137375us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:32.548943 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.549024 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.651157 (+1102133us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:33.705464 (+ 54307us) raft_consensus.cc:1685] finished
0709 13:06:33.705483 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.706097 (+   614us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":33534,"spinlock_wait_cycles":24692096,"thread_start_us":137082,"threads_started":1,"wal-append.queue_time_us":263040}
I20260709 13:06:33.715587 25357 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:33.716302 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.728819 21852 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:33.709502 25107 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:33.732656 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.696913 25394 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:33.748291 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47095.7us
I20260709 13:06:33.751606 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.725459 25372 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:33.737241 25023 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:33.753340 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.742555 25113 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:33.742057 25203 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:06:33.768172 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:33.768687 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.770027 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.770854 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.771394 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.771950 21852 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:33.758456 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:33.777475 21854 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:33.789146 22589 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
W20260709 13:06:33.800592 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53912.9us
I20260709 13:06:33.801209 21931 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:06:33.802852 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.804020 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.802661 25377 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 13:06:33.805584 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66874us
I20260709 13:06:33.805747 25377 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:33.806221 25377 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:33.807339 25377 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.809388 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.808771 25377 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.746021 25289 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:06:33.811451 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:33.811693 25289 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:33.812212 25289 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.812323 25310 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:33.743927 24788 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:06:33.813767 25324 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:33.814203 25324 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.814568 25226 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:33.815335 25324 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.816077 25343 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:33.816720 25343 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.818630 22585 raft_consensus.cc:1217] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Deduplicated request from leader. Original: 3.5->[5.6-10.8]   Dedup: 5.6->[8.7-10.8]
I20260709 13:06:33.819327 22585 pending_rounds.cc:85] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 6
I20260709 13:06:33.819624 22585 pending_rounds.cc:107] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:06:33.819139 25343 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.819926 22585 raft_consensus.cc:2889] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.820628 25343 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.821463 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:33.822248 21935 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:33.823318 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:33.824555 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:33.826179 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.812875 25310 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:33.827152 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.827515 25310 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.827800 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:33.828063 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.828460 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.801703 21931 pending_rounds.cc:85] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 3
I20260709 13:06:33.828541 21856 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:33.831280 21931 pending_rounds.cc:107] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:06:33.831838 21931 raft_consensus.cc:2889] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.781977 25113 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.836253 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.842450 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:33.847061 25324 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:33.847262 25387 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:33.850713 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.863317 25404 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.835631 25113 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.866751 25363 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:33.869156 24751 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:33.870762 24751 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:33.870941 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:33.873188 25287 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:06:33.875362 25182 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:33.875893 25182 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:33.854060 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:33.802598 25023 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:33.887748 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 129367us
I20260709 13:06:33.888777 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:33.889164 22591 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:33.878646 25182 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.889925 25023 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:33.891036 25329 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:33.891511 25329 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:33.892426 25329 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.893314 25408 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:33.894963 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.897130 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.897621 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:33.918378 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.918524 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:33.919134 21853 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 5.
I20260709 13:06:33.917912 25289 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.921893 25289 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:33.921739 25113 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:33.924006 25113 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:33.900558 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53543.9us
I20260709 13:06:33.932744 21855 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:33.899760 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:33.934739 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 95095.9us
I20260709 13:06:33.941547 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:33.942000 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:33.956959 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.962985 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:33.965508 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:33.972924 25404 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.978636 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2213) took 1153 ms. Trace:
I20260709 13:06:33.979082 21929 rpcz_store.cc:276] 0709 13:06:32.825271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.857122 (+ 31851us) service_pool.cc:224] Handling call
0709 13:06:33.978350 (+1121228us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540017f554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:33.978549 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823748864}
I20260709 13:06:33.980734 21929 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:33.981380 21860 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:33.932078 25203 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:06:33.983022 21930 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:33.984246 24759 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:33.985517 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 839) took 1035 ms. Trace:
I20260709 13:06:33.991358 25329 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.964612 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:33.994958 21935 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:33.982131 25331 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:34.006264 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:34.006991 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 118338us
I20260709 13:06:34.010690 22588 rpcz_store.cc:276] 0709 13:06:32.950240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.958082 (+  7842us) service_pool.cc:224] Handling call
0709 13:06:33.985297 (+1027215us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54006702d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:33.985439 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144788352}
I20260709 13:06:33.896924 25099 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.016723 21854 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:34.010219 25389 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:34.034520 24788 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:06:34.035135 24788 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:34.035373 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2097) took 1082 ms. Trace:
W20260709 13:06:34.036245 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.036676 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.037089 25239 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.037863 25239 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.004805 25357 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:34.012135 25113 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:34.025137 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.025630 24759 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:34.018671 25391 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:34.027796 25333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.012s	sys 0.020s
I20260709 13:06:34.028266 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.048790 21932 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:34.030966 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.033145 24866 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:34.037215 25436 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.404s	user 0.000s	sys 0.002s
W20260709 13:06:34.022583 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 186831us
I20260709 13:06:34.021303 25310 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.027411 21928 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:34.017917 25327 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:34.070672 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.062824 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:34.067046 21754 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 3 to 5, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 5 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:34.048362 25333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.013s	sys 0.021s
W20260709 13:06:34.084578 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.085263 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.035998 21857 rpcz_store.cc:276] 0709 13:06:32.952441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.106974 (+154533us) service_pool.cc:224] Handling call
0709 13:06:33.684535 (+577561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.713882 (+ 29347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.713909 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.714545 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.832488 (+117943us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:33.832523 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.832587 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.033162 (+200575us) raft_consensus.cc:1685] finished
0709 13:06:34.033182 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.034886 (+  1704us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540010cf50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:34.035154 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25331,"raft.queue_time_us":26620,"raft.run_cpu_time_us":487,"raft.run_wall_time_us":477,"spinlock_wait_cycles":968351616,"thread_start_us":117496,"threads_started":1,"wal-append.queue_time_us":293111}
I20260709 13:06:34.063024 25368 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:34.092677 22588 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:34.043859 25113 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.062769 25389 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.092168 25190 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.096186 25107 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:34.096565 25366 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:34.096820 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:34.096240 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50380.4us
W20260709 13:06:34.099155 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.099947 24866 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.101189 25368 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.106684 25239 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:34.111132 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:34.113363 25328 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:34.117940 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 585) took 1161 ms. Trace:
I20260709 13:06:34.090412 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.092213 21853 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:34.124630 21856 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:34.127916 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:34.129829 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.086170 24788 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:34.144811 21934 rpcz_store.cc:276] 0709 13:06:32.956045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.498480 (+542435us) service_pool.cc:224] Handling call
0709 13:06:33.498935 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.499387 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.499414 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.500113 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.600703 (+100590us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:33.600734 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.600836 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.117670 (+516834us) raft_consensus.cc:1685] finished
0709 13:06:34.117706 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.117863 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":92688,"threads_started":1,"wal-append.queue_time_us":94462}
I20260709 13:06:34.149447 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.150854 22006 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:34.086117 25327 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.146569 21900 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.153687 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.108408 25391 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.088526 25408 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:34.164063 24788 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.178295 22005 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:34.165450 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79122.2us
I20260709 13:06:34.166203 25273 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:34.168294 24866 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:34.171489 25113 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:34.174360 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77675us
I20260709 13:06:34.176829 25366 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:34.176945 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:34.144698 25328 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:34.191113 22004 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:34.194270 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.195778 25327 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:34.205281 21858 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:06:34.208052 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 340410us
I20260709 13:06:34.210819 22007 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:34.211738 24788 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.213207 25394 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:34.213790 25364 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:34.214398 25364 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.216143 25273 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:34.216586 25364 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:34.216779 25273 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.217361 25366 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:34.218621 25279 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:06:34.219594 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.220203 21934 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:06:34.221164 22001 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:34.222144 25443 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:34.223788 21932 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:34.224227 21926 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:34.228726 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:34.242017 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 115421us
I20260709 13:06:34.246354 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:34.252746 21853 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:34.254292 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.178503 25190 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.213786 25394 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:34.261348 21928 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:34.190317 25389 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.259830 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:34.273921 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 79811.7us
I20260709 13:06:34.274098 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.272341 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:34.271474 21754 catalog_manager.cc:5697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 4 to 5, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 5 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.274500 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.273630 21860 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:34.278717 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.282226 22008 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:34.256026 25331 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:34.256388 25366 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:34.283759 21929 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:34.288487 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.290349 22004 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:34.271173 25216 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.240348 25408 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.268117 25219 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:34.287461 25331 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:34.294473 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.295385 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:34.301674 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55459.9us
I20260709 13:06:34.295529 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:34.298408 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:34.298635 22001 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:34.302760 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:34.306635 21928 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:34.307432 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.305730 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:34.315537 25391 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:34.315783 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:06:34.317744 25444 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:34.320073 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.321036 25137 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:34.322593 22001 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:06:34.323109 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1801) took 2383 ms (client timeout 2077 ms). Trace:
I20260709 13:06:34.322594 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.324020 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.324694 22009 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:34.325511 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:34.325613 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.327317 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.327976 22003 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:34.329464 22000 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:34.329607 25366 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.330502 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.331275 22001 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:34.334748 25190 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.342890 25408 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.344959 25389 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:34.261880 25328 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:34.353816 25343 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.533s	user 0.005s	sys 0.006s
I20260709 13:06:34.354130 25328 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:34.297108 25216 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.323200 22006 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:06:34.357486 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:34.323637 21926 rpcz_store.cc:269] 0709 13:06:31.940138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.940949 (+   811us) service_pool.cc:224] Handling call
0709 13:06:34.322881 (+2381932us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b5400228654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:34.323018 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5119502720}
W20260709 13:06:34.368573 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63493.9us
I20260709 13:06:34.384860 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:34.385934 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:34.387723 21933 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 6: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:34.391645 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:34.306970 25310 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:34.394778 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.390709 25391 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.407156 21852 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:34.315537 25159 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:34.397424 21823 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.541s (SENT)
I20260709 13:06:34.410250 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:34.384564 25454 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:34.408327 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 114810us
I20260709 13:06:34.411798 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:34.408813 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 106176us
I20260709 13:06:34.412596 21929 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:34.414753 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.415237 24759 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:34.415565 21931 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:34.415885 24759 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:34.416992 24759 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.418251 25310 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:34.398033 25408 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.316881 25331 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.415819 25458 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.425374 24788 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:34.431077 21933 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:34.432199 25368 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.432492 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:34.433701 25394 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.440316 21754 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 6 to 7, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 7 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:34.416821 25456 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:34.436285 25256 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:34.391166 25279 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:34.428659 22006 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:34.437434 25331 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:34.437855 25389 consensus_queue.cc:1175] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8f3b35a35257416db3eaaf364cf6b52b" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:34.443450 25458 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.439512 25182 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.412180 25454 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:34.391326 25364 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:34.435189 25137 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:34.446570 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45844.9us
W20260709 13:06:34.458062 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66597us
I20260709 13:06:34.492444 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:34.458110 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.468173 22000 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:34.467425 21754 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 2 to 3, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.467957 25256 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.468915 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.469434 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:34.439400 21753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 190) took 18916 ms (client timeout 15000 ms). Trace:
I20260709 13:06:34.478015 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:34.406042 25391 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.506449 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.496457 21931 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:34.462881 25279 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.508865 22001 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:34.471390 25368 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:34.513638 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.510197 22006 raft_consensus.cc:2376] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8. Re-sending same reply.
I20260709 13:06:34.513633 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:34.457731 25219 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:06:34.519260 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:34.521777 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89425.7us
I20260709 13:06:34.524536 25239 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.525550 25458 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.529408 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.530769 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.531627 22002 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:34.532498 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.485260 25454 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.533464 22008 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:34.536432 25239 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:34.491143 25364 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:34.537812 24866 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:34.539399 24866 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:34.539798 24866 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:34.540695 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.520851 25357 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.541580 21927 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:34.541826 25357 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.542447 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.502898 25137 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:34.544086 22000 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.512364 25279 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:34.572500 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.578598 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:34.581329 25357 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.582377 25391 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:34.583221 25391 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:34.585119 25455 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:34.523471 25273 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:34.585592 25455 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:34.586051 25455 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:34.588666 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.537642 25256 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:34.594836 21933 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:34.596894 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:34.544109 25246 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:34.571044 25364 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:34.571460 25400 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:06:34.604638 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:34.606299 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:34.609731 22003 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:34.618248 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:34.619552 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:34.619885 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:34.537642 25239 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:06:34.622875 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:34.624042 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:34.627615 22005 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:34.575647 25458 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:34.540637 24866 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:34.672101 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 168162us
I20260709 13:06:34.681492 22008 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:34.637979 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:34.602051 25101 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:34.715242 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 72182.3us
I20260709 13:06:34.707578 22588 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:34.705644 22002 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:34.610306 25246 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:34.749323 22584 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:34.834163 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 245562us
I20260709 13:06:34.827574 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:34.844771 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 253530us
I20260709 13:06:34.691768 25408 raft_consensus.cc:2764] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:06:34.622249 25405 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
W20260709 13:06:34.495321 21753 rpcz_store.cc:269] 0709 13:06:15.522599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.531028 (+  8429us) service_pool.cc:224] Handling call
0709 13:06:15.935994 (+404966us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:34.439117 (+18503123us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:24.356680 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:24.357727 (+  1047us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:24.357798 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:06:24.357830 (+    32us) tablet.cc:662] Decoding operations
0709 13:06:24.359883 (+  2053us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:24.360030 (+   147us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:24.360075 (+    45us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:06:24.361919 (+  1844us) tablet.cc:701] Row locks acquired
0709 13:06:24.378728 (+ 16809us) write_op.cc:260] PREPARE: finished
0709 13:06:24.402849 (+ 24121us) write_op.cc:270] Start()
0709 13:06:24.432583 (+ 29734us) write_op.cc:276] Timestamp: P: 1783602384395608 usec, L: 0
0709 13:06:24.435743 (+  3160us) op_driver.cc:348] REPLICATION: starting
0709 13:06:24.570363 (+134620us) log.cc:844] Serialized 5958 byte log entry
0709 13:06:25.263864 (+693501us) op_driver.cc:464] REPLICATION: finished
0709 13:06:25.453229 (+189365us) write_op.cc:301] APPLY: starting
0709 13:06:25.459729 (+  6500us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:25.460186 (+   457us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:25.460340 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:25.832134 (+371794us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:25.832607 (+   473us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:06:25.832692 (+    85us) write_op.cc:312] APPLY: finished
0709 13:06:25.833294 (+   602us) log.cc:844] Serialized 107 byte log entry
0709 13:06:25.833521 (+   227us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:25.833718 (+   197us) write_op.cc:454] Released schema lock
0709 13:06:25.833823 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:25.834352 (+   529us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:25.834393 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28063,"reported_tablets":34,"spinlock_wait_cycles":15821056,"child_traces":[["op",{"apply.queue_time_us":122441,"apply.run_cpu_time_us":10390,"apply.run_wall_time_us":415952,"num_ops":15,"prepare.queue_time_us":159889,"prepare.run_cpu_time_us":25571,"prepare.run_wall_time_us":473043,"raft.queue_time_us":155037,"raft.run_cpu_time_us":20049,"raft.run_wall_time_us":50099,"replication_time_us":827926,"thread_start_us":408659,"threads_started":4,"wal-append.queue_time_us":287061,"wal-append.run_cpu_time_us":7792,"wal-append.run_wall_time_us":1925770}]]}
I20260709 13:06:34.679399 25391 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.166s
I20260709 13:06:34.677903 25458 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:34.851624 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.836165 25246 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:34.857421 25223 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:06:34.888828 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 271415us
I20260709 13:06:34.889967 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:34.862869 21754 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.864701 25190 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.622614 25404 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.224s
I20260709 13:06:34.827033 25101 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:34.858142 25464 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:06:34.853565 25400 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:06:34.846439 21929 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:34.915074 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.827112 25206 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:06:34.929685 21931 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:34.843441 24759 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
W20260709 13:06:34.943351 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77158us
I20260709 13:06:34.888783 25223 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:34.913090 25377 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:34.953375 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 104245us
I20260709 13:06:34.956964 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.914819 25101 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:34.957587 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:34.954667 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.958630 21858 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:34.925889 24866 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.866084 25473 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:06:34.864884 25454 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:34.972115 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.948523 25470 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.974443 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.971623 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:34.979741 21851 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:06:34.956792 25471 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:06:34.983363 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 151002us
I20260709 13:06:34.927608 25190 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:34.987004 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:34.989431 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:34.979074 25329 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.955s	user 0.018s	sys 0.001s
I20260709 13:06:34.995756 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:34.974418 25408 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:34.982580 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69825.8us
I20260709 13:06:34.977355 25474 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:35.005852 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:35.001672 21754 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 4 to 6, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 6 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:34.996596 25469 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:06:34.998159 25256 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:35.007876 25387 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:35.074090 25387 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:35.074589 25387 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:35.045552 25139 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:35.044582 24759 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.068879 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:35.082954 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.049904 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:35.050628 25479 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.005688 25333 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:06:35.074373 25216 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.061386 25477 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:35.088141 25477 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:35.088387 25475 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.089150 25404 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:35.089313 25477 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.090435 25377 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:06:35.062271 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66594.3us
I20260709 13:06:35.062397 25190 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:35.006376 25474 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:34.985416 25458 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.067600 25368 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:35.068288 25331 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:06:35.087044 25137 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.087827 25333 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:34.996480 25470 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.071046 25239 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:35.071856 25454 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:35.094354 22000 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.001s
I20260709 13:06:35.129829 22000 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:06:35.086412 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 109670us
W20260709 13:06:35.109534 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.131494 21935 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:35.092294 25479 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.034691 25391 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:06:35.041474 25223 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.122655 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.143695 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.134995 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.108587 25474 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.121452 25404 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:35.127944 25333 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:35.097376 25279 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.133401 25239 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:35.293478 25483 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.304629 25331 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:35.321152 25331 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.311877 25325 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.321094 25483 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.321691 25331 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:35.321913 25216 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:35.322477 25216 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.323356 25279 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.293094 22588 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:35.324142 25239 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:35.291788 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 304902us
I20260709 13:06:35.295948 25223 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:35.324576 25216 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:35.327925 25137 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:35.329210 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 198832us
I20260709 13:06:35.330096 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.331053 25246 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.331583 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.332360 25139 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:35.333845 21928 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:35.334653 25405 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.335155 25405 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.337162 25190 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:35.338094 25190 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:35.338647 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.299490 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.339711 22000 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:35.340363 25404 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.342530 25405 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.342723 25139 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.343286 25404 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:35.311215 25455 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.724s	user 0.015s	sys 0.007s
I20260709 13:06:35.344251 25455 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:35.291960 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 251814us
W20260709 13:06:35.315968 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.323407 25483 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:35.294418 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 466914us
I20260709 13:06:35.325496 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:35.321918 25325 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:35.338958 25364 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.008s
I20260709 13:06:35.343420 25139 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.178128 25329 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:35.291931 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 155319us
W20260709 13:06:35.306440 25182 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.821s	user 0.009s	sys 0.013s
I20260709 13:06:35.352427 25325 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:35.353621 25494 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:35.354202 25190 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.354507 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:35.382488 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:35.378441 22003 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:06:35.378554 25139 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.378741 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:35.379377 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.385089 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:35.383846 22001 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:35.358392 25329 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:35.377997 25494 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.390336 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.395381 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.336881 25387 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.381261 25454 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.389346 25325 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.399487 25489 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:35.385844 25491 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:35.401678 25488 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.368278 25364 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:35.454203 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46708.6us
I20260709 13:06:35.464233 21929 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:35.401983 21934 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:06:35.465425 21934 pending_rounds.cc:85] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 4
I20260709 13:06:35.465740 21934 pending_rounds.cc:107] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:06:35.466022 21934 raft_consensus.cc:2889] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:35.389462 25329 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.486526 25474 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:35.487792 25457 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.090s	user 0.002s	sys 0.001s
I20260709 13:06:35.404094 25343 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:35.368068 25391 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:06:35.489202 25343 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:35.368310 25137 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:06:35.407924 25454 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.439590 25455 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:35.501374 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.387514 25139 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.445237 25483 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.445664 25363 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:35.417984 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:35.420878 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:35.512009 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 91204us
I20260709 13:06:35.448316 25387 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:35.406710 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.490262 22589 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:35.493341 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.393807 25494 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.443723 25331 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.498891 25491 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:35.491961 25368 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:35.530910 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 113042us
I20260709 13:06:35.532467 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.500373 24866 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.496510 25391 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:35.499780 25489 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:35.540475 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:35.540657 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 134305us
W20260709 13:06:35.544780 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.512517 25488 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.516904 25470 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:35.531554 25101 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.555s	user 0.013s	sys 0.004s
I20260709 13:06:35.557828 25502 raft_consensus.cc:2764] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:06:35.558091 25159 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:35.557181 25139 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:35.558676 25159 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:35.560246 25331 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.561434 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:35.561966 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2223) took 1346 ms. Trace:
I20260709 13:06:35.562065 22007 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:35.563076 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:35.565407 25486 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:35.565960 25486 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.562378 21930 rpcz_store.cc:276] 0709 13:06:34.215956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.217859 (+  1903us) service_pool.cc:224] Handling call
0709 13:06:34.218238 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.301749 (+ 83511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.301781 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.379600 (+ 77819us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.576962 (+197362us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:34.577026 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.561590 (+984564us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b54002a8354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:35.561638 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.561680 (+    42us) raft_consensus.cc:1685] finished
0709 13:06:35.561691 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.561886 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":11854,"spinlock_wait_cycles":176214784,"thread_start_us":154844,"threads_started":1,"wal-append.queue_time_us":205273}
I20260709 13:06:35.568208 25486 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.568815 25485 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:06:35.558091 25101 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.573892 25101 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:35.574003 25343 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:35.576035 25246 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:35.577260 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:35.536965 25310 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:35.578972 24759 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.002s
I20260709 13:06:35.579352 24759 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.580519 25099 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.580739 25474 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:35.580865 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.581597 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:35.582023 25474 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.582297 22009 raft_consensus.cc:2368] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
I20260709 13:06:35.582505 25474 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.583714 25387 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:35.584580 25387 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:35.557179 25493 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:35.585203 25493 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:35.585611 25493 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:35.586874 25485 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:35.587271 25485 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:35.588071 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1866) took 1791 ms (client timeout 1886 ms). Trace:
I20260709 13:06:35.588399 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.588858 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:35.589314 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.589954 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.590541 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.590710 22002 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:35.590068 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.591290 25239 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:35.551057 25391 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.593650 25458 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.595026 25391 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.530540 25483 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.559787 25489 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.491415 25325 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.589910 22000 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:35.601788 25483 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:35.524238 25500 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:35.602267 21899 leader_election.cc:400] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
I20260709 13:06:35.602371 25500 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:35.602133 25325 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.602761 21899 leader_election.cc:403] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:06:35.602929 25500 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.603394 25325 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:35.588481 21852 rpcz_store.cc:269] 0709 13:06:33.796269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.801797 (+  5528us) service_pool.cc:224] Handling call
0709 13:06:35.587905 (+1786108us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400059bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:35.588013 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783768448}
I20260709 13:06:35.523686 25363 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.588096 25485 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.606798 25363 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.607417 25485 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:35.607846 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55713us
I20260709 13:06:35.565040 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:35.598310 25488 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.609716 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.609315 25492 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:35.610296 22006 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:35.610836 25492 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:35.611552 25488 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.611466 25492 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:35.599525 25491 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.612293 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2229) took 1233 ms. Trace:
W20260709 13:06:35.598415 25467 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.549s	user 0.000s	sys 0.018s
I20260709 13:06:35.612800 21935 rpcz_store.cc:276] 0709 13:06:34.379126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.379328 (+   202us) service_pool.cc:224] Handling call
0709 13:06:34.408549 (+ 29221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.131844 (+723295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.612066 (+480222us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b540017eb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:35.612226 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649834368}
I20260709 13:06:35.608888 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.616034 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:35.617131 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:35.621729 25442 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.403s	user 0.000s	sys 0.007s
I20260709 13:06:35.622148 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.623749 22002 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:35.582379 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.625993 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.583940 22007 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:35.624234 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.634100 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.634932 22003 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:35.638512 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.639371 25216 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:35.584719 25474 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.647702 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:35.642657 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.611351 25479 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:35.644098 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.648427 21853 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:35.644801 22009 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:35.649852 22009 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:35.645071 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:35.650784 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 85844.8us
I20260709 13:06:35.536453 25368 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.638106 25256 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:35.645256 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:35.653486 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:35.658592 21858 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.631842 25289 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.662252 21926 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:35.632114 25331 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.657181 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.662216 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.547618 24866 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.682639 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.644049 25190 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:35.687561 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.643373 25219 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:35.688872 22007 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:35.689335 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:35.666349 21851 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:35.697551 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.664634 25368 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.666661 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.662247 25206 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:35.706017 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:35.711025 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:35.652944 25488 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:06:35.711304 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:35.722508 22589 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:35.722707 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.723109 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.724002 21860 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:35.687319 25279 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.690829 25377 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.692272 25239 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.735993 21858 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:35.663722 25101 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.737856 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:35.683943 25404 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:35.706930 25470 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.668558 25289 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.750289 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:35.684593 25203 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:06:35.665091 25391 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:06:35.709439 25206 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.745268 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.752724 22590 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:35.766505 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:35.721236 25502 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.715065 25486 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:35.757476 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:35.752117 21972 connection.cc:256] RPC call timeout handler was delayed by -1.9501110380000455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
I20260709 13:06:35.728133 24759 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:35.751019 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:35.782517 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.632445 25492 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.781518 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:35.783159 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.705546 25219 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.782012 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.793146 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.794675 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.795503 21852 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:06:35.798115 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.798624 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:35.800083 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.781332 25331 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:35.773685 25206 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.741393 25279 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.782753 25387 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.781896 25486 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.809837 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.763199 25470 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:35.754238 25377 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.796334 25364 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:35.818380 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 236680us
I20260709 13:06:35.795821 25310 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:35.818176 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.797477 25483 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.802024 25363 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.802448 25223 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:35.805980 25246 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.806134 25331 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.840256 21851 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:35.856886 21929 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:35.819006 25486 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:35.814556 25394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.005s	sys 0.027s
I20260709 13:06:35.850046 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:35.799595 25405 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:06:35.817832 25481 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.798188 24759 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.857394 25478 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.839939 25310 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.875859 25216 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:35.863653 25294 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.162s
I20260709 13:06:35.877919 25279 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:35.885707 25325 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
I20260709 13:06:35.887504 22589 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:35.845382 25488 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:35.850575 25502 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.836405 25492 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:35.833527 25387 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:35.860157 25331 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:35.860648 25364 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:35.906976 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:35.863073 25394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.005s	sys 0.027s
I20260709 13:06:35.923702 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.890751 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
W20260709 13:06:35.938184 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 138333us
I20260709 13:06:35.910633 25310 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:35.958189 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:35.959761 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:35.960408 21853 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:35.970979 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.972675 25294 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.986891 25364 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:35.988022 25364 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:35.994599 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87729.1us
I20260709 13:06:35.995362 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.998127 25519 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.999048 25519 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:36.019037 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:35.902447 25481 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.021818 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:36.026424 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.992811 25101 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:36.029690 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 192480us
I20260709 13:06:35.939890 25488 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:36.025638 25494 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.001s
I20260709 13:06:36.036883 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.027168 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:35.911700 25478 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.027983 25480 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:36.049824 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.057233 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.060369 25408 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:36.029605 25101 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:36.022310 25502 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.033218 25190 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:36.066140 25391 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:36.084229 25518 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:36.084365 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.085500 25483 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.077971 25101 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:36.089474 25377 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.045713 25494 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:36.097509 25329 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
I20260709 13:06:36.098263 25485 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:36.098601 25478 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:36.045652 25203 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:36.232720 25505 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:06:36.101714 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.102635 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:36.103492 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:36.104945 22589 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:06:36.105749 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.107205 21853 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:36.110093 25515 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 13:06:36.110699 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73903.1us
I20260709 13:06:36.275679 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:36.119292 24759 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:36.120311 25507 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:06:36.139581 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.055675 25159 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:36.238903 25509 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 13:06:36.283643 21853 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:06:36.284049 22587 raft_consensus.cc:2368] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
W20260709 13:06:36.288542 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.288901 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:36.297904 21854 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:36.281159 25489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.047s	sys 0.085s
W20260709 13:06:36.284852 25256 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.000s
I20260709 13:06:36.300284 25256 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:36.305424 21855 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:36.310225 21853 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:36.311321 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 284204us
I20260709 13:06:36.314648 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.142222 25500 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.274960 25329 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:36.348552 25507 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:36.352485 21853 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:36.309607 25190 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:36.353246 22585 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
W20260709 13:06:36.302011 25489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.048s	sys 0.085s
I20260709 13:06:36.357112 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.232491 25391 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.446194 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:36.445399 21935 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:36.351213 25522 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:06:36.302657 25524 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:36.437527 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 122983us
W20260709 13:06:36.443930 25532 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:06:36.453024 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.455689 21851 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:36.349876 25408 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:36.443797 25256 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:36.458577 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.463080 25256 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:36.463990 25256 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.461198 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.462330 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.367455 25480 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.466461 21931 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:36.467470 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.438759 25519 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.468530 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:36.476557 25343 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:36.477517 25488 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.481215 25481 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.481913 25159 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:36.482522 25494 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.486230 25500 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:36.486399 25206 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.000s
I20260709 13:06:36.486985 25206 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.488067 25454 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.493130 21855 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:36.301133 25203 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.490350 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:36.499127 21926 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.500104 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:36.469417 25458 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.875s	user 0.005s	sys 0.007s
I20260709 13:06:36.465444 25256 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.500767 21857 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:36.498085 25494 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.506956 25494 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:36.513342 25493 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.927s	user 0.011s	sys 0.000s
I20260709 13:06:36.513859 25493 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.504812 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:36.522406 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61739.6us
I20260709 13:06:36.523393 21859 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:36.506129 25190 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:36.526120 21933 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:36.523324 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:36.500735 25159 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:36.503062 25481 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.500655 25206 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.504820 25488 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:36.536746 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:36.533092 25310 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.004s
I20260709 13:06:36.526701 25408 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:36.548254 21927 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:36.547684 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.553184 25536 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.553954 25536 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.554400 25333 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.555328 25333 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.493299 25480 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.563308 25546 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:36.549212 25310 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.587678 21927 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:06:36.565500 25289 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.566274 25329 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.469930 25524 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.547211 25494 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.546813 25488 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:36.575095 25486 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.000s
I20260709 13:06:36.576014 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.582095 25547 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:36.584089 25368 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.841s	user 0.010s	sys 0.001s
I20260709 13:06:36.538671 25481 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.566428 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1830) took 2209 ms. Trace:
I20260709 13:06:36.596763 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.589841 22591 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:36.556874 25536 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.561717 25159 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:36.586732 25480 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.598835 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.602299 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:36.526291 25190 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.591953 25455 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.607963 25546 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.609511 25455 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.611632 25481 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.612507 25455 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.613934 25488 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:36.615738 25492 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.677s	user 0.000s	sys 0.010s
I20260709 13:06:36.616091 25539 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:36.619675 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:36.597491 21928 rpcz_store.cc:276] 0709 13:06:34.357479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.378602 (+ 21123us) service_pool.cc:224] Handling call
0709 13:06:34.397543 (+ 18941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.334256 (+936713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.566166 (+1231910us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54002e8bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:36.566354 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152738176}
I20260709 13:06:36.600939 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:36.601516 21933 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:36.605899 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:36.602756 25519 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:36.625878 21927 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:36.627688 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.630824 21935 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:06:36.628808 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.628903 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.642190 22590 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:36.643854 21926 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:36.644908 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.600505 25256 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:36.626276 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.600112 25368 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.584043 25101 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:36.626601 25510 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.608s	user 0.004s	sys 0.000s
I20260709 13:06:36.654369 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.654423 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.621188 25547 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:36.655949 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.598320 25486 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.657954 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.672745 21931 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 10 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:36.632808 25536 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:36.648610 25256 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.676226 25256 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.659669 21928 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:36.679350 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.675062 25405 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.674834 25182 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:36.623108 25387 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.681958 25099 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:36.682403 25203 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.682961 25099 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:36.683077 25203 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:36.683434 25099 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.627708 25519 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:36.675724 25536 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:36.680815 25405 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.683585 25203 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:36.684911 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.685563 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:36.686877 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.651962 25408 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.687707 25310 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:36.687722 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.651762 25546 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:36.688392 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.656356 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.721356 21935 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:36.722390 21855 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:36.689154 25368 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.689319 21853 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:36.689414 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.724536 21857 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:36.690958 25101 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:36.627912 25524 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.694090 25387 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.695844 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:36.697636 25256 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.699337 21928 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:36.700265 21931 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:36.646950 25455 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:06:36.667841 25391 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.713179 22588 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:36.747560 21930 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:36.710762 25159 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:36.717317 21860 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:36.700767 25099 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:36.699833 25547 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:36.717290 25325 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:36.729940 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:36.751663 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:36.711032 25488 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.734813 21927 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:36.737203 22592 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:36.749300 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.754983 21855 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:36.759771 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.760480 22008 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:36.761067 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:36.764312 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:36.763535 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:36.762423 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.764271 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:36.733289 25454 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:36.767217 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:36.767728 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:36.765543 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 120723us
I20260709 13:06:36.771863 22008 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:36.768019 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.755537 25325 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.773921 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:36.774425 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:36.746968 25546 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:36.766459 22002 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:36.746755 25368 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.783460 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.784133 22005 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:36.769738 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:36.749014 25389 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.776161 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:36.778087 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.772790 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:36.776850 22000 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:36.777311 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.808557 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:36.786036 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:36.809727 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:36.809706 22007 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:36.811573 22593 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:36.814240 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.746163 25387 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.766088 25483 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:36.774549 25391 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:36.770802 25454 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.749945 25524 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:06:36.819170 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.826134 22001 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:36.765990 25481 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:36.776424 25333 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:36.776322 25325 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.763761 25488 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.846321 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.847002 22004 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:36.848781 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.816901 25529 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:36.778338 25329 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:36.783681 24759 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:36.824291 25289 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:06:36.869259 24759 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.816218 25536 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.869823 24759 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.853396 25368 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.837498 25363 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:36.872902 25368 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:36.823374 25391 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:36.838295 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.817083 25494 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:06:36.849682 22004 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:36.767207 25519 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.822969 25538 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:36.861279 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:36.858234 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.892167 22593 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.892349 22007 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:36.828629 25454 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:36.756844 25203 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.813773 25389 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.837277 25550 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:36.878913 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69260.3us
I20260709 13:06:36.878283 22592 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:36.881146 22006 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:36.839808 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.912914 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:36.817694 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50544.6us
I20260709 13:06:36.839587 25546 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:06:36.891276 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.854373 25529 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:36.873153 25363 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:36.825973 25483 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.884428 25391 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:36.894731 25331 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.914s	user 0.007s	sys 0.005s
I20260709 13:06:36.839880 25333 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:36.929706 25333 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:36.929955 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:36.930599 21928 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.916313 25561 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:36.921341 25405 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:36.933970 25488 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.873700 25159 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.941282 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.919363 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:36.939092 25333 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:36.944069 21931 raft_consensus.cc:2376] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10. Re-sending same reply.
I20260709 13:06:36.892005 25538 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.945866 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.964978 25538 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:36.969177 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:36.966302 21935 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:36.925088 25529 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:36.927994 25256 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.942937 25101 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:36.943651 25556 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:36.944446 25203 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:36.969437 25190 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:36.968245 22593 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:36.888757 25547 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.986384 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.987871 22000 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:06:36.984730 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:36.945461 25331 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:36.996320 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.001626 22008 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:36.998348 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:37.000535 22004 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:37.002919 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.009318 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:36.979046 25556 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:36.981462 25190 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:37.026011 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.030268 21928 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:37.026684 21932 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:37.035758 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 178638us
I20260709 13:06:37.011510 25494 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:37.014005 25328 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.037317 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.019423 25488 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:37.004422 25206 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:06:37.020088 25387 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:36.980765 25368 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.048233 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:37.044064 25328 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.051982 25488 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:37.047575 24909 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:06:37.063735 25328 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.062803 25325 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:37.087098 25558 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:06:37.089267 25333 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.064067 25190 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.092931 25556 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.099037 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:37.099512 25520 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.112s	user 0.003s	sys 0.000s
I20260709 13:06:37.075662 25535 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:37.130069 25557 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:06:37.040365 25458 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 13:06:37.186192 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 148958us
I20260709 13:06:37.186789 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.184916 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:37.188133 25566 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.163s	user 0.000s	sys 0.005s
I20260709 13:06:37.188393 25324 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:37.200851 21857 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.192207 25488 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.204440 25190 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.208626 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.212680 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.215296 25325 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.216189 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 515) took 1518 ms. Trace:
I20260709 13:06:37.216550 21856 rpcz_store.cc:276] 0709 13:06:35.697915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.698228 (+   313us) service_pool.cc:224] Handling call
0709 13:06:35.711976 (+ 13748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.735125 (+ 23149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.735161 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.753841 (+ 18680us) log.cc:844] Serialized 26 byte log entry
0709 13:06:35.849907 (+ 96066us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:35.849965 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.850065 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.984303 (+1134238us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:37.215910 (+231607us) raft_consensus.cc:1685] finished
0709 13:06:37.215930 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.216133 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":743,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":8130,"spinlock_wait_cycles":7358208,"thread_start_us":51633,"threads_started":1,"wal-append.queue_time_us":205080}
I20260709 13:06:37.202064 25577 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:37.206094 25099 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.210407 25324 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.213515 25556 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.241303 21857 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:37.245268 25328 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.218564 25481 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:37.243007 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 100856us
I20260709 13:06:37.251195 21858 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:06:37.240156 25324 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:37.259585 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47010us
I20260709 13:06:37.260686 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.242133 25331 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.261960 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.263165 25477 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:37.267769 25099 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.268416 25099 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:37.240360 25391 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.257026 25483 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.257344 25481 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:37.237941 25577 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.256170 25575 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:37.278527 25465 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 2.271s	user 0.003s	sys 0.003s
I20260709 13:06:37.279922 25328 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.281617 25454 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.302101 25454 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.290688 25581 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:37.268769 25099 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.248816 24759 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.303229 25581 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.303354 25099 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:37.303675 25581 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.323473 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:37.325764 22589 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:37.326241 25481 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.328222 25324 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.314631 25529 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.330919 25324 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.331048 25529 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.326719 25391 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:37.321645 25572 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:06:37.348750 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:37.349541 22592 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:37.355902 25099 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.358412 25099 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.362346 25577 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.314137 25538 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.365862 25219 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:37.366107 25538 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.337918 24759 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.373822 24866 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:37.379197 25502 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.014s	sys 0.044s
W20260709 13:06:37.383540 25502 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.014s	sys 0.044s
I20260709 13:06:37.383342 25488 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.384514 25552 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:37.386500 22592 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.387593 22592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a91310e68464d64aa8324092182518f"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:06:37.388273 22592 raft_consensus.cc:606] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:37.388590 22592 raft_consensus.cc:614] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:37.389825 25310 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:37.390244 25310 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:37.390475 25488 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.391040 25488 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.391489 25488 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.391211 25310 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.393043 25581 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.394230 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:37.394796 25488 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.395282 25581 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.395630 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:37.396914 25493 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.397648 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:37.405881 21857 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:06:37.406563 25586 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:37.406947 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:37.407572 21857 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:37.412511 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2245) took 1718 ms (client timeout 1505 ms). Trace:
W20260709 13:06:37.417327 21930 rpcz_store.cc:269] 0709 13:06:35.693592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.696028 (+  2436us) service_pool.cc:224] Handling call
0709 13:06:37.412295 (+1716267us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540017fa54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.412425 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2355248000}
W20260709 13:06:37.404776 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2124) took 1762 ms (client timeout 1584 ms). Trace:
W20260709 13:06:37.418454 21854 rpcz_store.cc:269] 0709 13:06:35.642378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.643874 (+  1496us) service_pool.cc:224] Handling call
0709 13:06:37.404590 (+1760716us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400016b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.404714 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477515648}
I20260709 13:06:37.404899 21856 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:37.401434 25310 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:37.414724 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 513) took 1717 ms (client timeout 1902 ms). Trace:
W20260709 13:06:37.422132 21859 rpcz_store.cc:269] 0709 13:06:35.697044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.697212 (+   168us) service_pool.cc:224] Handling call
0709 13:06:37.414526 (+1717314us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b540025a154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.414655 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897203072}
I20260709 13:06:37.403448 21853 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:37.423596 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:37.425964 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.427088 25223 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:37.403494 25474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.598s	user 0.017s	sys 0.029s
W20260709 13:06:37.430684 25526 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.897s	user 0.002s	sys 0.001s
I20260709 13:06:37.431041 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 868) took 1347 ms. Trace:
I20260709 13:06:37.431320 21853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:06:37.434697 21853 raft_consensus.cc:606] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.432318 21859 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.432916 21857 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:37.427628 25454 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:06:37.427517 25223 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:37.439306 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2244) took 1917 ms. Trace:
I20260709 13:06:37.439718 25223 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:37.431819 25474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.632s	user 0.018s	sys 0.029s
I20260709 13:06:37.437425 25470 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.439814 21934 rpcz_store.cc:276] 0709 13:06:35.522364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.522497 (+   133us) service_pool.cc:224] Handling call
0709 13:06:35.548614 (+ 26117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.578368 (+ 29754us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:35.806788 (+228420us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.028635 (+221847us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.028787 (+   152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.037816 (+  9029us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.038336 (+   520us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:36.038369 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.038456 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.628110 (+589654us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b180046d6e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:37.429857 (+801747us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:37.439131 (+  9274us) raft_consensus.cc:1685] finished
0709 13:06:37.439151 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.439255 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3459,"prepare.queue_time_us":281585,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":9901,"raft.queue_time_us":11181,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":1542,"spinlock_wait_cycles":12778752,"thread_start_us":391505,"threads_started":2,"wal-append.queue_time_us":619751}
I20260709 13:06:37.434365 22591 rpcz_store.cc:276] 0709 13:06:36.083955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.084076 (+   121us) service_pool.cc:224] Handling call
0709 13:06:37.430843 (+1346767us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54007300d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.430972 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019690752}
I20260709 13:06:37.439180 25454 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:37.442816 25454 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:37.443477 21854 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:37.446431 25454 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.447749 25454 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:37.449714 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 615) took 1712 ms. Trace:
I20260709 13:06:37.450147 21932 rpcz_store.cc:276] 0709 13:06:35.737385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.832685 (+ 95300us) service_pool.cc:224] Handling call
0709 13:06:37.449497 (+1616812us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540017e154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.449649 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2674735488}
I20260709 13:06:37.456023 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2254) took 1402 ms. Trace:
I20260709 13:06:37.456459 21933 rpcz_store.cc:276] 0709 13:06:36.053717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.528898 (+475181us) service_pool.cc:224] Handling call
0709 13:06:36.529379 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.626007 (+ 96628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.455787 (+829780us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b5400098dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:37.455964 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165155072}
I20260709 13:06:37.457645 25101 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:37.458658 25101 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.469509 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:37.470033 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.472828 22585 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 865) took 1410 ms (client timeout 1509 ms). Trace:
W20260709 13:06:37.473402 22585 rpcz_store.cc:269] 0709 13:06:36.062065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.062519 (+   454us) service_pool.cc:224] Handling call
0709 13:06:37.472582 (+1410063us) spinlock_profiling.cc:243] Waited 81.5 ms on lock 0x7b54006266d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.472748 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187458944}
I20260709 13:06:37.479425 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.483325 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.470939 25547 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:37.478780 25590 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:37.491482 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:37.479084 25481 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:37.493752 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.489081 25325 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.475458 25190 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:37.498718 21929 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:37.495348 21926 raft_consensus.cc:2368] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9.
I20260709 13:06:37.499343 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.484778 25533 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:37.498607 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.501724 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.482100 25256 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:37.499804 21933 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:37.500944 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.489790 24866 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.481336 25474 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:37.502345 21934 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.502440 21932 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:37.503604 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.511639 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.512220 21926 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:37.512598 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.485177 25329 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:06:37.494755 25481 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.514617 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.498317 25562 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:37.499809 25325 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:37.492264 25483 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:06:37.503098 25492 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.506336 25474 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.494273 25454 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:06:37.541184 21852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:06:37.537688 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.554193 21852 raft_consensus.cc:606] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:37.554332 21929 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:37.554581 21852 raft_consensus.cc:614] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:37.542702 21930 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:37.493697 25547 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.550359 21926 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:37.552098 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.553120 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.553546 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.555406 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.554929 22008 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:37.559154 22002 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:37.504987 24866 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:37.560168 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.561830 22009 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:37.560002 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.561362 22001 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:37.562669 22008 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.562772 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.543205 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.563620 21932 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:37.562288 22009 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:37.565610 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.566586 22002 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:37.566741 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.567270 22001 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.499006 25590 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.537336 25481 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.505340 25190 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.539987 25562 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:37.574193 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.564641 25137 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:37.575470 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.541412 25492 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.577584 25590 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.543390 25325 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.606392 25493 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.606714 25329 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.615897 25329 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.616827 21859 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:37.618284 21859 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:37.619403 21859 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:37.572599 22009 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.539749 25594 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:37.583139 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:37.590461 25547 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.591311 22004 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.599752 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:37.541085 25483 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:37.510573 25256 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.621451 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.506659 25533 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.568687 24866 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.607324 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.609891 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:37.625315 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.610110 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:37.639922 22006 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:37.609016 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:37.640697 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.640755 21926 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:37.615172 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:37.642426 21854 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.626053 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:37.612197 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.643556 22005 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:37.643463 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.610673 21856 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:37.634217 22009 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.634986 22001 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:37.645329 22585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:06:37.612449 25190 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:37.611009 25563 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.615s	user 0.002s	sys 0.000s
I20260709 13:06:37.560060 25223 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.636924 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.654335 21932 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:37.637629 22003 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:37.612411 25289 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:37.645753 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.645877 22585 raft_consensus.cc:606] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.656226 22585 raft_consensus.cc:614] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:37.636381 22007 raft_consensus.cc:2410] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:06:37.613699 25492 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:37.634809 25483 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:37.539830 25363 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.662501 22005 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:37.642576 25561 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:37.665519 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.665716 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.665820 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.671080 22001 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:37.645257 25502 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:06:37.614831 25493 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:37.610868 25137 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.673038 25470 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:37.673550 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.695822 22007 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:37.636248 25562 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.666785 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:37.667079 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.676532 21852 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:37.676476 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:37.677515 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.678753 22001 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:37.634497 25594 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.646760 25552 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 13:06:37.682794 22588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 699) took 1976 ms (client timeout 2213 ms). Trace:
I20260709 13:06:37.683185 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:37.777155 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 99734.5us
I20260709 13:06:37.777752 21935 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:37.778038 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.777223 22588 rpcz_store.cc:269] 0709 13:06:35.710547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.710689 (+   142us) service_pool.cc:224] Handling call
0709 13:06:37.682601 (+1971912us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54005dcad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:37.682738 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2302709120}
I20260709 13:06:37.686671 21823 leader_election.cc:400] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9.
I20260709 13:06:37.780223 22008 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:37.780514 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93974us
I20260709 13:06:37.782055 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.696111 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:37.783478 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:37.686872 22585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:06:37.784338 22585 raft_consensus.cc:606] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.784539 22006 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:37.784688 22585 raft_consensus.cc:614] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:37.688154 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:37.784839 22004 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:37.787258 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.689577 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.686061 22593 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:37.693316 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:37.648499 25547 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:06:37.655998 25289 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:37.790087 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 100599us
I20260709 13:06:37.673906 25190 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:37.796931 22003 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.731326 22005 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:06:37.803898 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:37.804448 22005 pending_rounds.cc:85] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 3
I20260709 13:06:37.674124 25598 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 13:06:37.801631 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.802906 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.677970 25474 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.696802 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.685807 25294 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:37.809653 25279 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:06:37.818611 25294 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:37.818944 25593 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:37.819479 25593 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:37.819792 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:37.801968 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.654171 25101 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:06:37.819578 25294 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.820747 25101 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:37.821164 25101 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:37.821273 25190 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:37.821718 25190 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:37.821491 25289 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.821997 25101 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.822808 25289 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.822589 25190 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.823179 25101 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.823801 25190 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:37.825780 25561 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:37.826516 21974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 2.213s (SENT) [suppressed 5 similar messages]
I20260709 13:06:37.826264 25561 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.697080 25256 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.675920 25343 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 13:06:37.809559 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.697227 25523 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.720297 25223 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.718886 25137 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:37.820839 25294 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.807624 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:37.811225 25475 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.846606 22006 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:37.674217 25492 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 13:06:37.698603 25329 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.828575 22008 raft_consensus.cc:2368] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:37.710665 25493 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.733482 25470 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:37.696735 25483 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.792088 25581 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:37.799248 25324 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:37.799383 25529 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.790829 25310 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.798480 25391 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:37.694626 25363 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:37.833513 21932 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:37.901753 21852 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:06:37.913429 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 86932us
I20260709 13:06:37.800153 25502 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:37.853962 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:37.697278 25577 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 13:06:37.805994 22001 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:37.833122 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:37.697021 25226 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 13:06:37.703128 25325 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.808401 25458 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:06:37.795586 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.928140 22009 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
W20260709 13:06:37.928314 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 132781us
W20260709 13:06:37.897532 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.891445 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:37.841241 25575 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.929034 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:37.915158 25502 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.822836 25552 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:37.900048 25324 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.936292 21927 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:37.919132 25533 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:37.919385 25391 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:37.920017 25256 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.919965 25481 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:37.920786 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67086.3us
I20260709 13:06:37.898025 25529 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:37.927182 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 94160.3us
I20260709 13:06:37.848136 25279 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:37.932806 25575 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:37.979349 25575 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.979357 24866 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:37.981132 21853 raft_consensus.cc:2410] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:06:37.831249 25182 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.804908 22005 pending_rounds.cc:107] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:06:37.985880 22005 raft_consensus.cc:2889] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:37.932094 25343 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:37.926334 25547 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:37.933033 25329 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:37.903487 25310 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.845963 25523 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.928493 22009 pending_rounds.cc:85] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:37.935211 25493 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:37.939222 25324 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.953233 25404 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.950294 25529 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:37.965344 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.964610 25533 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:37.939477 25587 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:06:37.969125 25325 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:37.974463 25502 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:37.977298 25604 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:37.978361 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:37.978900 25279 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.988248 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:38.019912 25279 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.993678 21854 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
W20260709 13:06:37.967698 25599 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 13:06:37.963161 25590 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:37.964202 25470 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:37.997876 22009 pending_rounds.cc:107] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:38.022660 22009 raft_consensus.cc:2889] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.024174 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:38.025471 25502 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:38.026021 25502 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:38.027033 25502 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.020042 22006 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:38.030428 25523 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:38.032742 25310 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:38.033353 25310 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.033798 25310 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.036381 25310 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.039719 25389 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.040197 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2139) took 1350 ms. Trace:
I20260709 13:06:38.040711 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:38.041381 22009 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:38.042320 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.998340 25547 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:38.042824 22009 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:37.996213 25391 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.045498 25206 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:38.045920 25206 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:38.047295 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.040565 21860 rpcz_store.cc:276] 0709 13:06:36.689400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.818705 (+129305us) service_pool.cc:224] Handling call
0709 13:06:36.818990 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.862974 (+ 43984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.863020 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:36.863076 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:36.863124 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.039958 (+1176834us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b54000fbc54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:38.040004 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.040141 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100555520}
I20260709 13:06:38.050721 22009 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:38.052086 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:38.053833 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:37.995069 25594 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:38.054536 22009 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.054572 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.055305 22585 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:38.056705 22006 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:38.057482 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:38.057859 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:38.058106 22006 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:38.058106 22003 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:38.059260 22005 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:06:38.059609 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.060691 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.059119 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:38.009800 25404 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.067534 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.009967 25493 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:38.000252 25343 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:38.068867 22003 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:38.097409 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.098412 22591 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:38.070815 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:38.068341 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 90058us
I20260709 13:06:38.099393 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.008410 25329 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:37.995123 25182 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.065635 25389 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:38.068234 24759 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:38.102970 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.103236 22009 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:38.088819 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:38.080024 25587 consensus_queue.cc:1243] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:38.069922 25523 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:38.109277 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:38.080263 25594 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:38.102941 21900 leader_election.cc:400] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
I20260709 13:06:38.116107 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:38.116396 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:38.119565 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:38.133468 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.082397 25328 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:38.107553 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:38.014254 25533 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.122587 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:38.133688 21853 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:38.106871 25519 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:38.103960 25458 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:38.152941 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 64190.4us
I20260709 13:06:38.154116 22557 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.089838 25256 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:06:38.148483 25479 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:38.153354 25289 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:38.131356 24759 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.196313 21854 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:38.120206 25593 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.153235 25328 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:38.130252 25389 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.188670 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1879) took 2450 ms. Trace:
I20260709 13:06:38.198520 21855 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:06:38.195220 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61865.8us
I20260709 13:06:38.195464 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:38.169778 25523 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:38.198375 21851 rpcz_store.cc:276] 0709 13:06:35.715850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.059618 (+343768us) service_pool.cc:224] Handling call
0709 13:06:36.059914 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.459345 (+399431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.165898 (+1706553us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b540028b754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:38.166067 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900168576}
I20260709 13:06:38.208412 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:38.209121 22004 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:38.156255 25458 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.217695 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:38.218609 21851 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:38.219138 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:38.223330 25611 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:06:38.223655 25328 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:38.225490 25610 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:06:38.205143 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 52302.9us
I20260709 13:06:38.227991 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:38.235357 22004 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:38.204892 25479 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:38.253197 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.254391 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:38.192690 25329 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.283345 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:38.284394 25475 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:38.286593 25474 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:38.287348 25256 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.287420 25182 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.287904 25256 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.289108 25479 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.290060 25256 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.169519 25493 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.256613 25458 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.207274 25481 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:38.229785 25609 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:06:38.266162 24759 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.266513 25486 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.252384 25552 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:38.269811 25533 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.273866 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:38.197047 25343 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.277540 22006 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:38.275527 25324 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.293056 22004 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:38.257849 25279 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.246793 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:38.316658 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69927.9us
I20260709 13:06:38.295753 21856 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
W20260709 13:06:38.298365 21898 connection.cc:256] RPC call timeout handler was delayed by -1.1636825349999071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260709 13:06:38.276969 24866 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.288247 25389 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.233606 25575 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.326267 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.327968 21927 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:38.326829 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:38.327087 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.353801 21927 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:38.357470 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.358716 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.359720 21929 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:38.361260 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.363282 21932 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:38.364109 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:38.364640 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 156972us
I20260709 13:06:38.364722 21856 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:38.366235 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.309038 25552 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.368216 21860 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:38.325824 25458 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.379154 25621 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:38.382863 25458 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.383352 25621 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.327184 25481 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:38.376905 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 211930us
I20260709 13:06:38.388109 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.388348 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.399159 21935 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:38.400012 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.402235 21935 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:06:38.402489 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.391829 21928 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:38.393941 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:38.354916 25619 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:38.395833 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.397971 21929 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:38.335266 25404 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:38.408699 21935 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:38.409336 21934 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:06:38.401794 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:38.410827 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.412338 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.412953 22592 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:38.413937 21930 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:38.411517 21928 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.410226 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.418532 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.406947 21927 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:38.366300 25529 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.423408 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.421506 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.422266 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.426508 21935 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:38.422756 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:38.427172 21856 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:38.398774 25481 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.423063 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.431151 21928 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:38.429476 25533 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:38.433015 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.405608 25256 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:38.430943 22589 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:38.429040 21926 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:06:38.434460 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.434448 21927 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:38.437490 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.438504 22584 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:38.439402 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.441740 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.441895 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:38.442804 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.444056 22585 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:38.420358 25523 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:38.419600 25454 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:38.459115 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.433349 25481 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:38.471706 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.433708 25389 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:38.415442 25619 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.435652 25256 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:38.475771 22590 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:38.430502 25545 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:06:38.480978 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.430135 25190 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:06:38.474364 25502 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:38.492801 25552 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.494473 25626 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:38.495750 25575 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.528023 22586 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:38.501559 22004 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:38.511191 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:38.501686 25279 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.507485 22009 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:38.508314 21927 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:38.509449 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:38.508183 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:38.534205 22003 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:38.508533 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:38.512125 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:38.531546 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.440653 25533 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:06:38.550184 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.452006 25101 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:06:38.473107 25454 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:38.576337 25454 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.577234 25529 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.508912 25458 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.578014 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.578613 21929 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:38.551949 25625 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.552845 25486 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:06:38.557875 25626 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:38.504186 25256 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.559903 25556 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:38.560180 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:38.561920 25552 raft_consensus.cc:2764] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:06:38.618827 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 58717.1us
I20260709 13:06:38.620410 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.563422 25575 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:38.565649 25324 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.566097 21933 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:38.566640 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:38.567039 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:38.573285 21926 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:06:38.572898 25294 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.000s
I20260709 13:06:38.574389 25619 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.574775 25502 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.575166 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:38.605646 21929 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:38.607242 21927 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:38.608697 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:38.616860 22589 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:38.562924 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.508963 25389 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:38.660771 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 218944us
I20260709 13:06:38.661303 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 826) took 2025 ms. Trace:
I20260709 13:06:38.669391 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:38.675529 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.679828 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.679935 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:38.681821 21825 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:38.699357 21935 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:38.705134 22585 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:38.649256 25101 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:38.712005 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.652917 25533 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:38.720750 22589 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:38.673815 22587 rpcz_store.cc:276] 0709 13:06:36.626642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.626901 (+   259us) service_pool.cc:224] Handling call
0709 13:06:37.472781 (+845880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.491074 (+ 18293us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:37.503322 (+ 12248us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.632904 (+129582us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.634427 (+  1523us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.638868 (+  4441us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.638980 (+   112us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:37.639001 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.639044 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.725349 (+ 86305us) raft_consensus.cc:1685] finished
0709 13:06:37.725378 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.042592 (+317214us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180013f540. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:38.650506 (+607914us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54006266d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:38.650692 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16706,"prepare.queue_time_us":13549,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":1440,"raft.queue_time_us":18929,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":13102,"spinlock_wait_cycles":1952928640,"thread_start_us":114430,"threads_started":1,"wal-append.queue_time_us":169483}
I20260709 13:06:38.707823 25620 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:38.728431 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:38.708712 25575 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:38.610740 25625 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.725888 25529 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:38.665436 25545 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:06:38.707082 25324 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:38.720404 25101 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:38.753093 25631 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:38.756980 25481 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:38.756434 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2257) took 1265 ms. Trace:
I20260709 13:06:38.805207 21931 rpcz_store.cc:276] 0709 13:06:37.491504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.491652 (+   148us) service_pool.cc:224] Handling call
0709 13:06:37.491976 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.649147 (+157171us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:37.781590 (+132443us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.011639 (+230049us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.070669 (+ 59030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.070717 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:38.070756 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.756193 (+685437us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400206a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:38.756243 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.756372 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67382,"prepare.queue_time_us":147655,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":94,"raft.queue_time_us":69441,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":30376,"spinlock_wait_cycles":360879488,"thread_start_us":356985,"threads_started":2,"wal-append.queue_time_us":308552}
I20260709 13:06:38.807672 25590 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:38.809419 25633 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.809985 25633 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.810166 25206 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.810668 22002 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:38.811262 25206 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:38.808776 25483 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.813212 25523 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:38.820724 25523 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.821571 25561 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.823369 25523 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.816838 22003 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:38.823953 25621 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.825798 25256 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:38.826341 22002 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:06:38.841270 22002 pending_rounds.cc:85] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 4
I20260709 13:06:38.841578 22002 pending_rounds.cc:107] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:06:38.841815 22002 raft_consensus.cc:2889] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.817086 25633 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:38.831844 25616 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.562s	user 0.001s	sys 0.002s
I20260709 13:06:38.758018 25325 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:38.835601 25533 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:38.759716 21935 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:38.808386 25590 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:38.763140 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:38.951776 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.779699 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2140) took 2090 ms. Trace:
I20260709 13:06:38.788276 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:38.932776 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.846218 22587 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:38.915858 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.914459 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.955088 22588 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:38.955487 22589 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 4.
I20260709 13:06:38.952551 21858 rpcz_store.cc:276] 0709 13:06:36.689636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.839256 (+149620us) service_pool.cc:224] Handling call
0709 13:06:36.840933 (+  1677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.251653 (+410720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.779445 (+1527792us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:38.779631 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942910208}
I20260709 13:06:38.961696 21930 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:38.781840 25614 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.517s	user 0.001s	sys 0.008s
I20260709 13:06:38.769551 25502 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.758849 25389 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.915194 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.967953 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.771111 25547 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:38.862562 25483 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:38.786768 25101 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.971426 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.779167 25625 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:38.974200 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:38.782967 25324 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:38.978992 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:38.765319 25279 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.209s
I20260709 13:06:38.960265 25486 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:38.981578 22588 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:39.025439 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.018117 25481 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.023492 25404 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:39.023497 25632 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.289s	user 0.004s	sys 0.000s
I20260709 13:06:38.771277 25190 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.205s
W20260709 13:06:39.001528 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 320038us
I20260709 13:06:38.767879 25626 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:38.972388 25226 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:38.780634 25529 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.030355 22586 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:39.019064 25590 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:38.977353 25547 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.033504 22589 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:39.038338 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.047677 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2263) took 1108 ms. Trace:
I20260709 13:06:39.055517 22001 rpcz_store.cc:276] 0709 13:06:37.938964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.966355 (+ 27391us) service_pool.cc:224] Handling call
0709 13:06:37.967137 (+   782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.967663 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.967705 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.996410 (+ 28705us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.146629 (+150219us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:38.146677 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.146738 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.781312 (+634574us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:39.036304 (+254992us) raft_consensus.cc:1685] finished
0709 13:06:39.036333 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.047584 (+ 11251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5883,"spinlock_wait_cycles":60695424,"thread_start_us":149290,"threads_started":1,"wal-append.queue_time_us":225061}
I20260709 13:06:38.977460 25483 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.057065 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.057806 25279 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.058220 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.058434 25279 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:39.058805 25483 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.059116 25279 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:39.059780 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.036405 25404 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:39.054777 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:39.060707 22588 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:39.063076 22584 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:39.065418 22584 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:39.061070 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.066538 25475 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:39.047431 25226 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.036326 25486 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.070060 25502 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:39.072561 25475 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:39.075294 25474 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.087241 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.093398 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:39.064206 25547 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.087062 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.101751 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:39.092381 25475 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:39.105773 25216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.007s	sys 0.017s
I20260709 13:06:39.073596 25324 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.102115 25256 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:39.092753 25474 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:39.127286 25216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.008s	sys 0.017s
I20260709 13:06:39.080328 25404 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.094740 25502 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.149852 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.161604 22007 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:39.162935 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.163565 22001 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:39.137190 25474 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.178191 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:39.168658 25404 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:39.186273 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:39.189460 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 230614us
I20260709 13:06:39.172245 25529 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.171224 25391 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.193118 22008 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:39.195784 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.196696 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.196877 21853 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:39.197340 21860 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:39.197634 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.198222 21853 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:39.199097 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.198302 21926 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:39.199968 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 116877us
I20260709 13:06:39.200271 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:39.195216 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.198444 21852 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:39.202868 21928 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 13:06:39.202919 21927 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:39.205612 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.206198 22008 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:39.206403 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.207636 21860 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:39.208005 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.208706 21928 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:39.201980 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.184847 25190 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.209826 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.210464 21857 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:39.212356 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.212468 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:39.214095 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 256168us
I20260709 13:06:39.215050 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2189) took 1227 ms. Trace:
I20260709 13:06:39.215490 21854 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:39.215404 22005 rpcz_store.cc:276] 0709 13:06:37.987404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.987998 (+   594us) service_pool.cc:224] Handling call
0709 13:06:38.058959 (+ 70961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.059716 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.214862 (+1155146us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b5400167450. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:39.214998 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133173504}
I20260709 13:06:39.215749 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.219130 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.220039 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.216022 25598 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:06:39.221191 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.222123 21857 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:39.231382 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:39.231648 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:39.232306 25206 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.232863 25206 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.233814 21928 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:39.234117 21926 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:39.210078 25529 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.235045 21927 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:39.239323 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.199115 25486 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.236172 21934 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:39.238369 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.234915 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:39.230010 25533 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:06:39.240758 25139 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:39.241480 25226 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:39.213186 25190 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.238240 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.248791 21931 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:39.248677 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.249303 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.255362 25556 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:39.256482 25630 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.521s	user 0.001s	sys 0.001s
I20260709 13:06:39.200760 25391 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.285137 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:39.291174 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.295643 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:39.291253 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.297868 25470 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:39.298483 25470 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:39.299419 25470 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.300642 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.301419 25343 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:39.301784 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.301904 25343 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:39.303311 25343 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.315955 21825 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:39.237854 25256 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 13:06:39.357075 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61510.3us
I20260709 13:06:39.258131 25454 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:39.359627 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:39.261175 24866 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:39.234741 25408 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:06:39.280225 25206 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.361131 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.285773 25486 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:39.243685 25481 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:06:39.314397 25519 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:39.362529 21858 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:39.260053 25190 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:39.430364 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 69317us
I20260709 13:06:39.434203 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 835) took 2211 ms. Trace:
I20260709 13:06:39.359210 25226 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:39.365339 25294 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:39.454175 25391 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.454854 25391 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:39.455304 25391 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:39.455906 25394 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:39.456403 25394 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:39.457353 25394 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.458509 25139 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:39.458896 25139 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:39.462278 25618 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:39.462729 25618 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:39.464599 25519 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:39.437337 24866 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:39.465902 24866 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.466012 25519 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.467739 25547 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:39.360098 25023 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:06:39.469686 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 837) took 1010 ms. Trace:
I20260709 13:06:39.470110 22585 rpcz_store.cc:276] 0709 13:06:38.459530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.471406 (+ 11876us) service_pool.cc:224] Handling call
0709 13:06:39.469522 (+998116us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54005631d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:39.469627 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470393216}
I20260709 13:06:39.459823 25139 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:39.440243 25638 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:06:39.472218 25139 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:39.473992 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.474727 21854 raft_consensus.cc:2368] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9.
W20260709 13:06:39.475778 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.440834 25206 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:06:39.359160 25479 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:39.362103 25454 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.439136 22593 rpcz_store.cc:276] 0709 13:06:37.157053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.185842 (+ 28789us) service_pool.cc:224] Handling call
0709 13:06:37.685880 (+500038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.788785 (+102905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.367787 (+1579002us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54005dcad0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:39.367951 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149031296}
I20260709 13:06:39.442148 25226 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:39.249874 25529 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:39.436937 25363 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.446519 25256 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:39.481088 21931 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:39.488837 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 257232us
I20260709 13:06:39.492091 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.534484 21858 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:39.535480 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.433926 25486 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:39.464648 25615 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.303s	user 0.001s	sys 0.011s
I20260709 13:06:39.538370 21931 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:39.538722 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.542207 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.438748 25502 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:39.553390 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.362237 25575 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 13:06:39.561290 25575 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:39.561710 25575 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:39.562723 25575 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.564250 25575 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:39.558298 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.548321 25363 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.567078 25363 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.567673 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.568588 25408 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:39.569172 25533 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:39.559832 25137 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:39.568926 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.573264 25137 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:39.574189 25137 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.574682 25533 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.575371 25137 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:39.548523 25256 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:39.576978 25256 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.577998 25256 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:39.574370 21972 leader_election.cc:400] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9.
I20260709 13:06:39.581276 21972 leader_election.cc:403] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:39.581848 25408 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:39.365314 25325 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 13:06:39.582456 25652 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:39.582940 25325 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:39.582990 25652 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.583570 25325 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.560309 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.560475 25502 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.485718 25479 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:39.497949 25454 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:39.444703 25561 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 13:06:39.533195 25226 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:39.530653 25101 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.002s
I20260709 13:06:39.639293 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.456113 25391 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.647804 21860 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 7: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:39.556144 25486 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:39.650909 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65786.8us
I20260709 13:06:39.641759 25479 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:39.658870 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.671144 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:39.637158 25502 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.672672 25654 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.673367 25654 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.674387 25502 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:39.675850 25654 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:39.653461 25652 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:39.676903 25529 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.643118 25488 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:39.683180 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2282) took 1581 ms. Trace:
I20260709 13:06:39.686730 21932 rpcz_store.cc:276] 0709 13:06:38.101937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.102062 (+   125us) service_pool.cc:224] Handling call
0709 13:06:38.102426 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.363674 (+261248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.682897 (+1319223us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400098dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:39.683114 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599402880}
W20260709 13:06:39.685581 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 383813us
I20260709 13:06:39.654968 25492 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.689886 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:39.659636 25533 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.655947 25101 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:39.712117 25636 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.699s	user 0.000s	sys 0.003s
I20260709 13:06:39.713392 25325 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:39.702414 25575 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:06:39.715049 25394 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:39.715468 25575 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:39.718885 25470 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:39.720539 25470 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:39.691663 25488 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.722815 25492 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:39.723763 25488 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:39.724546 25488 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:39.727561 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:39.730947 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:39.738605 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 882) took 1310 ms. Trace:
I20260709 13:06:39.704661 25529 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:39.739104 22592 rpcz_store.cc:276] 0709 13:06:38.428397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.428534 (+   137us) service_pool.cc:224] Handling call
0709 13:06:38.428841 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.429206 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:38.429234 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:38.441301 (+ 12067us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.716891 (+275590us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:38.716932 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.717014 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.711483 (+994469us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:39.731786 (+ 20303us) raft_consensus.cc:1685] finished
0709 13:06:39.731808 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.738517 (+  6709us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4209,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4701,"spinlock_wait_cycles":28013952,"thread_start_us":238000,"threads_started":1,"wal-append.queue_time_us":499439}
I20260709 13:06:39.753587 22587 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:39.758172 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 439233us
I20260709 13:06:39.759411 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.760012 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:39.760535 22008 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:39.760671 22006 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.764376 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.764652 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.765116 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.767114 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:39.767777 22004 raft_consensus.cc:2376] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10. Re-sending same reply.
I20260709 13:06:39.765223 22000 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:39.771711 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.772405 22000 raft_consensus.cc:2368] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:39.773191 22003 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:39.768868 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.775282 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.776109 22004 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:39.777292 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:39.775235 22006 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:39.779577 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.724956 25488 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:39.785884 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.789407 22008 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:39.790359 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.791237 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:39.791903 22006 raft_consensus.cc:2376] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8. Re-sending same reply.
I20260709 13:06:39.792111 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.792827 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:39.751484 25628 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:06:39.793201 22008 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 6: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:39.799471 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.800462 22008 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:39.787312 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.801285 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:39.708617 25324 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:06:39.787838 25640 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:39.801280 25294 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:39.755496 25590 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:06:39.797940 24759 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:06:39.821240 25343 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:39.790334 25363 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.866153 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:39.867690 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 108742us
I20260709 13:06:39.851297 25408 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9.
I20260709 13:06:39.879396 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:39.861788 25475 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.002s
I20260709 13:06:39.849709 25640 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:39.897224 25492 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:06:39.880661 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 143610us
I20260709 13:06:39.858798 25343 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:39.883865 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 139434us
I20260709 13:06:39.903255 25226 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.895098 25533 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:39.866176 25663 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.930428 25663 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:39.932248 25659 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:06:39.892900 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:39.901477 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:39.933840 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:39.934480 25590 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:39.934682 25216 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.934943 25590 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:39.935329 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:39.935503 25216 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:39.903622 25325 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:06:39.890975 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 189748us
W20260709 13:06:39.904907 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.910224 25363 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:39.898236 25640 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.946489 22005 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:39.946776 21860 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:06:39.948243 21860 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:39.948868 25226 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:39.949734 25226 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.949674 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:39.950278 25226 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:39.950731 25226 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:39.951000 25474 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.780s	user 0.006s	sys 0.006s
I20260709 13:06:39.951402 25474 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:39.953243 25101 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:39.896322 25631 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:39.954244 25631 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:39.955185 25631 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.939373 21899 leader_election.cc:400] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
I20260709 13:06:39.934508 25328 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:39.908339 25454 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.941849 24866 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:39.942346 25481 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:39.938030 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:39.942559 25479 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.935431 25590 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:39.953675 25663 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:39.916906 25294 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.957105 21899 leader_election.cc:403] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:39.902117 25343 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:39.958316 21935 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:39.868111 25488 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.896343 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:39.993911 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 54735.5us
I20260709 13:06:39.994488 25101 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:39.995229 25677 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 6. Current term is 7.
W20260709 13:06:39.995512 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 99273.4us
I20260709 13:06:39.996098 25677 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:39.900203 25475 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.920058 25324 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:06:39.992048 25404 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.000s
W20260709 13:06:40.005316 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.004s	sys 0.004s
I20260709 13:06:40.010170 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.010659 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.011489 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.012442 21933 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:40.012162 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.004s	sys 0.004s
I20260709 13:06:40.011683 22004 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:39.976256 25529 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.017635 21856 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:40.008581 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:40.010190 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:40.013233 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 572185us
W20260709 13:06:40.020262 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74652.4us
I20260709 13:06:39.963042 25328 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:39.958077 25664 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:06:40.024055 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:40.036068 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.027814 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:40.037357 21856 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:40.011484 25488 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.028162 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.040103 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:40.046115 22587 raft_consensus.cc:2376] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10. Re-sending same reply.
I20260709 13:06:40.040411 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.041409 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.048931 22585 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:40.043946 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:40.050302 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:40.051105 22000 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.044284 21932 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:39.996726 25481 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:06:40.014005 25325 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:40.009639 25575 consensus_queue.cc:1175] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 9 status { last_received { term: 9 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 9 index: 8 } }
W20260709 13:06:40.058876 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.061677 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:40.003037 25663 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:06:40.067591 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.067622 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.067482 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:40.091028 22588 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:40.091061 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:40.039012 25545 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:40.052349 24866 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:40.005867 25678 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:06:40.105034 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 167044us
I20260709 13:06:40.094165 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.018546 25216 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
W20260709 13:06:40.096448 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.057073 25474 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.094779 25545 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:40.103588 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 80495.6us
I20260709 13:06:40.116150 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:40.111750 25389 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:40.081710 25325 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.116788 22000 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:06:40.095552 25190 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.597s	user 0.003s	sys 0.010s
I20260709 13:06:40.090493 25391 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:40.060315 25394 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:40.126843 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44273.2us
I20260709 13:06:40.143237 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.110092 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:40.092617 25023 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:40.091028 25533 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:40.147219 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49769.5us
W20260709 13:06:40.153692 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 145188us
I20260709 13:06:40.154458 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.156226 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.157612 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.134686 24866 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.120164 25683 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:40.134262 25391 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.152696 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:40.156129 25582 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:40.144132 25619 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:40.156540 25679 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:40.156038 25519 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:40.161976 25665 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:40.134917 25389 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.142148 25190 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.184131 22588 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:40.166127 25677 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:40.169147 25683 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:40.190162 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:40.174223 25454 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:40.197566 22591 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:40.184687 25679 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.173182 25545 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.148432 25529 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.191344 25479 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.187453 25582 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.236338 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.184800 25690 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:40.237697 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:40.291307 22587 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12.
W20260709 13:06:40.291486 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 175438us
I20260709 13:06:40.293035 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:40.291302 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 146808us
I20260709 13:06:40.300236 21900 leader_election.cc:400] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12.
W20260709 13:06:40.212872 25486 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.000s
I20260709 13:06:40.300596 21900 leader_election.cc:403] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:06:40.221323 25519 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:40.196444 25683 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.221179 25679 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:40.217327 25670 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:06:40.221554 25545 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:40.286687 25475 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.222208 25626 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 13:06:40.211135 25689 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:06:40.229576 25226 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:40.312211 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 218093us
I20260709 13:06:40.207417 25695 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:40.208833 25325 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.210865 25389 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.346580 22587 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:40.223589 25190 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.335143 25519 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:40.323072 25678 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:40.350682 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 183970us
I20260709 13:06:40.206918 25391 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.336975 25700 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.371884 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.322072 25486 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.328220 25664 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.329093 25690 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:06:40.372848 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.374066 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.375442 25699 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:40.375761 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.383395 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.348920 25676 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:40.306108 24866 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:40.380754 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.387682 21929 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:40.394930 25664 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.403206 25479 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:40.403421 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.392454 21930 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:06:40.379329 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 236139us
I20260709 13:06:40.406754 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.408748 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.411347 25702 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:40.411886 25702 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.412320 25702 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.424278 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.390657 25475 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:40.382275 25692 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:40.391341 25678 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.406468 25391 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:40.404772 25663 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.419689 25645 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.455078 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.354964 25582 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.432374 25593 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:40.458573 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.443162 25649 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:40.499500 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.500350 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:40.457864 25325 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:40.404547 25676 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.458041 25677 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:40.467073 25689 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.510596 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 127255us
I20260709 13:06:40.432611 25519 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.505906 25699 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.458304 25663 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:40.552744 25590 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.000s
I20260709 13:06:40.516597 25689 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.519559 25703 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:40.558636 25343 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.006s
I20260709 13:06:40.517038 25677 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.559576 25486 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.545388 25678 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.563685 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.566150 25649 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:40.454335 25692 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.527630 25226 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:40.581807 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 124896us
I20260709 13:06:40.509764 25488 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:40.527840 25664 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.459790 25645 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.624907 25692 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.625361 22593 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:40.625823 25645 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.566052 25529 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:40.627143 25529 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.628196 25689 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.627950 25529 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.628939 25529 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.630239 25683 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.634277 25677 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:40.637197 25684 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.487s	user 0.000s	sys 0.004s
W20260709 13:06:40.640967 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 530883us
I20260709 13:06:40.641764 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.661188 25598 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:40.662153 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:40.662902 25654 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.663509 25654 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.664203 25389 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:40.665666 25654 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.664731 25389 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:40.669493 25394 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.669953 25394 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:40.670100 25389 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.608394 25703 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:40.680492 21931 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:40.682055 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.682649 21931 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:40.684161 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.686515 21931 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:40.688167 21931 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:40.607654 25699 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.690121 21931 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:40.692430 25699 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.693147 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.657534 25533 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:40.711802 22004 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:40.608544 25590 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.674688 25137 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.709547 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66096.1us
I20260709 13:06:40.657354 25669 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:40.714880 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:40.715549 21851 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.713173 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:40.618970 25343 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.694252 25486 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.717864 21860 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:40.722776 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:40.723044 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:40.723634 22007 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:40.693691 25703 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:40.674353 25101 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:06:40.698490 25587 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:40.645964 25649 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.698702 25492 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:40.709976 25556 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.734361 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:40.716988 25669 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.737497 21931 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:40.746045 25492 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.698884 25561 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:06:40.723274 25137 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.770216 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.748854 22009 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:40.783187 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.785293 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.723940 25294 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:40.792070 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.792387 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.792073 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.793825 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.794538 21851 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:40.732718 25587 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:40.795969 22588 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:40.779728 25216 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.759564 25676 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.794560 25483 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:40.727799 25486 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:40.794751 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 283217us
I20260709 13:06:40.810392 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:40.748740 25492 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:40.802668 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87747us
I20260709 13:06:40.802896 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:40.764248 25343 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.758790 25474 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.800124 25720 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:40.774736 25669 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.792157 25665 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.797091 25481 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:40.835250 22007 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:40.800416 25663 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.831846 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.750541 25679 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.762671 25556 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.866739 25556 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:40.868637 25679 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.872274 25702 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.807111 25279 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:40.809509 25216 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:40.848644 25713 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:06:40.849409 25719 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:06:40.853328 25699 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:06:40.854888 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 74016.4us
W20260709 13:06:40.884289 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 74004.1us
I20260709 13:06:40.886371 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.887459 22588 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:40.856537 25683 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.837044 25676 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:40.860239 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2201) took 1095 ms. Trace:
I20260709 13:06:40.798225 25625 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:06:40.899595 22009 rpcz_store.cc:276] 0709 13:06:39.764701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.764838 (+   137us) service_pool.cc:224] Handling call
0709 13:06:40.860043 (+1095205us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b54002710d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:40.860145 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260479744}
I20260709 13:06:40.911397 22587 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:06:40.912933 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.879504 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.914822 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.915469 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.882669 22005 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:40.916870 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.917261 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.883239 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.918470 21931 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:40.803604 25294 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.915639 22004 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
W20260709 13:06:40.889930 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55879.9us
I20260709 13:06:40.868691 25590 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.922394 22003 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:40.922911 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.918110 22005 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:40.838606 25678 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:06:40.924809 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:40.885118 25408 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.954052 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.929248 21926 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:40.897614 25256 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:40.957418 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.970022 21931 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.958488 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:40.928252 24759 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:06:40.967289 25598 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:40.973977 25598 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:40.974382 25598 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:40.975131 25731 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 13:06:40.886973 25279 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:40.959143 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:40.928086 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:40.991037 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:40.991598 22009 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:40.991525 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:40.992693 22009 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:40.960345 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.960553 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:40.992797 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:40.993726 22004 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:40.931464 22593 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:40.980039 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.840368 25474 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.011261 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.980181 25216 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:40.980692 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:40.981150 22588 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:40.977535 25137 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:40.949033 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.976847 25727 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.966372 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.898156 25676 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.989253 21930 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:40.911947 25717 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:40.977444 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:41.035277 22005 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:40.957983 25475 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.037559 21934 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:41.007990 25702 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:41.013962 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53735.8us
I20260709 13:06:41.020056 25675 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:41.015142 22004 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:41.016196 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:41.016794 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.023319 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:40.898496 24866 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.008667 24759 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:40.874646 25665 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.028597 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.969672 25256 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:41.033694 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.034384 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:41.018733 25723 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:41.046082 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:40.910838 25481 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:41.018684 25625 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:40.900508 25699 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:40.987993 25343 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:06:41.034679 25727 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.065018 25728 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.987380 25408 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.033293 25695 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:41.066062 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:41.068532 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45641.2us
I20260709 13:06:41.074457 22002 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:41.075229 25735 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:06:41.082316 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:41.026229 25720 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:06:41.034888 25676 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:41.064926 24866 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.043591 25137 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.101168 25216 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:41.064253 25640 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:41.102538 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56531.3us
I20260709 13:06:41.066668 24759 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:41.072283 25256 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:41.114248 21931 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:41.117987 22004 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:41.051942 25475 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.126053 25649 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.128239 25486 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.128489 22002 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:41.129225 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 57395.9us
W20260709 13:06:41.104490 25700 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.000s
I20260709 13:06:41.064323 25394 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.105857 25695 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:41.117264 25519 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:41.104617 25206 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:41.022215 25687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.014s	sys 0.025s
I20260709 13:06:41.088521 25481 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.088690 25625 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:41.162180 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 142940us
I20260709 13:06:41.072497 25723 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:41.219460 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.220446 21931 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:41.221745 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:40.919991 25587 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.171901 22001 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:41.171569 25328 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.177863 25475 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.178427 22002 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:41.179999 25732 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:41.181071 25733 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:06:41.183343 25408 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:41.184034 25715 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.423s	user 0.001s	sys 0.000s
W20260709 13:06:41.191591 21754 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 287) took 26701 ms (client timeout 15000 ms). Trace:
I20260709 13:06:41.192284 25669 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.201031 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:41.201867 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:41.105331 25728 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:41.255714 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 53913.3us
I20260709 13:06:41.169440 24759 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:41.205098 25405 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:06:41.225092 21926 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.150750 25137 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.227291 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:41.228715 21930 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:41.205264 25695 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.202549 25700 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12.
I20260709 13:06:41.204656 25479 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:41.210222 25687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.014s	sys 0.026s
I20260709 13:06:41.266840 25708 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.204828 25519 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:41.236217 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:41.268252 25343 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:41.241648 21931 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.225886 25723 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:41.123867 22005 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:41.262866 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61918.1us
I20260709 13:06:41.263573 21851 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:06:41.299391 21851 pending_rounds.cc:85] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 8
I20260709 13:06:41.299824 21851 pending_rounds.cc:107] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:06:41.300261 21851 raft_consensus.cc:2889] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:41.301637 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:41.264205 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:41.289531 25738 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:06:41.261188 25732 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:41.238952 25587 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:41.235592 25279 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.235836 25294 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:41.238389 25663 consensus_queue.cc:1175] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "821b3831b8a348c0826bd3a20a857f00" responder_term: 9 status { last_received { term: 9 index: 6 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:41.282402 22002 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:06:41.281952 22006 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:06:41.326637 22002 pending_rounds.cc:85] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:41.326964 22002 pending_rounds.cc:107] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:41.327196 22002 raft_consensus.cc:2889] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:41.284838 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.328222 21858 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:41.283016 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:41.287252 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.269347 25328 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:41.281208 25529 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.001s
I20260709 13:06:41.243068 25649 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:06:41.261379 25474 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.169229 25640 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:41.291188 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55077us
I20260709 13:06:41.339913 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:41.292117 21935 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
W20260709 13:06:41.244336 21754 rpcz_store.cc:269] 0709 13:06:14.489960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.490344 (+   384us) service_pool.cc:224] Handling call
0709 13:06:41.191490 (+26701146us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:35.501403 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:35.502209 (+   806us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:35.502268 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:06:35.502289 (+    21us) tablet.cc:662] Decoding operations
0709 13:06:35.503483 (+  1194us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:35.503567 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:35.503605 (+    38us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:06:35.504514 (+   909us) tablet.cc:701] Row locks acquired
0709 13:06:35.504576 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:06:35.504758 (+   182us) write_op.cc:270] Start()
0709 13:06:35.504958 (+   200us) write_op.cc:276] Timestamp: P: 1783602395504685 usec, L: 0
0709 13:06:35.504984 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:06:35.519069 (+ 14085us) log.cc:844] Serialized 2855 byte log entry
0709 13:06:36.580028 (+1060959us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:36.890881 (+310853us) op_driver.cc:464] REPLICATION: finished
0709 13:06:37.206651 (+315770us) write_op.cc:301] APPLY: starting
0709 13:06:37.213111 (+  6460us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:37.217245 (+  4134us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:37.217291 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:37.276025 (+ 58734us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:37.298948 (+ 22923us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:06:37.299006 (+    58us) write_op.cc:312] APPLY: finished
0709 13:06:37.299942 (+   936us) log.cc:844] Serialized 59 byte log entry
0709 13:06:37.300130 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:37.300254 (+   124us) write_op.cc:454] Released schema lock
0709 13:06:37.300318 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:37.300803 (+   485us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:37.300836 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5364949,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":279934,"apply.run_cpu_time_us":4561,"apply.run_wall_time_us":102546,"num_ops":7,"prepare.queue_time_us":313841,"prepare.run_cpu_time_us":34627,"prepare.run_wall_time_us":341231,"raft.queue_time_us":142033,"raft.run_cpu_time_us":36637,"raft.run_wall_time_us":344872,"replication_time_us":1353852,"spinlock_wait_cycles":8913152,"thread_start_us":695011,"threads_started":4,"wal-append.queue_time_us":466773,"wal-append.run_cpu_time_us":23430,"wal-append.run_wall_time_us":2314784}]]}
I20260709 13:06:41.264336 21860 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:06:41.297950 25708 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.313972 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.233683 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:41.314846 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 231831us
I20260709 13:06:41.342859 25652 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:41.342967 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 110288us
I20260709 13:06:41.343178 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:41.343456 25652 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.344503 25640 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.344825 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:41.345152 25640 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:41.345395 25652 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.345721 25640 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:41.339344 25492 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:41.345893 25739 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:41.346642 25492 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.346743 25640 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.347456 25687 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.347963 25739 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:41.347910 25687 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.269212 25735 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:06:41.332650 21855 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12.
I20260709 13:06:41.334048 21858 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:41.297274 25483 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:41.311192 21853 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.346323 25486 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:41.348392 25736 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:06:41.320117 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 120288us
I20260709 13:06:41.288736 25190 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.320437 25678 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.320973 25488 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.316442 25675 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:41.350636 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:41.372267 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.322147 25665 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.329135 25699 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.354393 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:41.374673 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.376017 22002 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.327569 24866 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.373085 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.383791 22585 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:41.413908 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.416189 21858 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:41.369215 25294 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:41.295141 25479 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.462829 25593 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:41.463438 25674 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.284s	user 0.002s	sys 0.006s
I20260709 13:06:41.439589 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:41.463510 25593 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.465225 25717 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:41.465756 21898 leader_election.cc:400] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12.
I20260709 13:06:41.349133 25727 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.416903 21858 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:41.493430 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1918) took 1836 ms. Trace:
W20260709 13:06:41.457952 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 105512us
I20260709 13:06:41.493829 21857 rpcz_store.cc:276] 0709 13:06:39.657232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.657451 (+   219us) service_pool.cc:224] Handling call
0709 13:06:39.678861 (+ 21410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:39.679290 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:39.679331 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:39.679890 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:06:39.990719 (+310829us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:39.990760 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.990821 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:41.411928 (+1421107us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:41.465656 (+ 53728us) raft_consensus.cc:1685] finished
0709 13:06:41.465679 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.493353 (+ 27674us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4164,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":5596,"spinlock_wait_cycles":4455680,"thread_start_us":294121,"threads_started":1,"wal-append.queue_time_us":444002}
I20260709 13:06:41.425832 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.436292 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.404666 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.469975 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.497400 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.498004 22588 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 6.
I20260709 13:06:41.472208 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:41.487177 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:41.499970 22558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.012s	sys 0.046s
I20260709 13:06:41.488840 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:41.370157 25733 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.503631 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.504541 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.373667 25652 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:41.505391 22590 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEARNER]: Advancing to term 9
W20260709 13:06:41.501659 22558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.012s	sys 0.046s
I20260709 13:06:41.513684 22584 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:41.494318 21858 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.518956 21858 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.521136 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.412981 25408 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:41.467073 25294 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.496383 25729 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:41.507611 25736 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:41.455260 25561 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:41.377617 25190 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.465698 25483 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:41.516556 25625 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.501165 25743 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:41.512017 25454 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.568871 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.455082 25739 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:06:41.389498 25678 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.385143 25675 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.554667 25294 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.524530 24866 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:41.578161 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94063.4us
I20260709 13:06:41.496699 25699 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 13:06:41.555567 25715 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:41.613401 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.614583 21930 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:41.389642 25488 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.617882 25488 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.618876 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2311) took 1240 ms. Trace:
I20260709 13:06:41.619473 21928 rpcz_store.cc:276] 0709 13:06:40.374690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.381601 (+  6911us) service_pool.cc:224] Handling call
0709 13:06:40.381984 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:40.391947 (+  9963us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:40.420265 (+ 28318us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.832568 (+412303us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:40.836886 (+  4318us) log.cc:844] Serialized 52 byte log entry
0709 13:06:40.837075 (+   189us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:40.837313 (+   238us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.860814 (+ 23501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.860887 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:41.183312 (+322425us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:41.614766 (+431454us) raft_consensus.cc:1685] finished
0709 13:06:41.614799 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.614987 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269428,"prepare.queue_time_us":51454,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":2026,"raft.queue_time_us":284438,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":26133,"spinlock_wait_cycles":86974208,"thread_start_us":169926,"threads_started":1,"wal-append.queue_time_us":290051}
I20260709 13:06:41.588526 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.625908 22002 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:41.627380 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.369936 25519 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.543273 25481 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.629819 25481 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.630498 25481 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.530699 25394 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:41.588251 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 97201.2us
W20260709 13:06:41.591941 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 107921us
I20260709 13:06:41.582063 25669 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.501341 25744 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:06:41.544487 25666 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:41.594940 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.645215 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.523001 25733 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:41.566555 21858 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:06:41.531733 25679 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:06:41.600471 22005 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:06:41.651006 21858 pending_rounds.cc:85] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 5
I20260709 13:06:41.544354 25343 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:06:41.592346 22584 raft_consensus.cc:2410] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:41.528605 25652 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.555167 25408 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.628233 22002 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:41.598202 25137 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.653627 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.582326 25561 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.574328 25749 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:06:41.601187 25743 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:41.652129 22589 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.651499 21858 pending_rounds.cc:107] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:06:41.651315 22005 pending_rounds.cc:85] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:41.667830 22005 pending_rounds.cc:107] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:06:41.668160 22005 raft_consensus.cc:2889] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:41.654264 21932 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:41.580775 25328 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:06:41.655167 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.670637 21930 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:41.655851 21851 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:06:41.656702 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.635255 25728 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.672740 21858 raft_consensus.cc:2889] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:41.646783 25593 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.600970 25454 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.682000 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.682930 22002 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:41.520853 25732 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.687080 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:41.656692 25669 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:41.694404 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.575695 25729 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.671059 25137 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.674770 25483 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.701984 25759 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.705715 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:41.702483 25742 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 13:06:41.702553 25545 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:41.703554 25746 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:41.705170 22586 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:41.688052 25363 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:41.716156 25759 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:41.717901 25751 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:41.718456 25757 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:41.718533 25729 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.719732 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:41.741164 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.741787 22008 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:41.688380 25593 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.684120 25481 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:41.737231 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 102914us
I20260709 13:06:41.746467 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.747345 21974 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:41.747640 25593 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.696745 25687 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.711993 25708 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.742789 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:41.656853 25750 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:06:41.731528 25363 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.652552 25666 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.753513 25758 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:41.753485 25523 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:41.759305 25523 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.759039 25758 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.759690 25523 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:41.756283 25363 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:41.761373 25761 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:41.763932 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.709735 25669 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.765568 22008 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:41.738596 25757 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:41.713919 25137 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.769688 25561 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.769737 25479 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.734110 25618 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.773887 22590 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:41.785714 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.780568 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.779583 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.788066 22587 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:41.787869 22584 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:41.788813 21860 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:41.788866 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.761968 25761 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.753676 25729 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:41.790957 22008 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:41.768080 25256 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.570s	user 0.005s	sys 0.006s
I20260709 13:06:41.792240 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.730479 25545 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.794320 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.794396 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:41.800184 22590 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:41.771170 25598 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.796s	user 0.006s	sys 0.006s
I20260709 13:06:41.800201 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.803157 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:41.771648 25703 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.006s	user 0.006s	sys 0.006s
I20260709 13:06:41.803196 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.745491 25695 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.806849 25703 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.807636 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.808275 22008 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:41.809176 25695 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.812745 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.813401 21857 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:41.809304 25703 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.770190 25733 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.811753 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:41.778931 25676 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.613s	user 0.005s	sys 0.005s
I20260709 13:06:41.783766 25137 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.812398 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:41.813185 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.786705 25618 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.810079 25679 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:41.774289 25190 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.782815 25666 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.783267 25763 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:41.771819 25652 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.775188 25454 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.833696 25763 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.834156 25763 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:41.821662 22009 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:41.804227 25343 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:41.811834 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:41.799149 25481 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:06:41.762844 25743 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.811486 25408 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.830780 25679 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.783998 25708 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:06:41.816185 25767 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:06:41.818173 25758 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.805256 25256 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.832921 25675 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.828408 22590 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:41.805887 25729 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:41.844695 25587 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:06:41.850520 21852 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:06:41.845114 24759 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 13:06:41.827270 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:41.853971 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:41.822274 25676 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.836562 25343 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.846864 25679 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.842957 25593 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:41.805708 25598 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.853613 25758 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:41.878284 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.870615 25767 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:41.856318 25729 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.858430 24759 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.881723 21931 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:41.882751 24759 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:41.859876 25762 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:06:41.862890 25721 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.869104 25708 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.871775 21927 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:41.862174 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.04787e+06us
W20260709 13:06:41.851694 25723 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:06:41.851735 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:41.881847 25755 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:06:41.871338 25343 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:41.891772 21856 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:41.905836 21856 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:41.894533 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.896147 21932 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:06:41.907289 22005 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:41.896821 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.908408 21928 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:41.908684 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.911191 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.911911 21928 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:41.906661 21856 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.830721 25757 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.874900 25758 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:41.922098 21856 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:41.879114 25767 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.922147 21851 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:41.933823 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:41.935009 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.923414 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.877414 25618 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:41.885381 25708 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.929504 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:41.943341 21857 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.881572 25523 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.885299 25721 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.936187 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.925786 21927 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:41.905397 24759 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:41.922303 25408 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:41.936159 21928 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:41.968317 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.969075 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.944684 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.905661 25745 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:41.935096 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:41.951284 22588 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.929181 25652 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.952127 22005 pending_rounds.cc:85] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:41.974476 22005 pending_rounds.cc:107] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:41.932992 25669 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:41.905483 25666 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:41.957141 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.964262 21856 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:41.951314 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2220) took 1038 ms. Trace:
I20260709 13:06:41.982553 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:41.935871 25767 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:41.974777 22005 raft_consensus.cc:2889] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:41.982475 22009 rpcz_store.cc:276] 0709 13:06:40.913182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.854862 (+941680us) service_pool.cc:224] Handling call
0709 13:06:41.875381 (+ 20519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.875562 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:41.891004 (+ 15442us) log.cc:844] Serialized 14 byte log entry
0709 13:06:41.950651 (+ 59647us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:41.950810 (+   159us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:41.950902 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.950997 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.951237 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13373,"prepare.queue_time_us":15713,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":15869,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":17410,"thread_start_us":45843,"threads_started":1}
I20260709 13:06:41.906399 25723 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.977824 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:41.973264 24759 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:41.986529 22584 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:06:41.988541 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.989416 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:41.992553 24759 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:41.994642 25772 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:41.996203 25256 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:41.950752 25759 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:42.003458 25723 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.003909 25758 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:41.956247 25757 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:42.035048 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:42.035722 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:42.038064 25216 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:42.041127 25759 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:42.041522 25759 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.041862 25759 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:41.950119 25618 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:42.040650 25216 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.069247 25470 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:42.066972 25699 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:42.079933 25676 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.081619 25678 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:42.082777 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.083189 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.083069 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2285) took 1204 ms. Trace:
I20260709 13:06:42.083909 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2297) took 1202 ms. Trace:
I20260709 13:06:42.083727 22007 rpcz_store.cc:276] 0709 13:06:40.878911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.880450 (+  1539us) service_pool.cc:224] Handling call
0709 13:06:40.955765 (+ 75315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.956036 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.956086 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:40.956147 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:40.956196 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.956273 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.082906 (+1126633us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b54001973d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:42.083018 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68931968}
I20260709 13:06:42.081650 25523 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:42.084401 22001 rpcz_store.cc:276] 0709 13:06:40.881967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.104284 (+222317us) service_pool.cc:224] Handling call
0709 13:06:41.165190 (+ 60906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.224877 (+ 59687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.083729 (+858852us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b540038b8d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:42.083857 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89632640}
I20260709 13:06:42.046283 25666 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:42.087815 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.088166 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.088223 25666 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.097714 25708 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:42.098361 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2291) took 1217 ms. Trace:
I20260709 13:06:41.984479 25652 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.098887 22006 rpcz_store.cc:276] 0709 13:06:40.880446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.880568 (+   122us) service_pool.cc:224] Handling call
0709 13:06:40.880856 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.326883 (+446027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.098042 (+771159us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54001e19d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:42.098279 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921700096}
W20260709 13:06:42.108621 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.123801 25645 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:41.981803 25408 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.153146 21857 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:41.981998 25745 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:42.104944 25363 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.068284 25294 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.198143 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2300) took 1315 ms. Trace:
I20260709 13:06:42.249207 22004 rpcz_store.cc:276] 0709 13:06:40.882636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.172452 (+289816us) service_pool.cc:224] Handling call
0709 13:06:41.172927 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.285684 (+112757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.285729 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.286883 (+  1154us) log.cc:844] Serialized 26 byte log entry
0709 13:06:41.529798 (+242915us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:41.529841 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.197624 (+667783us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540042b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:42.197678 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.197737 (+    59us) raft_consensus.cc:1685] finished
0709 13:06:42.197756 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.198065 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":395,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":3991,"spinlock_wait_cycles":258381568,"thread_start_us":204745,"threads_started":1,"wal-append.queue_time_us":308874}
W20260709 13:06:42.199213 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2327) took 1461 ms (client timeout 1500 ms). Trace:
W20260709 13:06:42.251524 21935 rpcz_store.cc:269] 0709 13:06:40.737924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.792146 (+ 54222us) service_pool.cc:224] Handling call
0709 13:06:42.199020 (+1406874us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400206a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:42.199145 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147363712}
I20260709 13:06:42.201015 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:42.205849 25737 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.012s	user 0.005s	sys 0.000s
I20260709 13:06:42.207816 25654 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:42.142366 25645 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:42.262562 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.299942 21932 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:06:42.281035 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.293385 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.149315 25763 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.319475 22593 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:42.295053 25699 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.372169 22008 raft_consensus.cc:2368] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
W20260709 13:06:42.336881 25775 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:06:42.305507 25654 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.414433 21926 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:42.392316 25618 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:42.377666 25699 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.518164 25669 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.460934 21747 maintenance_manager.cc:419] P 85f464a9537546118c8c1817bd5574c1: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033350
W20260709 13:06:42.648861 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.466892 25556 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:42.748760 25556 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:42.818962 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.929922 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:42.831399 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.930733 21851 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
W20260709 13:06:42.925292 21824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 2.050s (SENT) [suppressed 7 similar messages]
W20260709 13:06:42.929481 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 894422us
I20260709 13:06:42.932612 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:42.933597 21929 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:42.932736 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:42.927114 21930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.003s	sys 0.011s
W20260709 13:06:42.937229 21930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.003s	sys 0.011s
I20260709 13:06:42.927486 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:42.925747 22592 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:06:42.929382 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2186) took 1611 ms. Trace:
I20260709 13:06:42.962172 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:42.965446 21928 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:42.966411 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:42.967036 21928 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:42.956153 21859 rpcz_store.cc:276] 0709 13:06:41.318311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.323323 (+  5012us) service_pool.cc:224] Handling call
0709 13:06:41.323738 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.323994 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.324031 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.324562 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:06:41.637149 (+312587us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:41.637190 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.637265 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.929142 (+1291877us) raft_consensus.cc:1685] finished
0709 13:06:42.929168 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.929317 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":235158,"threads_started":1,"wal-append.queue_time_us":347899}
I20260709 13:06:42.926914 25598 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:42.977782 25598 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.978322 25598 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:42.784643 25575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.019s
W20260709 13:06:42.902889 25679 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.025s	user 0.010s	sys 0.002s
W20260709 13:06:42.984813 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 57434.9us
I20260709 13:06:42.985651 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:42.752767 25763 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.989334 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 765) took 2003 ms. Trace:
I20260709 13:06:42.996385 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:42.996730 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.997294 21854 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:42.598527 25669 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:42.997901 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1936) took 1023 ms. Trace:
W20260709 13:06:42.998006 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.001088 21854 rpcz_store.cc:276] 0709 13:06:41.974744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.990864 (+1016120us) service_pool.cc:224] Handling call
0709 13:06:42.997841 (+  6977us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:42.987025 25575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.004s	sys 0.020s
I20260709 13:06:42.951201 25556 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.944198 25761 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.998941 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:42.928781 25216 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.993676 22003 rpcz_store.cc:276] 0709 13:06:40.986290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.954474 (+1968184us) service_pool.cc:224] Handling call
0709 13:06:42.988615 (+ 34141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.988859 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.988899 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.988957 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:42.989029 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.989113 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.989255 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:42.982368 25408 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.982668 25645 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:42.988052 25389 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:42.982578 25363 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:42.989050 25294 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:42.990793 25652 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.058784 25669 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:42.989318 25679 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:43.003410 25776 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.859s	user 0.000s	sys 0.001s
I20260709 13:06:42.991005 25745 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.076663 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:42.990375 25023 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:43.080701 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1886) took 1101 ms. Trace:
I20260709 13:06:43.149602 21929 rpcz_store.cc:276] 0709 13:06:41.979736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.954078 (+974342us) service_pool.cc:224] Handling call
0709 13:06:43.080612 (+126534us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:43.082019 25740 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.794s	user 0.002s	sys 0.011s
I20260709 13:06:43.072129 25556 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:42.980952 25598 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:43.108880 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.135102 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:43.158670 22003 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:43.092128 25216 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:43.146842 25767 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.113s	user 0.013s	sys 0.000s
I20260709 13:06:43.103245 25645 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:43.108234 25699 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:43.162467 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 237182us
I20260709 13:06:43.165405 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 662) took 1042 ms. Trace:
I20260709 13:06:43.150648 25745 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:43.175860 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 190279us
I20260709 13:06:43.169479 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:43.167750 21929 rpcz_store.cc:276] 0709 13:06:42.122889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.150390 (+1027501us) service_pool.cc:224] Handling call
0709 13:06:43.164656 (+ 14266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.164946 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.164986 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.165071 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:43.165123 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.165188 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.165339 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.152020 25669 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:43.222489 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53107.5us
I20260709 13:06:43.152223 25023 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:43.166536 25645 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:43.162840 25761 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.233439 22003 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:43.167475 25598 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.176328 25745 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.234887 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 778) took 1151 ms. Trace:
I20260709 13:06:43.161787 25556 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.175685 25763 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:43.236493 25777 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:06:43.229686 25023 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:43.225024 25683 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:43.250710 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2289) took 2370 ms. Trace:
W20260709 13:06:43.252796 22009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.135s	user 0.002s	sys 0.000s
I20260709 13:06:43.249583 25763 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:43.253124 22009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.003s	sys 0.000s
I20260709 13:06:43.254225 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2225) took 2296 ms. Trace:
I20260709 13:06:43.254541 22009 rpcz_store.cc:276] 0709 13:06:40.957715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.985478 (+1027763us) service_pool.cc:224] Handling call
0709 13:06:42.064690 (+ 79212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.064961 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:42.092430 (+ 27469us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.253795 (+1161365us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.253934 (+   139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.253974 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:43.254016 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.254074 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.254178 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32302,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":110,"thread_start_us":1160714,"threads_started":1,"wal-append.queue_time_us":1160534}
I20260709 13:06:43.176247 25767 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.241842 25761 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.177348 25720 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:06:43.168385 25216 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.233610 25645 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.251024 22000 rpcz_store.cc:276] 0709 13:06:40.879944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.881742 (+  1798us) service_pool.cc:224] Handling call
0709 13:06:40.881998 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:40.962286 (+ 80288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.962333 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:40.967386 (+  5053us) log.cc:844] Serialized 26 byte log entry
0709 13:06:41.162646 (+195260us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:41.170246 (+  7600us) log.cc:844] Serialized 14 byte log entry
0709 13:06:41.170483 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.170576 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.953486 (+1782910us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:43.248090 (+294604us) raft_consensus.cc:1685] finished
0709 13:06:43.250545 (+  2455us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.250664 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79895,"prepare.queue_time_us":5068,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":13381,"raft.queue_time_us":94541,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":214,"spinlock_wait_cycles":2598272,"thread_start_us":154009,"threads_started":1,"wal-append.queue_time_us":232750}
W20260709 13:06:43.256599 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 323914us
I20260709 13:06:43.245052 25784 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.225409 25669 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.246793 25745 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:43.254935 25763 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:43.264170 25666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.003s	sys 0.000s
W20260709 13:06:43.265440 25666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.003s	sys 0.000s
I20260709 13:06:43.266398 25556 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.266877 25721 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.267184 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1931) took 1333 ms. Trace:
I20260709 13:06:43.267530 21856 rpcz_store.cc:276] 0709 13:06:41.933994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.988245 (+ 54251us) service_pool.cc:224] Handling call
0709 13:06:43.267002 (+1278757us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540025c454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.267129 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2937505664}
I20260709 13:06:43.269450 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:43.250226 25683 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:43.273351 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 457490us
I20260709 13:06:43.270622 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:43.279247 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:43.246296 25598 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.280865 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:43.243049 22003 rpcz_store.cc:276] 0709 13:06:42.083177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.100572 (+1017395us) service_pool.cc:224] Handling call
0709 13:06:43.234817 (+134245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.254676 25790 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.283993 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 770) took 1772 ms. Trace:
I20260709 13:06:43.284718 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 769) took 1791 ms. Trace:
I20260709 13:06:43.284466 22003 rpcz_store.cc:276] 0709 13:06:41.511783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.281980 (+1770197us) service_pool.cc:224] Handling call
0709 13:06:43.282335 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.282561 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.282596 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.282649 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:43.282692 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.282755 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.282871 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.285157 22009 rpcz_store.cc:276] 0709 13:06:41.493668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.273184 (+1779516us) service_pool.cc:224] Handling call
0709 13:06:43.284283 (+ 11099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.284446 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.284468 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.284506 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:43.284535 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.284585 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.284671 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.274212 25683 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:43.290592 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 772) took 1765 ms. Trace:
I20260709 13:06:43.292184 22003 rpcz_store.cc:276] 0709 13:06:41.525476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.285891 (+1760415us) service_pool.cc:224] Handling call
0709 13:06:43.290087 (+  4196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.290289 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.290312 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.290351 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:43.290379 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.290430 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.290540 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:43.289710 25759 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.247s	user 0.008s	sys 0.004s
I20260709 13:06:43.295400 25759 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.297266 25717 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:43.297657 25717 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:43.270768 25784 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:43.298492 25717 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.290269 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1929) took 1378 ms. Trace:
I20260709 13:06:43.293677 22009 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:43.299640 25717 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.299960 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2310) took 1743 ms. Trace:
I20260709 13:06:43.299752 25759 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.300354 22009 rpcz_store.cc:276] 0709 13:06:41.556697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.286506 (+1729809us) service_pool.cc:224] Handling call
0709 13:06:43.293385 (+  6879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.299711 (+  6326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.299908 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.299652 21851 rpcz_store.cc:276] 0709 13:06:41.911675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.990758 (+1079083us) service_pool.cc:224] Handling call
0709 13:06:43.003319 (+ 12561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.004766 (+  1447us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:43.010056 (+  5290us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.054819 (+ 44763us) log.cc:844] Serialized 26 byte log entry
0709 13:06:43.169331 (+114512us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:43.169374 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.169446 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.290059 (+120613us) raft_consensus.cc:1685] finished
0709 13:06:43.290078 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.290202 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":6835,"raft.run_wall_time_us":64385,"thread_start_us":113961,"threads_started":1,"wal-append.queue_time_us":193304}
I20260709 13:06:43.283543 25790 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.272500 25669 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:43.281121 25619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.001s	sys 0.004s
W20260709 13:06:43.307312 25779 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:06:43.305729 25789 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:43.307591 25784 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.307935 25789 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.308401 25789 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:43.308606 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2224) took 2387 ms. Trace:
I20260709 13:06:43.308960 22005 rpcz_store.cc:276] 0709 13:06:40.921098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.925154 (+1004056us) service_pool.cc:224] Handling call
0709 13:06:41.951992 (+ 26838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.984682 (+ 32690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.984712 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.985149 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.141499 (+156350us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:42.141543 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.141611 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.308368 (+1166757us) raft_consensus.cc:1685] finished
0709 13:06:43.308389 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.308545 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":156039,"threads_started":1,"wal-append.queue_time_us":1167537}
I20260709 13:06:43.309885 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2316) took 1302 ms. Trace:
I20260709 13:06:43.310935 22005 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:43.316141 25767 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:43.324985 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44189.3us
I20260709 13:06:43.331231 22005 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:06:43.348019 25800 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:43.348838 25800 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.310390 22003 rpcz_store.cc:276] 0709 13:06:42.008052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.308956 (+1300904us) service_pool.cc:224] Handling call
0709 13:06:43.309300 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.309508 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.309542 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.309596 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:43.309638 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.309702 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.309825 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.351652 25800 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.353207 25799 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:06:43.317560 25619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.002s	sys 0.004s
I20260709 13:06:43.302382 25679 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.324558 25791 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:43.368960 22005 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:06:43.377720 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.355372 25767 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:43.367069 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 143529us
I20260709 13:06:43.373983 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1873) took 1481 ms. Trace:
I20260709 13:06:43.354063 25363 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:43.395938 25363 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:43.397518 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 642) took 2484 ms (client timeout 2057 ms). Trace:
W20260709 13:06:43.398269 21934 rpcz_store.cc:269] 0709 13:06:40.913388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.915159 (+  1771us) service_pool.cc:224] Handling call
0709 13:06:43.397294 (+2482135us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540003b954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.397450 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278757120}
I20260709 13:06:43.397183 21930 rpcz_store.cc:276] 0709 13:06:41.892115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.892226 (+   111us) service_pool.cc:224] Handling call
0709 13:06:41.921619 (+ 29393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.921844 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:42.273485 (+351641us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.366019 (+1092534us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.373507 (+  7488us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.373580 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:43.373644 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.373743 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.373907 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62323,"prepare.queue_time_us":56077,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3990,"raft.queue_time_us":249407,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":12222,"spinlock_wait_cycles":5750144,"thread_start_us":1364545,"threads_started":3,"wal-append.queue_time_us":822944}
W20260709 13:06:43.399186 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50945.5us
I20260709 13:06:43.399576 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:43.398584 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:43.376796 25721 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:43.388453 25784 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:43.432734 25784 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.436221 25784 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.437111 25796 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:43.438354 22003 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:43.437909 25796 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.439447 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.440590 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:43.437594 22005 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:43.441901 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:43.442914 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:43.444701 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 869) took 1660 ms. Trace:
I20260709 13:06:43.445314 22584 rpcz_store.cc:276] 0709 13:06:41.784675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.788720 (+  4045us) service_pool.cc:224] Handling call
0709 13:06:41.789027 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.987033 (+198006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.444337 (+1457304us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b5400684f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:43.444619 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453667072}
I20260709 13:06:43.447827 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1881) took 1503 ms. Trace:
I20260709 13:06:43.448542 22003 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:43.448414 21926 rpcz_store.cc:276] 0709 13:06:41.944171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.944452 (+   281us) service_pool.cc:224] Handling call
0709 13:06:43.447604 (+1503152us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540003c354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.447750 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017993984}
I20260709 13:06:43.450206 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.451035 21926 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:43.452239 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:43.453576 22591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 913) took 2081 ms (client timeout 2649 ms). Trace:
I20260709 13:06:43.454074 22003 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:06:43.454200 22591 rpcz_store.cc:269] 0709 13:06:41.372421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.372819 (+   398us) service_pool.cc:224] Handling call
0709 13:06:43.453324 (+2080505us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54006676d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.453501 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934520576}
I20260709 13:06:43.455232 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.455830 22003 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:43.456477 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 655) took 2139 ms. Trace:
I20260709 13:06:43.456661 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.457373 22003 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:43.457144 21931 rpcz_store.cc:276] 0709 13:06:41.316792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.316959 (+   167us) service_pool.cc:224] Handling call
0709 13:06:41.336134 (+ 19175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.882235 (+546101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.456133 (+1573898us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54000983d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:43.456394 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254327168}
I20260709 13:06:43.458431 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.459239 22003 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:43.409920 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.459779 21932 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:43.460274 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1888) took 1378 ms. Trace:
I20260709 13:06:43.460567 21932 rpcz_store.cc:276] 0709 13:06:42.082162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.409568 (+1327406us) service_pool.cc:224] Handling call
0709 13:06:43.460216 (+ 50648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.461146 22005 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:43.461422 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.461982 21932 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:43.462994 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.463631 21932 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:06:43.464457 25734 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 2.271s	user 0.011s	sys 0.013s
W20260709 13:06:43.411830 22593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 724) took 2040 ms (client timeout 2148 ms). Trace:
I20260709 13:06:43.413069 21933 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:43.415066 22589 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:43.417343 25790 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:43.417573 25343 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.492s	user 0.014s	sys 0.000s
I20260709 13:06:43.379961 25598 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.425462 25761 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.426651 25363 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.428395 25683 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.406131 25216 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.429939 25645 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.431236 25619 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:43.464608 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.503297 21932 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:43.503705 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:43.428938 25023 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:43.505151 22593 rpcz_store.cc:269] 0709 13:06:41.371496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.371999 (+   503us) service_pool.cc:224] Handling call
0709 13:06:43.411591 (+2039592us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54005631d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.411760 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173252352}
I20260709 13:06:43.505597 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:43.458993 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.509763 21931 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:43.491937 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:43.504786 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.507318 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:43.513211 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.475019 25492 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.478600 25486 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:43.482748 25405 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:43.514545 25486 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.514943 25405 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:43.516222 25771 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.663s	user 0.000s	sys 0.015s
I20260709 13:06:43.468602 25523 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:43.517463 25523 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.518391 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2231) took 2147 ms. Trace:
I20260709 13:06:43.519400 25523 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.509984 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:43.391686 25699 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:43.518909 22006 rpcz_store.cc:276] 0709 13:06:41.370580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.517498 (+2146918us) service_pool.cc:224] Handling call
0709 13:06:43.517837 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.518025 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.518056 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.518103 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:43.518140 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.518199 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.518320 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.525445 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 661) took 1403 ms. Trace:
W20260709 13:06:43.521238 22008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2314) took 1711 ms (client timeout 2037 ms). Trace:
I20260709 13:06:43.514598 25475 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:43.521628 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:43.508520 25757 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.527601 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.525807 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.528821 21932 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:43.522265 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 860) took 2017 ms. Trace:
I20260709 13:06:43.529232 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:43.529836 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:43.523305 25730 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 2.555s	user 0.000s	sys 0.019s
I20260709 13:06:43.526918 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:43.508741 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 161358us
W20260709 13:06:43.526346 22008 rpcz_store.cc:269] 0709 13:06:41.809413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.811538 (+  2125us) service_pool.cc:224] Handling call
0709 13:06:43.521051 (+1709513us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54001982d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.521183 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190363008}
I20260709 13:06:43.536366 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.528872 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.524976 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.529431 22588 rpcz_store.cc:276] 0709 13:06:41.504462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.504595 (+   133us) service_pool.cc:224] Handling call
0709 13:06:41.505017 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.972062 (+467045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.522000 (+1549938us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54005000d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:43.522205 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003752832}
I20260709 13:06:43.531293 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.539795 22006 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:43.534163 22591 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:43.532027 25492 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:43.533737 22586 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:43.482679 25785 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.541818 25785 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.542346 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.525964 21928 rpcz_store.cc:276] 0709 13:06:42.122511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.968027 (+845516us) service_pool.cc:224] Handling call
0709 13:06:42.968415 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.272327 (+303912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.272369 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.277139 (+  4770us) log.cc:844] Serialized 26 byte log entry
0709 13:06:43.391269 (+114130us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:43.391309 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.391388 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.525218 (+133830us) raft_consensus.cc:1685] finished
0709 13:06:43.525237 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.525387 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29668,"raft.queue_time_us":262145,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":3234,"thread_start_us":359466,"threads_started":2,"wal-append.queue_time_us":240712}
I20260709 13:06:43.543524 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.543893 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:43.544127 21930 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:43.545215 25761 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:43.545693 25761 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:43.484699 25294 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.532711 25405 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.539908 21932 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:06:43.462003 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:43.540663 22008 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:43.522994 25486 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.546226 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 666) took 1391 ms. Trace:
I20260709 13:06:43.534413 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:43.508296 25619 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:43.482163 25545 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.513844 22003 raft_consensus.cc:2368] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9.
I20260709 13:06:43.512540 25796 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:43.544509 25789 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.512060 25343 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.413601 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2180) took 2149 ms. Trace:
W20260709 13:06:43.580726 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.139231 21852 rpcz_store.cc:276] 0709 13:06:41.264211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.264395 (+   184us) service_pool.cc:224] Handling call
0709 13:06:41.264643 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.850925 (+586282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.413315 (+1562390us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b540028cb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:43.413521 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346499456}
I20260709 13:06:43.557117 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:44.106410 21898 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.000s	sys 0.018s
I20260709 13:06:44.176283 25343 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:43.730966 25475 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:43.546090 25761 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.159042 25752 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:43.534394 25699 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.634577 25745 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:43.548339 25757 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:43.668839 25785 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:43.573894 25795 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:06:43.787988 25023 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.190729 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:43.739290 25683 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:43.666826 25794 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:06:44.109225 21860 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:43.859696 25729 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:43.558166 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:43.728981 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 92230.4us
I20260709 13:06:43.730758 25523 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.603060 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.180541 25767 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:44.200309 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 663) took 2075 ms. Trace:
W20260709 13:06:44.177419 21898 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.005s	sys 0.026s
I20260709 13:06:43.600320 25294 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.182719 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:44.183883 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.186324 22006 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:44.185375 25619 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:44.132632 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 733814us
I20260709 13:06:44.193653 21931 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:44.195390 25745 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.201503 25645 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:43.798156 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 865) took 2018 ms. Trace:
W20260709 13:06:44.204958 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 65332.6us
I20260709 13:06:44.208251 25523 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.211423 25523 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.212590 22584 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:44.202569 21929 rpcz_store.cc:276] 0709 13:06:42.123178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.224219 (+1101041us) service_pool.cc:224] Handling call
0709 13:06:43.235472 (+ 11253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.272865 (+ 37393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.272901 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.272948 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:43.272980 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.198350 (+925370us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b54003061d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:44.198444 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.198789 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85530880}
I20260709 13:06:43.571576 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.218017 22005 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:44.217898 25729 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:44.219192 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.221308 25216 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:44.221489 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1890) took 2012 ms (client timeout 1923 ms). Trace:
I20260709 13:06:44.217788 22589 rpcz_store.cc:276] 0709 13:06:41.779875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.781196 (+  1321us) service_pool.cc:224] Handling call
0709 13:06:43.414805 (+1633609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.508240 (+ 93435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.797807 (+289567us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54005631d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:43.798039 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3717405952}
I20260709 13:06:44.222194 25216 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.224399 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:44.222684 21935 rpcz_store.cc:269] 0709 13:06:42.208531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.220912 (+2012381us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:44.221112 (+   200us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:06:44.227798 22585 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:06:44.232276 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 880583us
I20260709 13:06:44.233457 25325 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:44.234421 25325 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.237114 25575 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:44.234210 25729 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.232614 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 767) took 3223 ms. Trace:
I20260709 13:06:44.225577 25789 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.240335 22007 rpcz_store.cc:276] 0709 13:06:41.008645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.231077 (+3222432us) service_pool.cc:224] Handling call
0709 13:06:44.231589 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.232042 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.232070 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.232199 (+   129us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:44.232243 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.232300 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.232414 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:44.247064 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.220094 25767 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.245618 25582 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 13:06:44.251549 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 690332us
I20260709 13:06:44.252452 25325 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.230031 21930 rpcz_store.cc:276] 0709 13:06:42.155336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.544893 (+1389557us) service_pool.cc:224] Handling call
0709 13:06:43.545696 (+   803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.545871 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.545898 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.545938 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:43.545975 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.546053 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.546172 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:44.244822 25727 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:44.239112 25683 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:06:44.292323 25519 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.298403 25654 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.301141 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 777) took 2470 ms. Trace:
I20260709 13:06:44.273442 25727 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.314888 22588 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:44.305374 25727 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.308516 25619 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.337236 25752 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:44.339803 22593 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:44.352669 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.353289 25294 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:44.353703 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 108399us
I20260709 13:06:44.353890 25645 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:44.354537 25645 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.355031 25645 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.357360 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.346524 25729 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.362531 25767 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.363268 25767 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:44.346750 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:44.366384 25683 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:44.366523 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71871.6us
I20260709 13:06:44.366967 25683 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:44.368407 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:44.369277 25654 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:44.371619 25721 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:44.374430 21857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1938) took 2372 ms (client timeout 2166 ms). Trace:
W20260709 13:06:44.374945 21857 rpcz_store.cc:269] 0709 13:06:42.002363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.373954 (+2371591us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:44.374128 (+   174us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:06:44.376933 21935 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:44.372429 25721 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.301616 22009 rpcz_store.cc:276] 0709 13:06:41.778480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.306910 (+1528430us) service_pool.cc:224] Handling call
0709 13:06:43.307292 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.307902 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:43.308384 (+   482us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.363809 (+ 55425us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.785973 (+422164us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b1800309dc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:44.235795 (+449822us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:06:44.236627 (+   832us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.239481 (+  2854us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:44.241712 (+  2231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.245142 (+  3430us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.245173 (+    31us) raft_consensus.cc:1685] finished
0709 13:06:44.245180 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.248400 (+  3220us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":205,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":223,"spinlock_wait_cycles":14285824,"thread_start_us":54221,"threads_started":1,"wal-append.queue_time_us":49269}
I20260709 13:06:44.333420 25575 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:06:44.387168 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.388054 22008 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.391566 25721 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.392716 25654 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:44.412047 25654 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:44.393360 21929 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:44.412180 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.394445 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.395927 25699 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:44.396402 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 645) took 3482 ms (client timeout 2198 ms). Trace:
I20260709 13:06:44.398507 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.400429 25757 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.401664 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.402307 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 556) took 2313 ms. Trace:
I20260709 13:06:44.404307 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.411103 22008 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:44.405484 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.412308 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:44.413671 21933 rpcz_store.cc:269] 0709 13:06:40.914145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.916572 (+  2427us) service_pool.cc:224] Handling call
0709 13:06:44.396008 (+3479436us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:44.396191 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5738527488}
W20260709 13:06:44.418922 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59882us
I20260709 13:06:44.418627 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.415884 21860 rpcz_store.cc:276] 0709 13:06:42.088783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.465895 (+1377112us) service_pool.cc:224] Handling call
0709 13:06:43.466413 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.203683 (+737270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.401863 (+198180us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54000591d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:44.402088 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476540800}
I20260709 13:06:44.412562 25654 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.426215 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.427033 21934 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
W20260709 13:06:44.427376 25816 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:06:44.428035 22586 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 6.
I20260709 13:06:44.428961 25545 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.429553 25545 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.429961 25545 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.430217 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 867) took 2649 ms. Trace:
I20260709 13:06:44.430738 22592 rpcz_store.cc:276] 0709 13:06:41.780502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.780626 (+   124us) service_pool.cc:224] Handling call
0709 13:06:41.780936 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.955366 (+1174430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.429470 (+1474104us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:44.429883 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222627712}
I20260709 13:06:44.431859 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2309) took 3128 ms. Trace:
I20260709 13:06:44.432300 22001 rpcz_store.cc:276] 0709 13:06:41.303150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.212346 (+2909196us) service_pool.cc:224] Handling call
0709 13:06:44.212863 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:44.222447 (+  9584us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:44.222483 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:44.225414 (+  2931us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.319121 (+ 93707us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:44.319177 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.319278 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.431510 (+112232us) raft_consensus.cc:1685] finished
0709 13:06:44.431532 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.431672 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1203,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":9380,"thread_start_us":77715,"threads_started":1,"wal-append.queue_time_us":91273}
I20260709 13:06:44.434266 25683 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.435913 25683 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.437422 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.425978 25817 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:44.424973 22009 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.445236 25817 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:44.445693 25817 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:44.446899 25817 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.448379 25817 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:44.455236 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.455827 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.457695 25575 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:44.458209 21824 leader_election.cc:400] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7.
I20260709 13:06:44.461337 25727 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.462936 25389 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.473560 25389 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:44.474731 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.474992 21926 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:06:44.475243 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.475760 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.476374 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1898) took 1084 ms. Trace:
I20260709 13:06:44.464136 25492 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.465117 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:44.483582 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2201) took 2486 ms (client timeout 1926 ms). Trace:
I20260709 13:06:44.484163 21860 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:44.485553 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.485031 25645 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.485958 21857 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:44.487211 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.488338 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.489321 21858 raft_consensus.cc:2368] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11.
I20260709 13:06:44.491328 25492 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:44.491824 25492 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.496052 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.476776 21926 rpcz_store.cc:276] 0709 13:06:43.392344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.510284 (+117940us) service_pool.cc:224] Handling call
0709 13:06:44.476085 (+965801us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54002e95d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:44.476306 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2209917312}
I20260709 13:06:44.497786 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.498922 21854 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:44.499970 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.501752 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:44.502473 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.503715 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 75613.9us
I20260709 13:06:44.505862 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:44.484092 21859 rpcz_store.cc:269] 0709 13:06:41.995341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.481629 (+2486288us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:44.481775 (+   146us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:06:44.511513 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.514467 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.515004 22001 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:44.515177 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.515676 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.515919 21858 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:44.517835 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.518528 21859 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.519790 25545 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.516891 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.522141 22009 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:44.521374 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:44.525931 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61026us
I20260709 13:06:44.510356 25645 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.534761 21857 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:44.535336 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.535755 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.536708 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.535513 25727 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:44.537657 25669 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:44.538101 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.537657 25618 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:06:44.539093 21860 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:44.540185 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.538151 25669 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:44.538139 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.541250 25669 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.536780 21857 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:44.541808 22001 raft_consensus.cc:2376] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9. Re-sending same reply.
I20260709 13:06:44.542611 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:44.543038 25810 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.329s	user 0.000s	sys 0.004s
I20260709 13:06:44.543949 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.545305 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.546657 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.547621 21854 raft_consensus.cc:2410] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:06:44.548446 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.550189 25545 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:44.550259 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 665) took 2406 ms. Trace:
I20260709 13:06:44.550866 21928 rpcz_store.cc:276] 0709 13:06:42.144260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.544732 (+1400472us) service_pool.cc:224] Handling call
0709 13:06:43.545066 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:43.545423 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.545454 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:43.545918 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:06:44.213972 (+668054us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:44.214068 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.214185 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:44.542260 (+328075us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:44.549856 (+  7596us) raft_consensus.cc:1685] finished
0709 13:06:44.549878 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.550084 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11794,"raft.queue_time_us":975,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":345,"spinlock_wait_cycles":15838464,"thread_start_us":2411,"threads_started":1,"wal-append.queue_time_us":667033}
I20260709 13:06:44.527279 21856 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:44.556613 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.557519 21856 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:44.557049 25761 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:44.558430 25781 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.407s	user 0.002s	sys 0.011s
I20260709 13:06:44.558609 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.565591 25389 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.565650 25654 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.561550 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2183) took 3273 ms. Trace:
W20260709 13:06:44.575703 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.562968 21851 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:44.563202 25492 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.537187 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:44.567397 22002 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:44.575860 21853 rpcz_store.cc:276] 0709 13:06:41.288308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.288874 (+   566us) service_pool.cc:224] Handling call
0709 13:06:41.289382 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.384153 (+ 94771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.561030 (+3176877us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b54003d3954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:44.561343 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32983936}
I20260709 13:06:44.568094 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1933) took 2623 ms. Trace:
I20260709 13:06:44.568866 22008 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:06:44.569624 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:44.570147 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.570438 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.567477 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.544835 21857 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:44.571386 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.541523 21858 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:44.574750 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:44.568074 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:44.587182 21935 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:44.584972 21855 rpcz_store.cc:276] 0709 13:06:41.944574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.567046 (+2622472us) service_pool.cc:224] Handling call
0709 13:06:44.567396 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.567643 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.567670 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.567707 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:44.567739 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.567806 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.567918 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:44.592952 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.593497 21859 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:06:44.594105 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59523.4us
I20260709 13:06:44.595844 21854 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:44.596328 22001 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.597993 22006 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:44.598299 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:44.597784 21926 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.599730 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.602303 21852 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:44.597890 21929 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:44.604959 25815 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:44.616950 21933 raft_consensus.cc:2368] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7.
I20260709 13:06:44.617231 25815 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.619625 25815 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.625075 21757 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 4 to 7, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 7 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:06:44.627435 21757 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:06:44.630018 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1887) took 2563 ms (client timeout 1940 ms). Trace:
W20260709 13:06:44.630458 21927 rpcz_store.cc:269] 0709 13:06:42.066720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.629645 (+2562925us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:44.629812 (+   167us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:06:44.631669 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.632428 21853 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 6: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:44.633227 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.634527 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.635041 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.635864 21851 raft_consensus.cc:2368] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11.
I20260709 13:06:44.636072 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.634764 25645 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:44.639212 25785 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:44.639820 25645 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:44.640343 25645 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:44.640767 25645 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.641248 25785 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:44.641721 25785 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:44.643313 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.643549 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.644089 22585 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:44.644407 22584 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:44.599583 25761 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:44.633497 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.645447 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.645761 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:44.645926 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75958.2us
I20260709 13:06:44.636771 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.646813 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.647207 21851 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.647249 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.608156 25758 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.606225 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:44.649734 22588 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:44.609928 25216 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.613128 25683 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.646718 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.651362 25683 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.647316 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.614298 22004 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:44.635667 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:44.605615 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.653903 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.654110 21853 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:44.603933 21928 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:44.648008 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.647994 25545 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:44.673859 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.675634 22586 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:44.591871 25817 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:44.675691 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.675825 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 768) took 3188 ms. Trace:
I20260709 13:06:44.675681 21930 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:44.680563 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.677130 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:44.679772 22004 rpcz_store.cc:276] 0709 13:06:41.487771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.613499 (+3125728us) service_pool.cc:224] Handling call
0709 13:06:44.613922 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.668771 (+ 54849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.675753 (+  6982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:44.683113 25761 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.685619 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.685942 25761 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.691836 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.693877 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:44.671294 25545 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:44.697072 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49226.2us
I20260709 13:06:44.696913 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.698479 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.700434 22002 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:44.700541 21852 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:44.702723 22586 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:44.714190 21859 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:44.714617 21859 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:44.715463 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 117347us
I20260709 13:06:44.716391 22593 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.711295 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.710807 21858 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:44.719403 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.699052 25545 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.696838 25727 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:44.698833 25216 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:44.724170 22588 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
W20260709 13:06:44.724660 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 187631us
I20260709 13:06:44.642776 25785 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.727386 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1892) took 1370 ms. Trace:
I20260709 13:06:44.716496 25729 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:44.719347 25575 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.742789 25654 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.722010 25817 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.744529 21855 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:44.745191 25732 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.745635 25732 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.746021 25732 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.750180 25683 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.751884 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.753214 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.754289 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.755795 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.757428 25830 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:44.757763 25761 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:44.757895 25830 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:44.758319 25830 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.759003 25727 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:44.759462 25727 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.719014 25767 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:44.773211 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:44.777719 21851 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
W20260709 13:06:44.786907 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45054.6us
I20260709 13:06:44.728982 21932 rpcz_store.cc:276] 0709 13:06:43.284663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.151609 (+866946us) service_pool.cc:224] Handling call
0709 13:06:44.152258 (+   649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.152796 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:44.153449 (+   653us) log.cc:844] Serialized 14 byte log entry
0709 13:06:44.309639 (+156190us) log.cc:844] Serialized 14 byte log entry
0709 13:06:44.327769 (+ 18130us) log.cc:844] Serialized 14 byte log entry
0709 13:06:44.335558 (+  7789us) log.cc:844] Serialized 14 byte log entry
0709 13:06:44.518056 (+182498us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18003dbd60. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:44.526214 (+  8158us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b18003dbd60. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:06:44.654900 (+128686us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.654972 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:44.655026 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.655105 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.655215 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1158,"prepare.queue_time_us":12884,"prepare.run_cpu_time_us":198,"prepare.run_wall_time_us":190,"raft.queue_time_us":33061,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":1856,"spinlock_wait_cycles":21156224,"thread_start_us":282134,"threads_started":2,"wal-append.queue_time_us":197401}
I20260709 13:06:44.734855 25729 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.789222 22004 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:44.793939 21926 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:44.790292 25618 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:44.798693 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:44.799295 25823 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:44.800137 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.799574 21757 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.800853 22006 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:44.799337 21854 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.796478 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:44.792897 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 43954.2us
I20260709 13:06:44.779186 25631 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:44.831089 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.795820 25729 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:44.779410 25582 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:06:44.837545 22593 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:44.807654 25618 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:44.823830 25631 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.779774 25745 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:06:44.835224 21859 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.857577 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.777618 25575 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.857969 21757 catalog_manager.cc:5697] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 1 to 5, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
W20260709 13:06:44.861466 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 100894us
I20260709 13:06:44.861799 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.827593 25683 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:44.869100 25683 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.869688 21926 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:44.870885 25683 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.871294 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:44.873034 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.863473 25631 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.842533 25343 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:44.872779 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.881366 25824 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:44.881886 25824 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:44.882270 25824 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:44.885125 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 88754.5us
W20260709 13:06:44.885113 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.861521 25618 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.887308 21899 leader_election.cc:400] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9.
I20260709 13:06:44.890137 21899 leader_election.cc:403] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:06:44.859558 25492 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:44.835381 25767 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:44.874325 25619 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:44.873596 25325 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:44.893020 21757 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 3 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.897289 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:44.900125 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:44.900765 25745 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9.
W20260709 13:06:44.908279 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.866850 25817 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:44.911966 25785 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:44.912531 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.911453 25619 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:44.914016 21823 connection.cc:256] RPC call timeout handler was delayed by -2.5260938249999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 13:06:44.914534 21823 connection.cc:256] RPC call timeout handler was delayed by -2.1078543099999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:06:44.914832 21823 connection.cc:256] RPC call timeout handler was delayed by -1.7379543099999637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 13:06:44.915131 21823 connection.cc:256] RPC call timeout handler was delayed by -1.7370543099999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20510000000000003s for a response
I20260709 13:06:44.915921 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:44.917624 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:44.918139 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.917512 21757 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 3 to 4, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.919557 25023 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:06:44.920083 25757 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:44.909166 25325 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:44.921571 25828 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:44.921758 25216 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:44.924778 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.928432 21854 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:44.926220 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.936267 21926 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.887694 25666 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.927335 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:44.890520 25582 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:44.954731 22004 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 7: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:44.954875 25023 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:44.903097 25727 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:44.965258 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.966238 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.966593 25727 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:44.967316 25727 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:44.967428 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.935640 25325 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.967623 25814 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:06:44.968317 25833 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:44.968509 25814 consensus_queue.cc:1243] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:44.968758 25833 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:44.969211 25833 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:44.970016 25394 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.970657 25394 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.941463 21757 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 3 to 6, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 6 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:44.872450 25669 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:44.937067 22007 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
W20260709 13:06:44.973430 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48480.5us
I20260709 13:06:44.974128 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.941987 22001 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:44.926090 25343 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:44.959903 25582 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:44.961608 25631 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:44.962390 21900 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:44.962692 25619 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:44.962972 25666 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:44.963680 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 137294us
I20260709 13:06:44.963867 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:44.964231 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48433.9us
I20260709 13:06:44.964478 25683 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.964622 25023 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:44.975301 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67768.9us
I20260709 13:06:44.978153 25817 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:06:44.973436 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 65548.2us
I20260709 13:06:44.980914 22005 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:44.889436 25523 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:06:44.996095 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:44.967716 25727 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:45.000903 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:45.005120 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.005151 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.006254 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.007098 22001 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:45.007973 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.012470 22587 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:45.014533 21859 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.015620 22588 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:06:45.015731 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.016836 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.018532 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.021641 22004 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.993559 25683 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:45.024048 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.024694 22593 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.016422 22002 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:45.008198 25727 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.026999 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:45.023892 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64715.8us
W20260709 13:06:45.025226 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 62924.9us
I20260709 13:06:45.027941 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.029574 22009 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:45.010028 25758 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:45.041271 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.043622 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.044524 21854 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:45.021166 25294 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:45.046548 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.048146 21860 raft_consensus.cc:2368] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8.
I20260709 13:06:44.995085 25405 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:06:44.961639 25767 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.027638 25683 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.014034 25645 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.029458 25492 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:45.074707 22001 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:45.024159 25394 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:45.056033 21854 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:45.079416 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.054062 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:45.048473 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 322177us
I20260709 13:06:45.050239 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:45.048816 25294 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:45.022352 25814 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:06:45.076360 22000 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:45.093241 25723 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:45.097064 25817 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:45.097738 21933 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:45.097805 25838 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:45.098837 25619 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:06:45.101822 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47857.5us
I20260709 13:06:45.077632 25492 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.127533 22002 raft_consensus.cc:2376] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12. Re-sending same reply.
I20260709 13:06:45.060468 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:45.078912 25689 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:45.138183 25689 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:45.084589 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.089654 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.140311 21854 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:45.071620 25760 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:45.139497 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.200138 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.200510 21860 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:45.178197 22589 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:45.064707 25758 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.182713 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:45.185627 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:45.186775 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.190464 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.192317 21859 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:06:45.192502 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.193442 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.124292 25752 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:45.134276 25492 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.187758 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:45.197626 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4284007250000741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
I20260709 13:06:45.105027 25839 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:45.128885 25761 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:45.139976 25723 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:45.052536 25343 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.210019 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:45.209400 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4185007250000581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 13:06:45.210624 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 199762us
I20260709 13:06:45.222535 21853 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:45.222553 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4131007250000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
I20260709 13:06:45.216847 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:45.223345 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4131007250000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
I20260709 13:06:45.223371 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:45.223861 21972 connection.cc:256] RPC call timeout handler was delayed by -1.3983726379999553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 13:06:45.210314 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 125458us
I20260709 13:06:45.224206 21856 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:45.061863 25582 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.240232 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:45.244594 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.139605 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.219727 21852 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:45.244499 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:45.178112 25405 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:06:45.251145 25832 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
W20260709 13:06:45.251825 25834 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:06:45.252394 25842 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:45.206321 25757 raft_consensus.cc:2764] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:06:45.261761 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.210071 25839 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:45.254086 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 118843us
I20260709 13:06:45.203470 21859 pending_rounds.cc:85] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 7
I20260709 13:06:45.257134 22593 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:45.205408 25846 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:45.280936 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46647.1us
I20260709 13:06:45.082501 25023 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:45.251134 25843 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:06:45.283102 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.287081 21857 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:45.268020 21859 pending_rounds.cc:107] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:06:45.287652 21859 raft_consensus.cc:2889] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:45.247107 25631 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:45.265410 21929 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 7: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:45.291041 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50940.5us
I20260709 13:06:45.313099 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:45.311048 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:45.315162 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 254714us
W20260709 13:06:45.315812 25840 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:06:45.319382 25833 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.320960 25325 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.322664 25850 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:06:45.323033 25811 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:06:45.323729 25830 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.005s
I20260709 13:06:45.324115 25799 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:45.324712 21972 connection.cc:256] RPC call timeout handler was delayed by -1.3357007250000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
I20260709 13:06:45.325966 25814 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:45.326573 25847 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.327360 21974 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:45.349658 21853 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:45.357689 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2880007250000745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 13:06:45.290258 25666 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:45.367106 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.329337 25582 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.355005 21852 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
W20260709 13:06:45.377122 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 199138us
W20260709 13:06:45.372952 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 84952.8us
I20260709 13:06:45.329187 25817 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:06:45.390133 25325 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:45.390210 25853 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.373600 25486 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.390803 25853 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:45.390975 25486 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:45.391489 25486 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.342917 25669 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:45.330265 25575 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:45.266341 25760 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:45.384485 25847 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.405877 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.254683 25294 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.388201 25855 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:45.389135 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.424285 22004 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:45.389217 25852 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:45.392475 21929 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:45.350060 25850 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:45.384830 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 57579.5us
I20260709 13:06:45.400856 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:45.404054 25732 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.658s	user 0.000s	sys 0.010s
I20260709 13:06:45.267762 25689 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:45.404865 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:45.435504 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2556007250000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 13:06:45.435873 21972 connection.cc:256] RPC call timeout handler was delayed by -1.225900724999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 13:06:45.436206 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2097007250000615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
W20260709 13:06:45.436535 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2079007250000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 13:06:45.436923 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1917007250000324s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
W20260709 13:06:45.437328 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1296007250000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2255s for a response
I20260709 13:06:45.386123 25839 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.366524 25814 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:45.391384 25853 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.328504 25824 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.427687 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:45.331082 25654 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:06:45.432572 21859 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:45.432467 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:45.441591 25815 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:06:45.442562 25618 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.000s
I20260709 13:06:45.418691 25405 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.445815 25760 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.446926 21852 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:45.448089 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:45.450371 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.459497 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.370144 25830 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.389137 25723 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.474184 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:45.453853 25732 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:45.474915 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 103944us
W20260709 13:06:45.485404 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53056.5us
I20260709 13:06:45.428771 25645 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:45.478015 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:45.487975 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 87308.7us
W20260709 13:06:45.489086 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 65928.5us
I20260709 13:06:45.280369 25846 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:45.496454 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:45.463971 25815 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:45.498109 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:45.500084 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.501868 21974 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:45.502019 21860 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:45.504889 22006 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.508517 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:45.508886 22593 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:45.486459 25839 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.489337 25689 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:45.513484 21928 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:45.511610 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:45.492875 25294 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:45.522469 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 269180us
I20260709 13:06:45.492463 25669 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:45.493113 25666 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:45.528491 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50584.4us
W20260709 13:06:45.542444 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.545702 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:45.531549 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.533169 25814 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.533514 25846 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.537982 22004 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:45.552722 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.553632 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:45.449326 25850 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.557267 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.554100 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:45.558463 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.559696 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.482172 25618 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.512847 25855 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.560253 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.561486 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.525645 25852 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.563138 21934 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:45.566834 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:45.567171 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.500922 25683 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.554388 25757 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:45.567898 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:45.562419 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:45.540251 25405 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.610915 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:45.614223 22002 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:45.640446 21927 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:45.650203 22009 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.641836 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.646473 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.646458 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:45.606215 25666 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:45.652040 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:45.652235 25492 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.653059 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.653709 21852 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:45.654695 25492 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:45.659765 25545 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.927s	user 0.007s	sys 0.002s
I20260709 13:06:45.660207 25545 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.663570 25486 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.611512 25654 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:45.641203 25757 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.610157 25631 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:45.607694 25645 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:06:45.678094 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 176282us
I20260709 13:06:45.555123 25823 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:06:45.549999 25817 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:06:45.567325 25846 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.609278 25761 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:45.650482 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.679045 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.676002 21928 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:45.747154 21852 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:45.679565 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 120472us
I20260709 13:06:45.741684 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.702677 22005 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:45.562806 25785 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.000s
I20260709 13:06:45.704396 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.750650 21851 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:45.673962 25863 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.754391 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:45.755004 25486 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:45.755726 25486 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:45.681949 25631 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.756796 25631 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.649746 25758 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:45.743582 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 242860us
I20260709 13:06:45.758201 22592 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:45.649159 25796 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:45.761854 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.762876 21860 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:45.678617 25757 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.699921 25815 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:45.774132 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.774977 22590 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:45.651110 25405 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.761868 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:45.769588 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:45.766939 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:45.774250 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 214323us
I20260709 13:06:45.757452 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.703481 25654 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:45.744899 25833 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:45.779060 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:45.787874 21935 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:45.788971 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 46625.3us
I20260709 13:06:45.756228 25486 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:45.794819 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.652303 25666 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.789697 21930 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:06:45.794116 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:45.787010 25729 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.915s	user 0.011s	sys 0.000s
I20260709 13:06:45.782027 25405 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.809962 22009 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:06:45.810938 25861 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:06:45.811148 25545 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:45.811868 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.812309 25486 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.814061 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.814647 21934 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.815267 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.816038 21926 raft_consensus.cc:2368] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:45.816788 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.812211 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.816823 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.821265 21900 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:45.821575 21856 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:45.825409 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.827477 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:45.827857 25857 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:45.828421 25857 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:45.824709 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:45.825670 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.836879 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.838021 21856 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:45.842718 25758 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:45.843531 25758 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:45.845227 25689 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:45.844730 25758 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.886464 25683 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:45.811895 25869 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:06:45.888378 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:45.889289 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:45.804179 25325 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:45.920702 21898 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.519s (SENT)
I20260709 13:06:45.927881 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.932020 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:45.927459 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 110183us
I20260709 13:06:45.802168 25847 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:45.929550 21860 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:45.932888 21851 raft_consensus.cc:2368] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8.
I20260709 13:06:45.811190 25761 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:45.809309 25830 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:45.937431 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.940191 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:45.891393 25823 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:45.960160 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:45.961071 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70803us
I20260709 13:06:45.960897 21851 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:45.819290 25729 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:45.942320 25732 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:45.980438 22008 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:45.969977 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:45.984804 21851 raft_consensus.cc:2376] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7. Re-sending same reply.
I20260709 13:06:45.971994 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.987202 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:45.988068 22009 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.998386 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:45.955837 25761 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:46.001421 22556 leader_election.cc:400] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7.
I20260709 13:06:46.004115 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:45.964529 25618 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.004207 22556 leader_election.cc:403] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:06:45.972837 25847 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.013908 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.015360 25886 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:06:46.016577 25886 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:46.016906 25886 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:46.017433 25871 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:06:46.020004 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66557.1us
W20260709 13:06:46.020236 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99608.9us
I20260709 13:06:46.031827 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:46.028265 21926 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 5.
I20260709 13:06:46.038600 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:46.047742 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46402.3us
I20260709 13:06:45.967605 25857 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.051445 25857 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.053428 25729 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.055568 25814 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.968060 25654 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.058143 25850 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:46.060743 25727 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.005s	user 0.012s	sys 0.001s
I20260709 13:06:46.062256 25727 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.065213 22004 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:46.021162 25865 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 13:06:46.062966 21935 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:46.028520 25582 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.002s
I20260709 13:06:46.068506 25575 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:46.018715 25862 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:46.026615 25732 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:46.079470 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80341.1us
I20260709 13:06:46.079777 22585 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:46.082816 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.087683 21858 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:46.094669 21926 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:46.090039 25405 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.067536 25885 raft_consensus.cc:2764] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:06:46.104442 25575 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.087198 25862 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.106331 25862 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.106155 25631 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.106771 25575 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.106860 22009 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:46.064894 25727 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.108508 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:46.110239 22589 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:46.110862 25631 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.111222 25618 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:46.113297 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.114003 25824 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.115109 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2264) took 1128 ms. Trace:
W20260709 13:06:46.117295 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.120250 22009 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:46.121281 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:46.121505 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:46.120645 25885 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:06:46.136757 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 127626us
I20260709 13:06:46.121376 25618 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:46.121827 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:46.141924 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93178.8us
I20260709 13:06:46.142112 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:46.139178 25618 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:46.143040 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.145133 25645 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:46.145608 25294 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:06:46.146325 25294 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:46.146049 25618 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.146847 25294 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:46.146767 25645 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:46.147471 25618 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.147913 25294 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.149405 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.150060 21852 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:46.020865 25761 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.152807 25325 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:46.157430 22586 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
W20260709 13:06:46.158766 25853 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.684s	user 0.008s	sys 0.000s
I20260709 13:06:46.159217 25853 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.160247 21853 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:46.160892 21930 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:46.164120 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:46.164780 22590 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
I20260709 13:06:46.035063 25823 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.165802 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:46.166976 25823 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.167202 22590 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:06:46.168874 25839 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.000s
I20260709 13:06:46.118644 25575 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.169620 25575 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.172163 25405 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:46.172917 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 59718.6us
W20260709 13:06:46.124440 25855 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:06:46.177862 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:46.183511 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.184845 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65453.3us
I20260709 13:06:46.185549 25683 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.186316 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.186769 25683 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.186043 25876 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:46.139652 22007 rpcz_store.cc:276] 0709 13:06:44.986674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.084671 (+ 97997us) service_pool.cc:224] Handling call
0709 13:06:45.128399 (+ 43728us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:45.357212 (+228813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:45.357258 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:45.386841 (+ 29583us) log.cc:844] Serialized 52 byte log entry
0709 13:06:45.463389 (+ 76548us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:45.498919 (+ 35530us) log.cc:844] Serialized 14 byte log entry
0709 13:06:45.547899 (+ 48980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.547966 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:46.114892 (+566926us) raft_consensus.cc:1685] finished
0709 13:06:46.114912 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:46.115050 (+   138us) inbound_call.cc:177] Queueing success response
0709 13:06:46.120681 (+  5631us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b180039e6e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
Metrics: {"mutex_wait_us":48678,"prepare.queue_time_us":39822,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":140530,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":17320,"spinlock_wait_cycles":10333056,"thread_start_us":210273,"threads_started":2,"wal-append.queue_time_us":110516}
I20260709 13:06:46.188737 25683 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.096213 25757 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:46.207777 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 62623.5us
I20260709 13:06:46.209270 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:46.122419 25817 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.213241 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:46.208613 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:46.137712 25729 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:46.215309 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.215291 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:46.218071 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.220619 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.212680 25817 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.205574 25855 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.223760 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:46.226243 21933 raft_consensus.cc:2368] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:06:46.223798 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:46.229475 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:46.232069 25757 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.222028 25343 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.237782 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.243089 22007 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:46.245358 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.251456 22000 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:46.223315 25575 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.252607 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.253752 22009 raft_consensus.cc:2410] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:46.255610 22007 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:46.250553 25343 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.257898 25729 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:46.258343 25853 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.259554 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 744) took 1603 ms. Trace:
I20260709 13:06:46.259918 22592 rpcz_store.cc:276] 0709 13:06:44.650799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.668874 (+ 18075us) service_pool.cc:224] Handling call
0709 13:06:46.253550 (+1584676us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400595454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:46.253666 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464830336}
I20260709 13:06:46.246994 25645 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:46.243191 25761 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.299532 22009 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:46.121325 25886 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.216298 25862 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.303063 25343 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.305368 25898 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:46.306095 25854 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:46.242733 25325 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 13:06:46.317575 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 98768.6us
I20260709 13:06:46.322893 21932 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:06:46.324615 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.325169 22009 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:46.301384 25729 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:46.327646 25729 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.329071 25862 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:46.330355 25863 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.569s	user 0.006s	sys 0.003s
I20260709 13:06:46.330822 25863 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.317500 25846 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:06:46.332053 25846 consensus_queue.cc:1243] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:46.317713 25645 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:46.333220 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71996.3us
I20260709 13:06:46.333389 25645 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.335076 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.335879 25814 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:46.336278 25814 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:46.336794 25814 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.333392 22591 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:46.338179 25732 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 6. Current term is 7.
I20260709 13:06:46.325749 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.341423 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.341589 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.342288 22007 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:46.343204 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.343771 22007 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:46.313148 25761 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:46.349238 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.351696 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.352535 21934 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:46.354931 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.355865 21934 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:46.356285 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.357388 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.320613 25854 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:46.361749 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.363082 21926 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:46.362401 21935 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:46.366212 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.366833 21929 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 5.
W20260709 13:06:46.367362 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.350663 25898 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:46.371377 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:46.368137 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.325717 25857 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:06:46.361723 25885 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:46.381515 25885 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.381956 25885 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:46.384281 22007 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:46.384322 25814 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.385363 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.385418 25767 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:46.386003 25767 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:46.386402 25767 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:46.387158 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.374876 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:46.387948 21931 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.388537 25814 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.389271 25876 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.348317 25862 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:06:46.401894 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49892us
I20260709 13:06:46.401116 25850 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:06:46.406028 25898 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.407953 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.408785 22007 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:46.409626 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.409312 25854 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.410384 22007 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:46.410491 25854 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.411209 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.412320 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.413781 25817 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.414736 25850 consensus_queue.cc:1243] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:46.414526 25885 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.415023 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:46.415288 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 89638.7us
I20260709 13:06:46.404783 25767 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.416157 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.416260 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.417165 25876 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:46.417528 25849 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:46.417881 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:46.418129 25876 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:46.418069 25575 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.418561 25823 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:46.418687 25876 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:46.419253 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.419950 25767 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.412822 22007 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.419912 25876 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.421347 25876 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.422567 25817 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.423048 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:46.423185 25817 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.423779 21852 raft_consensus.cc:2368] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:06:46.423585 21854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:06:46.424063 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:46.424295 21854 raft_consensus.cc:606] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.424793 21860 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 5: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:46.414526 25862 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:46.426944 25862 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:46.428138 25757 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:46.427855 25862 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:46.428654 25757 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:46.429311 25862 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:46.429802 25757 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.432248 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.433050 21926 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:46.396607 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:46.434101 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.436069 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:46.416751 25885 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.436810 21935 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:46.438941 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.447912 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 782) took 1719 ms. Trace:
I20260709 13:06:46.448421 22008 rpcz_store.cc:276] 0709 13:06:44.728844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.756799 (+ 27955us) service_pool.cc:224] Handling call
0709 13:06:44.761458 (+  4659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.981097 (+1219639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.447574 (+466477us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000db6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:46.447802 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799418752}
I20260709 13:06:46.450735 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.451474 22008 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:46.452490 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.456115 22008 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:46.452904 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.458938 21931 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:46.458581 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:46.461531 22004 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:46.463967 21926 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:46.464435 21931 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: [7.5-9.6]
W20260709 13:06:46.467331 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 51145.3us
I20260709 13:06:46.453183 25823 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:06:46.468964 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51142.7us
I20260709 13:06:46.471091 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:46.471941 22009 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:46.472666 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.458636 25767 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:46.476364 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.487859 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:46.489946 22007 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:46.492278 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.486996 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.497670 25631 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.485802 25885 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.493436 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.468936 25857 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:46.491988 25823 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:46.493983 22004 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:46.493436 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.502364 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.504868 22008 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:06:46.500821 25631 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.513160 22008 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:46.515321 25823 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.516186 25631 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.486625 25767 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.519878 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.521059 22584 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:46.494441 25723 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:46.492821 25859 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:46.506712 25618 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:46.507287 25904 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:46.522336 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.522087 22009 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:46.506821 25857 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.530020 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.531038 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:46.501565 25325 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:06:46.524308 25903 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.523138 25767 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:46.534303 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.535213 22008 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:46.527756 25859 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.539029 22008 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:46.521267 25848 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.540099 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.540886 22009 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:46.534554 25582 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:46.541533 25683 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:46.541836 25582 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:46.542311 25582 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:46.535734 25631 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:46.545218 25575 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.545394 25631 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:46.545894 25631 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:46.546866 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.547318 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.548404 25767 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.549100 25859 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:46.549842 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.551028 25767 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.527719 25814 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.553416 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.527208 25885 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:46.524932 25723 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.557166 21935 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:46.557485 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.557801 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.529347 25904 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.532524 25857 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:46.570739 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 315781us
I20260709 13:06:46.566092 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:46.570802 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.574908 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2336) took 1523 ms. Trace:
I20260709 13:06:46.576267 21927 rpcz_store.cc:276] 0709 13:06:45.045753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.045907 (+   154us) service_pool.cc:224] Handling call
0709 13:06:45.060449 (+ 14542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.654993 (+594544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.560627 (+905634us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54002e86d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:46.568633 (+  8006us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333435136}
I20260709 13:06:46.533831 25752 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:46.578955 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.581780 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.568004 25814 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:46.570043 25723 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.592332 21855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:06:46.574144 25847 raft_consensus.cc:2764] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:06:46.593040 21855 raft_consensus.cc:606] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.577847 25817 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.603149 25631 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.569391 25683 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:06:46.590833 25575 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:46.606318 25582 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.607602 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.578701 25849 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.577139 25904 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:46.609664 22556 leader_election.cc:336] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.774s (SENT)
I20260709 13:06:46.575110 25885 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:46.581240 25761 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.555932 25903 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.624393 21854 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:46.579731 25752 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:46.627897 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 784) took 1837 ms. Trace:
I20260709 13:06:46.641197 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.629122 25862 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
W20260709 13:06:46.645699 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.843s (SENT)
W20260709 13:06:46.656359 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.005s (SENT)
I20260709 13:06:46.657275 25903 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.651217 25855 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.658108 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.658502 25855 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:46.659231 25855 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:46.659616 25855 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:46.660020 25855 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.660172 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.660609 25853 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.649637 25885 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.651412 22006 rpcz_store.cc:276] 0709 13:06:44.790400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.801536 (+ 11136us) service_pool.cc:224] Handling call
0709 13:06:44.801849 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.510641 (+708792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.627631 (+1116990us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b540045b0d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:46.627830 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616271616}
I20260709 13:06:46.656023 25752 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:46.657094 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.645931 25023 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:46.648523 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 91024us
I20260709 13:06:46.663899 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.663655 25654 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.669397 25752 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.669932 25911 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.672821 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:46.673237 25898 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.677111 25839 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:46.678923 25645 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:46.679971 22556 leader_election.cc:400] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:06:46.638448 25575 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:46.686844 25859 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.662647 25910 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:46.692847 22003 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:46.735421 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44565.1us
I20260709 13:06:46.661473 25853 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:46.738296 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58456.6us
W20260709 13:06:46.736267 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45329.1us
I20260709 13:06:46.741868 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2337) took 1207 ms. Trace:
I20260709 13:06:46.742034 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.733985 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:46.743345 21927 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:46.743757 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:46.742273 22003 rpcz_store.cc:276] 0709 13:06:45.532029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.532137 (+   108us) service_pool.cc:224] Handling call
0709 13:06:46.629127 (+1096990us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:06:46.739103 (+109976us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300288}
I20260709 13:06:46.744972 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.748553 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:46.751227 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2340) took 1199 ms. Trace:
I20260709 13:06:46.733100 25761 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:46.751688 22005 rpcz_store.cc:276] 0709 13:06:45.551425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.612452 (+ 61027us) service_pool.cc:224] Handling call
0709 13:06:46.750981 (+1138529us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b540040d154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:46.751149 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89298688}
I20260709 13:06:46.756057 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:46.756978 21852 raft_consensus.cc:2410] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:46.756946 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.748484 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.758302 21854 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:46.759341 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.757642 21856 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:46.759634 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:46.734191 25911 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.672891 25857 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:46.750574 25823 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.758230 25814 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:46.865012 21854 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:06:46.861720 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 113290us
W20260709 13:06:46.897660 21858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1962) took 1790 ms (client timeout 1528 ms). Trace:
W20260709 13:06:46.931519 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.765947 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.901968 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:46.996677 22001 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:06:46.865044 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.772105 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2335) took 1750 ms. Trace:
I20260709 13:06:46.760493 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:46.996795 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:46.855504 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 111695us
I20260709 13:06:46.758832 25654 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:46.784706 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:46.853566 25817 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.946858 22007 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:06:46.770509 25877 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.828s	user 0.004s	sys 0.000s
I20260709 13:06:46.908119 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:46.752285 25849 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:46.976455 22003 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:46.977229 22005 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:06:46.997563 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.997974 21932 rpcz_store.cc:276] 0709 13:06:45.021308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.022156 (+   848us) service_pool.cc:224] Handling call
0709 13:06:45.042150 (+ 19994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.323342 (+1281192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.771799 (+448457us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002a8354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:46.772028 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944683264}
I20260709 13:06:46.999702 21934 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:46.923893 25895 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.680s	user 0.008s	sys 0.000s
I20260709 13:06:46.771991 25575 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.000910 22008 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:46.770992 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2338) took 1238 ms. Trace:
I20260709 13:06:46.756083 25761 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:46.902230 25890 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.668s	user 0.002s	sys 0.000s
I20260709 13:06:47.006075 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:47.006537 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.088745 22001 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:47.005147 21898 leader_election.cc:400] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8.
I20260709 13:06:47.005584 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.089913 22006 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:46.995990 21858 rpcz_store.cc:269] 0709 13:06:45.107324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.127349 (+ 20025us) service_pool.cc:224] Handling call
0709 13:06:46.897404 (+1770055us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:46.897560 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033203584}
I20260709 13:06:46.879513 25886 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:46.942808 25853 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:46.863449 25325 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.029107 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 127237us
W20260709 13:06:47.006754 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 247174us
I20260709 13:06:46.998226 21856 raft_consensus.cc:2376] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8. Re-sending same reply.
I20260709 13:06:47.085870 22000 rpcz_store.cc:276] 0709 13:06:45.532292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.532418 (+   126us) service_pool.cc:224] Handling call
0709 13:06:45.558433 (+ 26015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.252624 (+694191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.770761 (+518137us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54004380d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:46.770937 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593290112}
I20260709 13:06:46.912986 25913 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:46.765749 25582 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.003029 25654 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:46.902081 25838 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.195s
I20260709 13:06:46.892454 25904 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.086304 25849 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.910018 25847 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:46.773440 25910 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:47.091857 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 183842us
I20260709 13:06:47.089175 25761 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:46.914258 25859 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:46.752437 25863 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.289s
I20260709 13:06:46.997495 25868 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:47.099411 22002 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 13:06:47.100485 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 95622.5us
I20260709 13:06:47.107394 21934 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:47.004508 21742 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8f3b35a35257416db3eaaf364cf6b52b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:47.102221 25854 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:47.121476 25849 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:47.121567 25325 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.151010 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.000926 25915 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.137868 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:47.056569 25874 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:46.769905 25631 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.088116 25846 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.114432 21852 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:06:47.115782 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.118304 21856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:06:47.102176 25876 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:47.161881 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2223) took 1500 ms. Trace:
I20260709 13:06:47.162531 21859 rpcz_store.cc:276] 0709 13:06:45.655333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.657122 (+  1789us) service_pool.cc:224] Handling call
0709 13:06:45.657570 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:45.658105 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:45.658147 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:45.658732 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:06:45.700262 (+ 41530us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:45.700307 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.700373 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:46.769958 (+1069585us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:47.152532 (+382574us) raft_consensus.cc:1685] finished
0709 13:06:47.152553 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.155917 (+  3364us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139157,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":89118,"spinlock_wait_cycles":12522240,"thread_start_us":40214,"threads_started":1,"wal-append.queue_time_us":131237}
I20260709 13:06:47.057255 25814 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.006246 25817 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:47.112193 25848 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.000s
I20260709 13:06:47.056735 25823 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:47.153330 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.154222 21742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 149888us
I20260709 13:06:47.154886 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:47.111259 25654 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:47.173590 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2225) took 1517 ms. Trace:
I20260709 13:06:47.133513 25904 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.149673 25868 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:47.152778 25852 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.156762 25325 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.111455 25885 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:06:47.131287 25911 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.135040 25857 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.157011 25874 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:47.182075 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44296.5us
I20260709 13:06:47.175444 25863 consensus_queue.cc:1243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:47.149173 25847 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.126392 25838 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:06:47.189255 25898 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.195377 25852 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.180403 21853 rpcz_store.cc:276] 0709 13:06:45.656087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.660564 (+  4477us) service_pool.cc:224] Handling call
0709 13:06:45.661052 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.160619 (+499567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.173326 (+1012707us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400017f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:47.173526 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147601664}
I20260709 13:06:47.158309 25915 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.114078 25913 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.126245 25619 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:06:47.129633 25723 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.155723 25854 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.190047 25853 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:47.200636 25618 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:47.202598 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 753) took 1061 ms. Trace:
I20260709 13:06:47.204509 25847 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.181195 25523 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:06:47.191712 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 174967us
I20260709 13:06:47.181900 25848 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:47.178282 25876 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:47.191272 25868 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:47.173365 25846 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.196161 25631 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.193511 25823 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:47.207909 25885 consensus_queue.cc:1243] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:06:47.210124 25898 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.229471 25618 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:47.231340 25839 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:47.229545 22584 rpcz_store.cc:276] 0709 13:06:46.140750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.140922 (+   172us) service_pool.cc:224] Handling call
0709 13:06:46.156638 (+ 15716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.521548 (+364910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.202310 (+680762us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54004c22d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:47.202503 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822320896}
I20260709 13:06:47.236445 21853 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:47.224803 25910 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.238461 25631 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:47.258337 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:47.258966 21934 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:47.181067 25920 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:47.257056 25855 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.597s	user 0.013s	sys 0.000s
I20260709 13:06:47.227291 25853 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:47.260341 25853 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:47.266757 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.267545 21929 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:47.262495 22589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 940) took 1579 ms (client timeout 1749 ms). Trace:
W20260709 13:06:47.269203 22589 rpcz_store.cc:269] 0709 13:06:45.683462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.703171 (+ 19709us) service_pool.cc:224] Handling call
0709 13:06:47.262318 (+1559147us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400668fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:47.262442 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832264064}
I20260709 13:06:47.273283 21926 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:47.274873 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.276015 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.278702 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:47.279316 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:47.252071 25848 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.281639 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.280905 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:47.282321 21932 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.261956 25631 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.281759 25847 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:47.294152 25917 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:06:47.295735 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.297161 22589 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:06:47.313499 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.299464 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:47.318562 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:47.316200 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:47.322585 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:47.323452 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:47.326197 22587 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:47.294620 25854 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:47.328195 22591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 938) took 1688 ms (client timeout 1954 ms). Trace:
I20260709 13:06:47.298753 25824 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:47.300207 25325 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:06:47.296006 25918 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.330610 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:47.364406 22593 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:47.283310 25876 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.331593 25817 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.271795 25855 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.370039 25855 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:47.375679 25863 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:47.332895 25898 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.376372 25863 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.378970 25732 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:47.379524 25732 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.324470 25618 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.382058 25618 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.382210 25325 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:47.382805 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.383229 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.388939 25666 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:47.355688 25824 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:47.389734 25824 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:47.389590 25666 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.265909 25910 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.365962 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.399291 21859 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:47.395242 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:47.389220 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52226.2us
I20260709 13:06:47.393774 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:47.336257 22591 rpcz_store.cc:269] 0709 13:06:45.639621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.639783 (+   162us) service_pool.cc:224] Handling call
0709 13:06:47.328015 (+1688232us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54005325d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:47.328135 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568193280}
I20260709 13:06:47.395016 21932 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:47.379328 25863 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.401454 22008 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:47.400617 22000 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:47.402398 25631 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.391788 25666 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.405368 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.317075 25848 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.405512 21929 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:47.408445 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:47.410250 21852 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:47.413712 21860 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:47.398787 25523 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:47.415661 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 264700us
I20260709 13:06:47.417424 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:47.399653 25729 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:47.418938 25523 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.316913 25823 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.428412 25903 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:47.446791 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.447544 22008 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:06:47.446883 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 83043.5us
I20260709 13:06:47.434721 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:47.418443 25922 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:06:47.427838 21860 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:47.445528 22003 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:47.436015 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:47.451735 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.452399 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:47.449966 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 89966.6us
I20260709 13:06:47.456204 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:47.457571 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:47.459053 25732 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.460906 25523 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:47.461087 25683 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.461637 25619 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.461715 25683 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.450533 25903 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:47.463667 25903 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.462224 25619 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.407840 25919 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:06:47.453320 22000 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:47.449949 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:47.465579 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 691) took 1642 ms. Trace:
W20260709 13:06:47.478618 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1918) took 1721 ms (client timeout 2173 ms). Trace:
W20260709 13:06:47.479043 21933 rpcz_store.cc:269] 0709 13:06:45.756897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.979646 (+222749us) service_pool.cc:224] Handling call
0709 13:06:47.478453 (+1498807us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540003c354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:47.478565 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507823488}
I20260709 13:06:47.464735 25903 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:47.478971 21930 rpcz_store.cc:276] 0709 13:06:45.822846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.012177 (+189331us) service_pool.cc:224] Handling call
0709 13:06:47.465394 (+1453217us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:47.465497 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263861504}
I20260709 13:06:47.490693 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.491024 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.491500 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.422273 25913 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.445044 25723 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.513598 21926 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:47.492118 21930 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:47.514588 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:47.491741 21934 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:47.518546 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 101210us
I20260709 13:06:47.519690 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:47.517159 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.494705 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.494016 21851 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:47.470582 25846 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.522274 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.523380 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:47.464890 25852 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.530005 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:47.531011 25899 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.211s	user 0.002s	sys 0.009s
I20260709 13:06:47.527449 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.537375 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.523226 25761 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.651934 25852 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:06:47.656203 25761 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:47.491103 25920 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:47.541978 25910 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.657135 25910 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:47.652076 25918 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:47.657514 25920 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.657636 25910 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.546486 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.659029 22004 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:47.549815 25846 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:47.449786 25729 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.558104 25847 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:47.524222 25868 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.488852 25619 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.517715 25817 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.699632 25817 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.702293 25868 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.702293 25817 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.703209 25815 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:47.670027 25654 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:47.706007 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.450927 25859 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 13:06:47.641266 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 94581.3us
I20260709 13:06:47.707302 25654 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:47.708348 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:47.709358 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.710150 21854 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:47.664397 25853 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.711889 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.712163 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2241) took 1615 ms. Trace:
I20260709 13:06:47.465200 25666 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:47.522429 25886 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:06:47.527810 25885 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:47.719216 25885 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.719673 25885 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.720116 25874 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.527964 25930 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:06:47.645346 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 125754us
I20260709 13:06:47.645613 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.647317 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.724498 22003 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:47.724521 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:47.513302 25575 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.644002 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.648527 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.651481 21931 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:47.651829 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.654280 21857 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:47.654384 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.660312 25910 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:47.656736 25852 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:47.641680 25023 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:06:47.663465 25849 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.010s
I20260709 13:06:47.662359 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:47.661705 25933 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.691267 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:47.662897 25846 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.670212 25757 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.742733 25874 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.699018 25683 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.707269 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:47.800132 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 92917.2us
I20260709 13:06:47.801097 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:47.801268 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 138966us
I20260709 13:06:47.708520 25848 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:47.641577 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 97312.8us
I20260709 13:06:47.641297 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.714252 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:47.712587 21851 rpcz_store.cc:276] 0709 13:06:46.096515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.096672 (+   157us) service_pool.cc:224] Handling call
0709 13:06:46.097108 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.521679 (+1424571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.711931 (+190252us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540028bc54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:47.712107 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3182282496}
I20260709 13:06:47.717878 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.718150 25853 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.718250 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:47.718693 21859 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:47.725984 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:47.741804 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:47.742455 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 94008.1us
W20260709 13:06:47.812048 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86144us
I20260709 13:06:47.753911 25619 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.744405 25852 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:47.757730 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:47.817521 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 176323us
I20260709 13:06:47.817940 22586 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:47.758448 21856 raft_consensus.cc:606] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.738238 25930 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:47.744812 25817 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:47.723871 25666 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:47.771636 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.823341 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:47.819569 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:47.779974 21931 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:06:47.745349 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 217417us
I20260709 13:06:47.785676 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.777505 25023 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:47.803799 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:47.814343 21855 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:47.825423 21857 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:47.809211 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:47.814723 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:47.824798 21972 connection.cc:256] RPC call timeout handler was delayed by -2.0848280999998678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25320000000000004s for a response
W20260709 13:06:47.827381 21972 connection.cc:256] RPC call timeout handler was delayed by -1.6664005139999745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2645s for a response
I20260709 13:06:47.826241 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:47.818725 21856 raft_consensus.cc:614] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:47.802772 25929 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.809330 25903 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.798704 25846 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.829591 21851 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:47.777204 25575 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.838488 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:47.839833 21972 connection.cc:256] RPC call timeout handler was delayed by -1.0119383400000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 13:06:47.841768 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.813580 25683 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.845155 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.846112 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:47.852757 25936 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.794123 25933 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.854050 25817 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.853819 25928 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:47.854449 21857 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:47.826706 25857 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:47.855352 25930 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.856551 25915 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.860553 25666 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.863176 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:47.992126 21932 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:47.992789 21856 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.826902 25850 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.838037 25911 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:47.866240 25824 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.823144 25815 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.867417 25023 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.828235 25582 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.833518 25839 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:47.822963 25752 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.817184 25325 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:47.854418 25931 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:47.836802 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:47.853164 25935 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:06:47.827492 25941 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.108s	user 0.004s	sys 0.000s
I20260709 13:06:47.888140 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.873842 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:47.946735 22008 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:47.937558 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.947613 22002 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:47.948722 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:48.024784 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 198562us
W20260709 13:06:48.027702 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 191004us
I20260709 13:06:47.874552 22005 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:48.029616 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:48.029846 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.030377 22005 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:47.956421 21931 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:48.031800 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.032893 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:48.028867 22591 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.957595 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:48.034354 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 225204us
I20260709 13:06:48.035130 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.033612 21931 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:48.036696 21931 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:47.830901 25814 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:47.807578 25849 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.986645 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:47.988739 21934 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:47.827603 25937 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:47.853842 25523 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:48.058691 25868 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:48.060099 25868 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:47.839192 25903 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.061189 25903 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.062386 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:47.995978 21933 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:06:48.064695 25618 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:47.946049 25683 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.065253 25654 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.066026 25935 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:48.066586 25935 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.066154 25654 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.067515 25935 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.068671 25935 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.000236 25933 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:47.988000 25823 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.069473 25823 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.071154 25886 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:48.071733 25823 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.007578 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:48.073444 25876 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:48.074133 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:47.967314 25936 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.076444 25575 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.080154 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.080881 25868 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:48.081372 25868 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.082221 25868 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:47.827418 25910 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.142s
I20260709 13:06:47.828658 25848 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.166s
I20260709 13:06:48.085336 25910 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:47.839459 25853 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 13:06:48.085654 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.086396 22000 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:48.086918 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.087333 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.087625 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:48.088078 22000 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:48.088356 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.089327 25853 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.089816 25853 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.090943 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.091554 22001 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:48.092656 25853 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.027721 21851 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:48.038232 25839 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:47.838001 25631 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.038573 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.012347 25911 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.094883 25911 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:48.094439 25839 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.037689 21931 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:48.095888 25729 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.041234 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.029418 25023 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.085686 25910 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:48.097496 22005 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:48.097925 25910 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.099252 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.099308 25910 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.100126 22003 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:48.041854 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.040288 21934 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:48.101433 22005 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:48.044342 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.047997 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:48.052328 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.102612 22000 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:48.103130 21929 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:48.053956 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67393.5us
I20260709 13:06:48.027281 25752 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.105406 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.105559 25814 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:47.995306 25857 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:47.990384 25817 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.057190 25920 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:48.018579 25815 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.057978 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:48.018728 25582 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.058166 25859 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:48.058805 25523 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.993830 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.016400 25934 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:48.065579 25683 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.998715 21927 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:48.082644 25885 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.015408 25850 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:48.029655 25852 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.083945 25936 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.086256 25824 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.025480 21928 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:48.093912 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.101081 21926 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:48.164644 21860 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:48.106456 22009 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:48.107100 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.166124 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.111147 21856 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:48.114143 22002 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:48.114642 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79575.7us
I20260709 13:06:48.114037 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.167270 22005 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:48.117607 22007 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:48.094521 22008 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:48.119843 21857 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:47.863027 25619 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.169951 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.161214 22593 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:06:48.173555 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:48.123905 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:48.108392 21825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.532s (SENT) [suppressed 40 similar messages]
I20260709 13:06:48.149794 21932 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:48.149981 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 92056.5us
I20260709 13:06:48.043143 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:48.155342 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.044575 25929 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:48.038213 25915 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:48.102576 21855 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:48.167620 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.074178 25876 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:48.205572 21856 raft_consensus.cc:1217] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Deduplicated request from leader. Original: 9.8->[10.9-12.10]   Dedup: 10.9->[12.10-12.10]
I20260709 13:06:48.214438 21856 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:06:48.206347 21851 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:48.183691 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:48.213714 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43867.4us
I20260709 13:06:48.217136 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:48.219548 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 46070.8us
I20260709 13:06:48.222556 21855 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:06:48.223520 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:48.223924 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.224149 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:48.228580 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 104761us
I20260709 13:06:48.225947 21856 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:48.157557 25523 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:48.226913 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 183810us
I20260709 13:06:48.235651 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.227525 21858 raft_consensus.cc:2368] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:06:48.165447 25920 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.229279 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:48.244297 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.244536 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:48.031141 25931 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.230378 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:48.248142 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.230978 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.127473 25575 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:06:48.163614 25939 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:48.065281 25618 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.237705 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:48.221094 25936 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:48.238161 21851 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:48.216121 25929 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.223393 25933 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:06:48.223547 25950 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:48.239038 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.239600 25847 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.231545 25631 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:48.240532 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.240429 25938 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:48.228168 21855 pending_rounds.cc:85] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 4
I20260709 13:06:48.158936 25859 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:48.224920 25868 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:48.254742 21935 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:48.254879 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.225379 25852 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:48.173328 25934 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.300823 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.301736 22586 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
W20260709 13:06:48.267889 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162562us
I20260709 13:06:48.302912 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.302994 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.231207 25824 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:48.230757 25654 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.158340 25937 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:06:48.273305 22588 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:06:48.276435 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.241679 25928 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.280709 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:48.244832 25930 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:48.247781 25848 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:48.289213 21855 pending_rounds.cc:107] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:06:48.219295 25952 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:06:48.289968 22585 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:48.289220 25932 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.263293 21859 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:48.303721 22586 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:48.308523 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 91747.4us
I20260709 13:06:48.308480 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:48.265095 25823 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:06:48.314610 21855 raft_consensus.cc:2889] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:48.317283 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.290486 25847 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:48.329851 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.307307 25752 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.310844 25939 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.284979 25936 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.310477 25930 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.348978 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.363718 22590 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:48.364491 22585 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:48.364812 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.365469 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.365895 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.366667 22586 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:48.368932 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.370144 22008 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:48.373730 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.374534 22008 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:06:48.276306 25814 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.332409 25847 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:48.346525 25654 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:48.345057 25839 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.378947 22002 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:48.381304 25767 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:48.389901 25824 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:48.391673 25949 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:48.391433 25767 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.391836 25857 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.369086 25930 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.392243 25824 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.360244 25939 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:48.397878 25839 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.378295 25761 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:48.399350 25618 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.395244 25729 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:06:48.400362 25885 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.401316 25815 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:48.389215 25874 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.586s	user 0.006s	sys 0.003s
W20260709 13:06:48.399088 25951 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:06:48.327075 25876 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:48.410081 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80347.9us
I20260709 13:06:48.305608 25929 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.381651 25850 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.419396 25931 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.321499 25928 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.389698 25847 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.389066 25849 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.454528 22002 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:48.456120 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:48.427942 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 381309us
I20260709 13:06:48.332585 25932 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.398542 25767 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.435652 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:48.478530 22589 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:48.440939 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 332537us
I20260709 13:06:48.441694 21972 leader_election.cc:400] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:48.388504 25959 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:06:48.353740 25859 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.462118 25958 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:06:48.462760 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:48.463558 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.484292 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.464439 25846 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.004s
I20260709 13:06:48.474133 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.474419 21853 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:48.472810 21860 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:48.480396 21972 leader_election.cc:403] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:06:48.482337 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.469103 25618 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.460003 25817 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:48.484122 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 89919.9us
W20260709 13:06:48.499459 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61039.1us
I20260709 13:06:48.502027 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:48.502684 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.506639 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.507305 22586 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:06:48.508271 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.508716 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.509570 22586 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:48.504166 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.513797 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.514691 22591 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:48.515807 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.516659 25973 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:48.517427 25931 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:48.519871 25962 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.127s	user 0.004s	sys 0.000s
I20260709 13:06:48.516919 25654 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:48.524894 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:48.487739 25294 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.500387 25618 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.530566 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.504881 25761 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:48.507544 25937 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:48.535017 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.510748 25938 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:48.524868 25848 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:48.501160 25846 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.540654 25934 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.555948 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:48.556842 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43830.6us
I20260709 13:06:48.558322 21898 leader_election.cc:400] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:06:48.558617 21898 leader_election.cc:403] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:48.524438 25757 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:06:48.544787 25729 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:48.543951 25938 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:48.546445 25859 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.565828 21972 leader_election.cc:400] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:06:48.567162 21972 leader_election.cc:403] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:06:48.540160 25294 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.550410 25936 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.551091 25937 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:48.585119 25963 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:06:48.585958 25963 consensus_queue.cc:1243] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:06:48.587167 25957 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:06:48.589084 25928 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.591500 25523 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.592935 22585 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:48.563788 25938 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:48.602227 25523 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.603714 25847 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.604249 25937 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.607582 25937 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.555392 25767 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:06:48.571429 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.604667 25839 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:48.605715 25023 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.003s
I20260709 13:06:48.605870 25930 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.544680 25966 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.120s
W20260709 13:06:48.618240 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 246801us
I20260709 13:06:48.630587 22591 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:48.631106 22584 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:48.615092 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.658777 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.610499 25847 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.569892 25761 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:48.668265 25761 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:48.669142 25683 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.669433 25761 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:48.562678 25729 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:48.570856 25823 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.671480 25729 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.631796 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.671670 25683 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.613504 25654 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.635052 25967 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:06:48.673823 25767 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:48.674350 25767 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:48.675352 25767 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.636720 25846 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.639189 21758 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:48.641467 25932 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:48.643786 25911 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.000s
W20260709 13:06:48.646968 25903 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.000s
I20260709 13:06:48.648005 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.680095 22588 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:06:48.650871 25954 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.494s	user 0.002s	sys 0.000s
I20260709 13:06:48.581339 25961 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:48.656872 22593 raft_consensus.cc:2410] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:48.655893 25618 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.656651 25523 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:06:48.652796 25325 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.002s
I20260709 13:06:48.666805 25939 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.692533 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:48.679472 25911 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.690340 25523 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.695672 25523 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.696568 25911 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.690320 25932 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.697533 25932 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.698062 25294 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:48.700057 25932 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.688129 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.701378 25631 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:48.701574 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:48.701956 25631 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:48.702435 25898 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:48.702629 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70981.1us
I20260709 13:06:48.703934 25863 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:48.687620 25654 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:48.705932 25920 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:48.706391 25920 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:48.688113 25903 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.707118 25618 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:48.707633 25618 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.708068 25618 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.708406 25903 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.708601 25023 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:48.709417 25325 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.710063 21854 raft_consensus.cc:2368] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
W20260709 13:06:48.684470 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69597.1us
I20260709 13:06:48.711036 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:48.713660 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:48.714552 22585 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:48.676898 25767 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:48.707384 25920 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.708490 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:48.740692 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.741838 25839 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:48.740664 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 459996us
I20260709 13:06:48.742806 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:48.745196 25868 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:48.745690 25868 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:48.746218 25868 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.747166 21932 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:48.748325 25938 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.749410 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2368) took 2171 ms. Trace:
I20260709 13:06:48.749778 21935 rpcz_store.cc:276] 0709 13:06:46.578486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.578684 (+   198us) service_pool.cc:224] Handling call
0709 13:06:48.749251 (+2170567us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54002e8bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:48.749350 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851627904}
I20260709 13:06:48.750926 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.751477 25938 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.751832 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.752089 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.752444 22003 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:48.752746 22002 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:48.753324 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.753855 22003 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.719441 25654 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.722144 25523 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.761394 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.761965 21935 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:48.762673 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.763854 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.764374 22000 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.765988 21926 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:06:48.767709 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.768324 22002 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.718176 25023 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:48.729555 25958 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:48.769389 25023 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.769731 25958 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:48.704885 25863 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:48.770604 25863 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.770419 25023 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.771236 25933 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:48.771893 25023 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:48.771643 25863 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.773066 25863 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:48.774065 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.770440 25958 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:48.777415 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:48.732517 25956 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.571s	user 0.001s	sys 0.000s
I20260709 13:06:48.703104 25631 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:48.736863 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.738301 25915 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:48.739601 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1968) took 1032 ms. Trace:
I20260709 13:06:48.803098 21853 rpcz_store.cc:276] 0709 13:06:47.706976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.708060 (+  1084us) service_pool.cc:224] Handling call
0709 13:06:48.739443 (+1031383us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b540008a2d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:48.739543 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667423104}
I20260709 13:06:48.740160 21856 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:48.744949 25839 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:48.810231 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.777557 25958 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:48.785565 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.720690 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:48.729948 22585 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
W20260709 13:06:48.825419 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.811188 21932 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:06:48.826100 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.826241 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:48.827265 22002 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:06:48.829771 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109155us
I20260709 13:06:48.835201 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.837293 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:48.838125 21859 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:48.840359 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:48.841060 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.844159 21926 raft_consensus.cc:2368] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:48.846362 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:48.771721 25933 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:48.853983 21858 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:48.855046 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:48.855319 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:48.851248 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:48.855675 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2294) took 1014 ms. Trace:
I20260709 13:06:48.852200 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.852393 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.811815 25752 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:48.812014 25631 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:48.856307 22006 rpcz_store.cc:276] 0709 13:06:47.841681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.853982 (+ 12301us) service_pool.cc:224] Handling call
0709 13:06:48.096963 (+242981us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b5800003f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:06:48.097145 (+   182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.097562 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.097591 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.098030 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.161713 (+ 63683us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:48.161765 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.161832 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.731866 (+570034us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:48.855406 (+123540us) raft_consensus.cc:1685] finished
0709 13:06:48.855440 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.855601 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":994,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":69,"spinlock_wait_cycles":53753088,"thread_start_us":63179,"threads_started":1,"wal-append.queue_time_us":63486}
W20260709 13:06:48.842831 25982 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.097s	user 0.000s	sys 0.004s
I20260709 13:06:48.848390 25911 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:48.873595 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:48.890407 22002 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.809829 25857 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.893352 25857 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.895493 25857 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:48.875154 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:48.877250 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.878736 21927 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:48.861124 21926 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:48.879391 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.881520 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.883260 21860 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:48.883176 21930 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:48.903210 22002 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:48.849278 25654 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:48.859143 21932 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:06:48.887624 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 362768us
W20260709 13:06:48.842973 25977 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.084s	user 0.000s	sys 0.010s
I20260709 13:06:48.885824 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:48.824235 25967 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:06:48.872786 25931 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.950101 22003 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
W20260709 13:06:48.953295 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67539us
I20260709 13:06:48.860987 25874 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:48.876406 25752 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.860400 25732 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:06:48.915825 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60598.2us
I20260709 13:06:48.964136 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:48.916555 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 442480us
I20260709 13:06:48.906617 25343 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.908049 25575 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.909152 25911 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:48.921137 25979 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:06:48.921583 25930 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:48.921864 25683 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:48.802203 25959 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:06:48.927032 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:48.957154 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:48.980388 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.999224 25980 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.999729 25969 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:49.000643 25984 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:06:48.961685 25752 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.996125 25853 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:49.005170 25911 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:49.007408 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50341.1us
I20260709 13:06:49.038607 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:49.006539 25729 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.040745 22003 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:49.041775 25683 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:49.041867 25729 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:49.042443 25683 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:49.043385 25683 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:48.997640 25876 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:48.961994 25874 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.012763 25343 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.058720 25886 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.072582 25886 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:49.073268 25989 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:49.073559 25343 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.074725 25886 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.014879 25838 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:49.013818 25575 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.075373 25838 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.075807 25838 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:49.018460 25969 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:49.019022 25980 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.033185 25325 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:49.021785 25929 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.965523 25932 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:06:49.023283 25983 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.023638 25928 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.024356 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.027096 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.027316 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:49.027148 21931 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:49.027906 22591 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:49.030066 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.033285 22002 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:48.958863 25931 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:48.925954 25933 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.060307 22000 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:48.925800 25915 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.069870 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.070250 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 610) took 1259 ms. Trace:
I20260709 13:06:49.154049 21857 rpcz_store.cc:276] 0709 13:06:47.810308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.825991 (+ 15683us) service_pool.cc:224] Handling call
0709 13:06:49.070044 (+1244053us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b540018f1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:49.070180 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60582144}
I20260709 13:06:49.154584 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:48.967484 25618 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:49.015262 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 51225.1us
I20260709 13:06:49.058502 25729 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.084195 22005 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:49.079504 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:49.159320 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:49.160027 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:49.083881 22009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2297) took 1239 ms (client timeout 1579 ms). Trace:
W20260709 13:06:49.161424 22009 rpcz_store.cc:269] 0709 13:06:47.844275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.844506 (+   231us) service_pool.cc:224] Handling call
0709 13:06:49.083701 (+1239195us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002710d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:49.083823 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367665408}
I20260709 13:06:49.108778 21935 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:49.149078 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.207965 21855 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:49.151042 21928 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.070786 21852 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:49.155084 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:49.155108 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:49.209080 21857 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.145969 21933 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
W20260709 13:06:49.072619 25981 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:06:49.160717 22008 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:49.218897 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:49.219367 21929 raft_consensus.cc:2368] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:49.219301 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.165988 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.220162 22000 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:49.220609 21859 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:49.079304 25980 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:49.223739 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.164278 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:49.176276 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.177778 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.181417 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.225597 22002 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:49.080710 25874 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.076957 25575 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:49.157294 25859 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.000s
I20260709 13:06:49.157614 25983 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:49.230713 25985 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:06:49.230880 21855 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:49.192480 25654 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:49.232213 25983 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.232589 25654 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.165875 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:49.234054 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 68282.4us
I20260709 13:06:49.234724 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:49.236361 25874 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:49.236783 25874 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.237134 25874 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:49.237620 25859 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:49.238101 25859 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:49.238309 25986 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 13:06:49.238504 25859 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:49.210071 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55031.3us
I20260709 13:06:49.080828 25325 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.239977 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:49.240589 21933 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.241163 25575 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:49.241662 25325 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.241787 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:49.210599 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.242501 21934 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:49.242630 25575 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:49.242949 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:49.243393 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:49.243916 21934 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:49.244372 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.245249 25294 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:49.245426 25953 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:49.245816 25952 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.245954 25294 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:49.246822 25988 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:06:49.247134 25294 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.060871 25937 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.146538 25343 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:49.249253 25343 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.250140 25937 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.077705 25969 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.219007 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:48.915124 25958 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.207864 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.254612 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.225919 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.226733 21860 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:49.160609 25936 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:49.227643 22591 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:49.252676 25343 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.209743 25903 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:49.152060 25857 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.147328 25868 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.256474 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2263) took 1213 ms. Trace:
I20260709 13:06:49.081143 25582 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:49.257050 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 199886us
I20260709 13:06:49.331425 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.257093 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:49.332819 21855 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:49.329288 21851 rpcz_store.cc:276] 0709 13:06:48.042615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.276862 (+234247us) service_pool.cc:224] Handling call
0709 13:06:48.277216 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.739655 (+462439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.739684 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.740190 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.973197 (+233007us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:48.973238 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.973326 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.256288 (+282962us) raft_consensus.cc:1685] finished
0709 13:06:49.256308 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.256421 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":537822,"raft.queue_time_us":501857,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":22209,"thread_start_us":229563,"threads_started":1,"wal-append.queue_time_us":344016}
I20260709 13:06:49.333292 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:49.259315 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.334618 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:49.315994 25343 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:49.338577 25950 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:49.339176 25950 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:49.338649 25343 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.341403 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 757) took 1020 ms. Trace:
I20260709 13:06:49.341799 22589 rpcz_store.cc:276] 0709 13:06:48.320710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.322158 (+  1448us) service_pool.cc:224] Handling call
0709 13:06:48.478345 (+156187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.481600 (+  3255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.341125 (+859525us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400609ad0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:49.341329 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291945216}
I20260709 13:06:49.342306 25343 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.343443 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:49.346113 25618 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.347698 25993 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:49.347968 25920 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:49.348278 25993 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.349133 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:49.351876 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.352792 21935 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:49.355125 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:49.351876 22584 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:49.355974 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.357136 21855 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:49.358283 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.169600 25729 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.263806 21857 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:49.224644 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:49.362004 22008 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:49.261983 21933 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:49.158470 25928 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.312283 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.207059 25876 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 13:06:49.312639 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:49.362761 21930 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:49.362936 22587 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.363031 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:49.271502 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.302352 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.310381 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:49.266635 21934 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:49.300740 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:49.350644 25993 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:49.311233 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:49.361279 21855 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:49.400969 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.270167 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:49.395996 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.395778 21933 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:49.395913 21859 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:49.403498 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.364374 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.397181 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:49.399710 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:49.402280 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:49.397492 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66165.8us
I20260709 13:06:49.405109 21855 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:49.405414 21932 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:49.405558 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.264748 25983 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.336762 25936 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:49.426824 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.404950 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:49.428937 22589 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:49.435597 22008 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:49.257704 25933 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:49.440932 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:49.445309 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.445609 22590 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:49.446559 22002 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:49.442698 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.352164 25886 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:49.449362 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.245249 25978 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.447943 21930 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:06:49.448222 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 137982us
I20260709 13:06:49.450160 22590 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:06:49.447461 21933 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:49.259294 25325 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.476325 25325 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.478322 25931 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.478677 25325 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:49.479463 25582 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:49.480370 25582 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:49.480708 25582 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:49.481532 25582 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.482126 25868 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:49.482757 25582 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:49.482636 25838 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.259147 25839 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:49.488469 25952 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:49.465361 25761 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:49.490211 25823 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.001s
I20260709 13:06:49.490622 25823 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:06:49.490918 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.491544 25929 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.492089 25929 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.314652 25732 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:06:49.499044 22592 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:49.437825 25935 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:06:49.451257 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.501626 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:49.502408 21851 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:49.502499 21934 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:49.502262 22586 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:49.507529 21928 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:49.465628 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:49.376849 25903 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.344807 25752 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.466478 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:49.363189 25857 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:49.472759 25886 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.473507 25932 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:49.475832 21933 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:49.520407 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51533us
I20260709 13:06:49.302352 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:49.465112 25023 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:49.315831 25930 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.129s
I20260709 13:06:49.508445 22584 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.522486 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.522486 22004 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:49.527935 21930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:06:49.438941 25343 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:06:49.511178 25523 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.513592 25903 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:49.558455 21857 raft_consensus.cc:2368] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
I20260709 13:06:49.515883 25980 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.445091 25983 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:49.558810 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 205563us
I20260709 13:06:49.559971 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.571303 21859 pending_rounds.cc:85] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 5
I20260709 13:06:49.571905 21859 pending_rounds.cc:107] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:49.572391 21859 raft_consensus.cc:2889] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:49.528685 21930 raft_consensus.cc:606] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:49.441874 25993 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:06:49.572618 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.576047 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.528244 25932 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:49.587642 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 1972) took 1874 ms (client timeout 1776 ms). Trace:
W20260709 13:06:49.588927 21854 rpcz_store.cc:269] 0709 13:06:47.713280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.714087 (+   807us) service_pool.cc:224] Handling call
0709 13:06:49.585843 (+1871756us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b540010c554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:49.587565 (+  1722us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2069895424}
I20260709 13:06:49.509732 25978 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.564375 25023 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:49.565012 25631 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:49.649487 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73509.1us
W20260709 13:06:49.642714 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 237242us
I20260709 13:06:49.581125 25919 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:49.583030 25838 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:49.654892 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 149019us
I20260709 13:06:49.658156 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:49.683017 22000 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:49.589732 25654 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:49.683495 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.568559 25523 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:49.646907 25954 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:49.654471 25847 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:49.569551 25980 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.587550 25823 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:49.648273 25935 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.652225 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:06:49.696195 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.681228 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.680687 25631 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.700708 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:49.705478 25955 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.543s	user 0.009s	sys 0.003s
I20260709 13:06:49.701407 25839 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.590371 25859 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.651568 25729 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:49.782774 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 101617us
I20260709 13:06:49.693310 25952 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:49.788138 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 104714us
W20260709 13:06:49.790144 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 100731us
I20260709 13:06:49.680176 25936 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:49.805058 25998 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 13:06:49.685442 25654 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:49.816946 22585 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:06:49.784731 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88656.8us
I20260709 13:06:49.709426 25582 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.778107 25931 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:49.858325 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:49.922719 22000 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:49.926383 22584 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:49.910487 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 807) took 1544 ms. Trace:
W20260709 13:06:49.931072 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 92209.4us
I20260709 13:06:49.809409 25930 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.946007 22005 rpcz_store.cc:276] 0709 13:06:48.365629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.365786 (+   157us) service_pool.cc:224] Handling call
0709 13:06:49.083831 (+718045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.158705 (+ 74874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.910186 (+751481us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54002710d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:49.910407 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1650730240}
I20260709 13:06:49.954532 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:49.856308 25986 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:06:49.838231 25868 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:49.964788 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:49.969966 25575 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.727s	user 0.006s	sys 0.004s
I20260709 13:06:49.970537 25575 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:49.971475 25998 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:49.982277 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2374) took 1973 ms. Trace:
I20260709 13:06:49.982924 22001 rpcz_store.cc:276] 0709 13:06:48.008525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.094353 (+ 85828us) service_pool.cc:224] Handling call
0709 13:06:48.094656 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.095017 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.095042 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.095515 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.163690 (+ 68175us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:48.163730 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.163803 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.527448 (+1363645us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:49.981878 (+454430us) raft_consensus.cc:1685] finished
0709 13:06:49.981910 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.982195 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1496,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":118156160,"thread_start_us":67810,"threads_started":1,"wal-append.queue_time_us":66947}
I20260709 13:06:49.986029 25983 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:49.987015 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55092.5us
I20260709 13:06:49.988001 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:49.989365 25928 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.990476 25993 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:49.991611 25857 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.993613 25980 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.856277 25952 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.999665 25952 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:50.002347 25978 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.005632 25859 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:50.013067 25996 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.531s	user 0.004s	sys 0.007s
W20260709 13:06:50.014245 26001 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 13:06:49.847755 25932 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.856413 25023 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.025162 26005 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:06:50.026803 26005 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:50.027263 26005 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:50.030465 25523 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:50.034571 25874 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.797s	user 0.010s	sys 0.000s
I20260709 13:06:50.035277 25874 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.811609 25631 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.040337 25631 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.041456 25654 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.043431 25654 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.787628 25935 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.047297 25935 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.815425 25839 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:50.049564 25994 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.460s	user 0.001s	sys 0.000s
I20260709 13:06:50.051151 26007 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:50.052426 26007 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:50.052948 26007 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:50.054075 26007 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:49.932646 25582 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.063112 25582 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.077466 25839 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:49.914960 25936 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.082844 26006 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:50.083827 25936 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.083637 26006 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.085242 22588 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:50.086508 26006 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.088042 22000 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:06:50.090929 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 103040us
I20260709 13:06:50.092270 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:50.100471 22587 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.002s
I20260709 13:06:50.101663 22587 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:50.114183 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:50.117707 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:50.140295 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:50.144694 22008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.014s	sys 0.023s
I20260709 13:06:50.145020 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:50.145442 22008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.014s	sys 0.023s
I20260709 13:06:50.180084 21853 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:50.185467 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:50.187187 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:50.198037 21853 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:50.116107 25935 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.116731 25960 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.015s
I20260709 13:06:50.203543 21853 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:50.118729 26006 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.199173 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:50.215996 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 357761us
I20260709 13:06:50.194046 26007 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.218392 25857 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:50.219537 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:50.254092 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:50.219997 22004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.003s	sys 0.010s
W20260709 13:06:50.205468 25960 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.000s	sys 0.018s
I20260709 13:06:50.265806 26014 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:50.198132 25582 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:50.205791 25993 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:50.197125 25936 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:50.255107 22004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.003s	sys 0.010s
I20260709 13:06:50.257546 25343 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:50.259708 25666 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:50.028949 26005 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.205499 25859 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:50.224815 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.318451 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.300735 21926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.003s	sys 0.038s
W20260709 13:06:50.319697 21926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.003s	sys 0.038s
I20260709 13:06:50.206801 25729 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:50.305995 21899 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.006s	sys 0.015s
W20260709 13:06:50.255069 25984 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:06:50.308887 22007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.017s	sys 0.004s
W20260709 13:06:50.304090 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.302179 25933 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.003s
W20260709 13:06:50.322670 21899 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.006s	sys 0.016s
W20260709 13:06:50.317544 25920 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.969s	user 0.014s	sys 0.000s
W20260709 13:06:50.327890 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 926366us
I20260709 13:06:50.304057 25857 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:50.208482 25935 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:50.325984 22007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.017s	sys 0.004s
I20260709 13:06:50.271341 26006 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:06:50.307521 25993 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:50.411151 25993 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.414178 25978 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.414820 25978 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.417773 25978 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:50.383872 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 237321us
W20260709 13:06:50.315282 25990 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.007s	sys 0.013s
W20260709 13:06:50.419518 25990 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.008s	sys 0.013s
W20260709 13:06:50.384610 21854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.009s	sys 0.056s
I20260709 13:06:50.316349 25936 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:50.394011 26020 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.421110 26020 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.322405 25966 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.421523 25936 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.423563 25966 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:50.378966 25903 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.797s	user 0.010s	sys 0.002s
I20260709 13:06:50.426014 25903 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.428175 25903 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.326970 25666 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:50.329159 25859 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:50.357249 21851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.007s	sys 0.016s
W20260709 13:06:50.430303 21851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.007s	sys 0.021s
I20260709 13:06:50.430190 25874 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.429899 25859 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.409258 25857 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.429840 25666 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.328691 22584 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:50.407837 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2323) took 1303 ms. Trace:
W20260709 13:06:50.429811 21854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.009s	sys 0.057s
I20260709 13:06:50.437469 22007 rpcz_store.cc:276] 0709 13:06:49.104097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.253546 (+149449us) service_pool.cc:224] Handling call
0709 13:06:49.253847 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.254121 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:49.519349 (+265228us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.407476 (+888127us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:50.407540 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:50.407583 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.407649 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.407780 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":384501,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":1006,"raft.queue_time_us":25854,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":1088070,"threads_started":2}
W20260709 13:06:50.457702 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4216871100001072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
I20260709 13:06:50.396857 26005 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:50.440450 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:50.502447 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 62079.6us
I20260709 13:06:50.504552 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:50.463392 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.405345 25980 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:50.473290 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2324) took 1351 ms. Trace:
I20260709 13:06:50.460599 25729 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:06:50.498544 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2398) took 1454 ms. Trace:
I20260709 13:06:50.468497 26019 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.464532 26006 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:50.505718 22009 rpcz_store.cc:276] 0709 13:06:49.122004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.255591 (+133587us) service_pool.cc:224] Handling call
0709 13:06:49.255850 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.256048 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:49.644425 (+388377us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.180054 (+535629us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.472455 (+292401us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:50.472819 (+   364us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:50.472897 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.473051 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.473217 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86114,"prepare.queue_time_us":440004,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":106,"raft.queue_time_us":285959,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":11656,"spinlock_wait_cycles":114831616,"thread_start_us":1036206,"threads_started":3,"wal-append.queue_time_us":561683}
I20260709 13:06:50.521071 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 823) took 1011 ms. Trace:
I20260709 13:06:50.512837 21934 rpcz_store.cc:276] 0709 13:06:49.044599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.396685 (+352086us) service_pool.cc:224] Handling call
0709 13:06:50.498350 (+1101665us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b54000979d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.498477 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197158272}
I20260709 13:06:50.472329 25920 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:50.521548 22004 rpcz_store.cc:276] 0709 13:06:49.509286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.526816 (+ 17530us) service_pool.cc:224] Handling call
0709 13:06:49.564373 (+ 37557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:49.565467 (+  1094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.565506 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:49.581889 (+ 16383us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.305319 (+723430us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:50.305680 (+   361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.305756 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.520810 (+215054us) raft_consensus.cc:1685] finished
0709 13:06:50.520831 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.520977 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":719,"raft.queue_time_us":1222,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":625113,"threads_started":1,"wal-append.queue_time_us":780088}
I20260709 13:06:50.532241 26006 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.533432 26030 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.537350 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:50.500983 25582 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:50.508329 25932 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.534381 26027 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.539297 25729 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.573195 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 68697.4us
I20260709 13:06:50.581804 26035 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.535578 25920 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.550522 25993 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.565791 25936 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.528563 26019 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.540155 26030 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.565459 25932 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:50.593658 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:50.594911 26025 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
W20260709 13:06:50.594754 26018 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.095s	user 0.000s	sys 0.002s
W20260709 13:06:50.595147 26015 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.198s	user 0.002s	sys 0.000s
I20260709 13:06:50.592335 26029 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:50.572412 25729 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.624153 25961 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:50.624764 25961 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:50.592711 26035 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.625667 25961 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.626776 25961 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:50.626982 25932 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:50.627853 26035 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:50.629603 25929 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:50.631028 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.617690 25935 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.635183 26019 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:50.635906 25343 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.637076 25920 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.637524 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2268) took 1378 ms. Trace:
W20260709 13:06:50.620947 26017 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:50.567188 26027 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.639020 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2271) took 1378 ms. Trace:
I20260709 13:06:50.638407 25950 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.639400 21854 rpcz_store.cc:276] 0709 13:06:49.260137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.591511 (+331374us) service_pool.cc:224] Handling call
0709 13:06:49.639265 (+ 47754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:49.639677 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:49.639713 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:49.640145 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.438091 (+797946us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:50.438173 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.438338 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.638826 (+200488us) raft_consensus.cc:1685] finished
0709 13:06:50.638847 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.638964 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2503,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":11427,"thread_start_us":656930,"threads_started":1,"wal-append.queue_time_us":807960}
I20260709 13:06:50.637921 26006 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:50.643185 25654 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.000s
I20260709 13:06:50.636932 26030 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.604714 25582 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:50.646870 25997 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.169s	user 0.011s	sys 0.010s
I20260709 13:06:50.648552 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2391) took 1622 ms. Trace:
W20260709 13:06:50.650084 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82681.8us
I20260709 13:06:50.650377 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2269) took 1390 ms. Trace:
I20260709 13:06:50.651728 21851 rpcz_store.cc:276] 0709 13:06:49.259644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.528647 (+269003us) service_pool.cc:224] Handling call
0709 13:06:49.580547 (+ 51900us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:49.586876 (+  6329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.586913 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:49.639798 (+ 52885us) log.cc:844] Serialized 52 byte log entry
0709 13:06:50.431211 (+791413us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:50.565203 (+133992us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.567537 (+  2334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.567621 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.650170 (+ 82549us) raft_consensus.cc:1685] finished
0709 13:06:50.650191 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.650319 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2006,"prepare.queue_time_us":164420,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":7227,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":17285,"thread_start_us":879445,"threads_started":2,"wal-append.queue_time_us":806997}
W20260709 13:06:50.654559 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.640681 26027 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.655608 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2393) took 1904 ms. Trace:
I20260709 13:06:50.655992 22003 rpcz_store.cc:276] 0709 13:06:48.751457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.952590 (+201133us) service_pool.cc:224] Handling call
0709 13:06:48.952883 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.041148 (+ 88265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.655342 (+1614194us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b5400389ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:50.655547 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201608960}
I20260709 13:06:50.648921 21927 rpcz_store.cc:276] 0709 13:06:49.025823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.028191 (+  2368us) service_pool.cc:224] Handling call
0709 13:06:49.028510 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:49.028870 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.028899 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:49.029350 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:06:49.161240 (+131890us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:49.161284 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.161367 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.646227 (+1484860us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:50.648287 (+  2060us) raft_consensus.cc:1685] finished
0709 13:06:50.648309 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.648487 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7086,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":105,"spinlock_wait_cycles":54383872,"thread_start_us":131515,"threads_started":1,"wal-append.queue_time_us":414719}
I20260709 13:06:50.658568 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2390) took 1633 ms. Trace:
I20260709 13:06:50.661867 21926 rpcz_store.cc:276] 0709 13:06:49.025496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.027764 (+  2268us) service_pool.cc:224] Handling call
0709 13:06:49.028025 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:49.466678 (+438653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.466721 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:49.510156 (+ 43435us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.320066 (+809910us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:50.320105 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.581692 (+261587us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54003066d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:50.581749 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.649796 (+ 68047us) raft_consensus.cc:1685] finished
0709 13:06:50.649828 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.658498 (+  8670us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11655,"raft.queue_time_us":2392,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":1003609856,"thread_start_us":759281,"threads_started":1,"wal-append.queue_time_us":938382}
I20260709 13:06:50.603885 25936 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.677690 25936 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.596964 25993 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.651042 21972 leader_election.cc:400] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:50.637902 21859 rpcz_store.cc:276] 0709 13:06:49.259351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.528653 (+269302us) service_pool.cc:224] Handling call
0709 13:06:49.570880 (+ 42227us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:49.583911 (+ 13031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.583959 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:49.586083 (+  2124us) log.cc:844] Serialized 52 byte log entry
0709 13:06:50.143224 (+557141us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:50.143282 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.143395 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.637295 (+493900us) raft_consensus.cc:1685] finished
0709 13:06:50.637319 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.637468 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2800,"spinlock_wait_cycles":97459968,"thread_start_us":485004,"threads_started":1,"wal-append.queue_time_us":661736}
I20260709 13:06:50.689929 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:50.690582 21972 leader_election.cc:403] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:50.666333 25849 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:50.696465 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 95823.1us
W20260709 13:06:50.682578 25911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.638s	user 0.034s	sys 0.017s
I20260709 13:06:50.696971 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:50.688090 25930 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.000s
I20260709 13:06:50.699158 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:50.713874 21926 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.682489 26035 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.680516 25857 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:50.712582 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.683580 25903 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.691658 26014 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:50.732357 26014 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.733376 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2260) took 2881 ms. Trace:
I20260709 13:06:50.693626 25868 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.733767 21856 rpcz_store.cc:276] 0709 13:06:47.852115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.229567 (+377452us) service_pool.cc:224] Handling call
0709 13:06:48.739995 (+510428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.804630 (+ 64635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.733177 (+1928547us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b540008b6d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.733319 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173187840}
I20260709 13:06:50.726872 25966 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.727890 26028 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:50.729141 25980 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:50.753494 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.731560 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 772) took 1152 ms. Trace:
I20260709 13:06:50.755125 22590 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:06:50.755555 22590 pending_rounds.cc:85] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 6
I20260709 13:06:50.755793 22590 pending_rounds.cc:107] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:06:50.756042 22590 raft_consensus.cc:2889] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:50.698513 25911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.654s	user 0.035s	sys 0.017s
I20260709 13:06:50.739441 26043 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:50.739885 25874 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:50.742211 26014 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.743923 25886 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.718825 25849 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.744328 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 1982) took 1910 ms. Trace:
I20260709 13:06:50.761241 21852 rpcz_store.cc:276] 0709 13:06:48.833972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.836517 (+  2545us) service_pool.cc:224] Handling call
0709 13:06:49.070615 (+234098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.208537 (+137922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.744143 (+1535606us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540018f1d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.744271 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537463168}
I20260709 13:06:50.744975 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 902) took 1434 ms. Trace:
I20260709 13:06:50.762318 22592 rpcz_store.cc:276] 0709 13:06:49.310826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.310971 (+   145us) service_pool.cc:224] Handling call
0709 13:06:50.744829 (+1433858us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.744925 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196718848}
I20260709 13:06:50.744593 25868 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.745751 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 980) took 1231 ms. Trace:
I20260709 13:06:50.717310 25930 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.754168 22588 rpcz_store.cc:276] 0709 13:06:49.579189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.848495 (+269306us) service_pool.cc:224] Handling call
0709 13:06:50.731314 (+882819us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400684f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.731489 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297478400}
I20260709 13:06:50.766403 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:50.758390 21926 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:50.754241 22006 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:50.759245 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.767110 22008 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:50.768213 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.760460 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:50.763823 22585 rpcz_store.cc:276] 0709 13:06:49.514139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.700391 (+186252us) service_pool.cc:224] Handling call
0709 13:06:50.745608 (+1045217us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b54006b3ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.745702 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81274752}
W20260709 13:06:50.760154 22593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 892) took 2540 ms (client timeout 1887 ms). Trace:
W20260709 13:06:50.774122 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 122966us
W20260709 13:06:50.770815 22593 rpcz_store.cc:269] 0709 13:06:48.213976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.216876 (+  2900us) service_pool.cc:224] Handling call
0709 13:06:50.754334 (+2537458us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54003791d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.754482 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009105280}
W20260709 13:06:50.758330 25683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.269s	user 0.016s	sys 0.022s
I20260709 13:06:50.779292 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:50.785341 25868 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.757650 25980 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:50.747542 26028 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:50.789713 26028 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.790920 26028 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:50.795168 26044 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:50.802002 26042 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:50.803828 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:50.763221 26050 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.779647 26053 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.069s	user 0.003s	sys 0.000s
I20260709 13:06:50.725389 26035 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.738688 25903 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.823625 25993 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.795357 25939 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:50.824507 25903 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:50.795704 25683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.306s	user 0.016s	sys 0.023s
I20260709 13:06:50.782013 25582 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:50.826063 25582 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:50.826920 25582 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.827618 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 911) took 1427 ms. Trace:
I20260709 13:06:50.828116 25582 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:50.801614 26007 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.000s
I20260709 13:06:50.810321 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.810516 25849 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.818493 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:50.820844 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.734138 25857 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.821532 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:50.824719 22586 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 964) took 2290 ms (client timeout 1862 ms). Trace:
I20260709 13:06:50.827963 22584 rpcz_store.cc:276] 0709 13:06:49.399804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.953498 (+553694us) service_pool.cc:224] Handling call
0709 13:06:49.965083 (+ 11585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:50.437120 (+472037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.827388 (+390268us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540060d6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:50.827562 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835312000}
I20260709 13:06:50.832109 22007 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:50.831470 22008 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:50.838521 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.840327 22008 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:50.841871 25993 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:50.842406 22005 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:50.841490 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:50.836812 22586 rpcz_store.cc:269] 0709 13:06:48.534537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.534712 (+   175us) service_pool.cc:224] Handling call
0709 13:06:50.824533 (+2289821us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b5400664254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.824648 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207558528}
I20260709 13:06:50.845252 22008 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:50.852464 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.821934 26042 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:50.852761 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:50.853690 22007 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:06:50.857442 25993 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:50.858865 25993 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:50.859105 25857 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:50.829672 26050 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.859877 25993 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:50.862385 26050 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.864198 26035 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:50.864920 26020 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.868044 25980 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.868884 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.873461 22007 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:50.874428 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:50.874734 25752 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:50.875597 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:50.875721 21857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2270) took 1615 ms (client timeout 1961 ms). Trace:
W20260709 13:06:50.876132 21857 rpcz_store.cc:269] 0709 13:06:49.259895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.396931 (+137036us) service_pool.cc:224] Handling call
0709 13:06:50.875539 (+1478608us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.875665 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353230208}
W20260709 13:06:50.884023 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.885900 26020 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.886451 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2322) took 1839 ms. Trace:
I20260709 13:06:50.886622 26042 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:50.886476 26020 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:50.891662 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.900342 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.876748 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.903378 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.904248 22593 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:06:50.906158 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.906536 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.907387 22593 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:50.908574 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.909397 22585 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:50.889981 22002 rpcz_store.cc:276] 0709 13:06:49.046714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.236853 (+190139us) service_pool.cc:224] Handling call
0709 13:06:49.237146 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.447002 (+209856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.886210 (+1439208us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540042b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:50.886387 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480753920}
I20260709 13:06:50.915328 21858 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:50.916429 22007 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:50.919600 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 612) took 1772 ms. Trace:
I20260709 13:06:50.919812 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.919986 21858 rpcz_store.cc:276] 0709 13:06:49.147531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.147659 (+   128us) service_pool.cc:224] Handling call
0709 13:06:49.148130 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.911330 (+1763200us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54003b3920. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:06:50.919359 (+  8029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.919546 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23416064}
I20260709 13:06:50.920475 22003 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:50.921444 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.921638 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.922015 22003 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:50.921446 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:50.922490 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:50.923470 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52424.7us
I20260709 13:06:50.921711 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:50.922822 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.924543 22591 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:50.924885 22003 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:50.923998 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.925752 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.926451 22003 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:50.927729 22003 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:06:50.930904 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.931838 22593 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:50.936292 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:50.937228 21926 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.941815 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:50.945732 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.947738 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.947966 22592 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 6.
I20260709 13:06:50.948254 21934 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:50.922363 22589 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:06:50.948715 21927 raft_consensus.cc:2368] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:50.956710 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:50.957508 21857 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:50.918181 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:50.958773 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:50.958879 22007 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:50.959636 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:50.960340 21856 raft_consensus.cc:2368] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:50.961333 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:50.961526 21926 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:50.957533 22585 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:50.965653 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.966374 21854 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:50.966442 22584 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:50.943866 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:50.967988 21858 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:50.969504 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:50.970548 22002 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:50.972782 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:50.977303 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:50.977972 21851 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:50.978783 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:50.979382 21930 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:50.980233 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.980718 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:50.965572 25752 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:06:50.981477 21935 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:06:50.982573 25953 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:50.982853 26020 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:50.983168 25953 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:50.962301 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:50.980799 22589 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:06:50.985664 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.986227 22589 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:50.990761 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:50.991763 22585 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:50.991679 22591 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:50.911044 25618 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.993155 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 909) took 1667 ms. Trace:
I20260709 13:06:51.001312 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.993877 22593 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:51.011211 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:50.947690 26050 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.023905 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.024519 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.000267 22590 rpcz_store.cc:276] 0709 13:06:49.325401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.474168 (+148767us) service_pool.cc:224] Handling call
0709 13:06:50.754716 (+1280548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:50.756543 (+  1827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.992950 (+236407us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003791d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:50.993101 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944191232}
W20260709 13:06:51.025498 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45059.4us
I20260709 13:06:51.030664 21854 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:51.063856 21934 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:51.065091 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.065296 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.090621 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.091550 21858 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:51.030104 25929 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.106179 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.106950 21851 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
W20260709 13:06:51.133836 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.092581 25963 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:51.030308 25618 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.089586 25582 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.065825 25868 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:06:51.089393 26050 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.104233 25938 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.265292 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 181889us
W20260709 13:06:51.199415 26056 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:06:51.269869 26060 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:06:51.271376 26058 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:06:51.271701 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 912) took 1583 ms. Trace:
W20260709 13:06:51.272959 26059 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:06:51.272070 22587 rpcz_store.cc:276] 0709 13:06:49.688264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.197150 (+508886us) service_pool.cc:224] Handling call
0709 13:06:50.215734 (+ 18584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.216268 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.216310 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.794572 (+578262us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b2c001465b8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:06:50.795050 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.911746 (+116696us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:50.911881 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.912131 (+   250us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.271404 (+359273us) raft_consensus.cc:1685] finished
0709 13:06:51.271425 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.271642 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":37509,"spinlock_wait_cycles":2545408,"thread_start_us":115911,"threads_started":1,"wal-append.queue_time_us":286535}
I20260709 13:06:51.274827 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:51.276526 21927 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:51.262044 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.263470 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:51.265355 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:51.171679 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 218958us
I20260709 13:06:51.261143 22002 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:51.284148 21855 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:51.285434 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.171010 25752 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.288591 21972 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.776s (SENT)
I20260709 13:06:51.311941 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:51.288446 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.267886 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:51.288524 21935 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:51.315073 21930 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:06:51.315260 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47465.7us
I20260709 13:06:51.286258 21851 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:51.291844 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.318020 21934 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:51.244328 26005 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:51.318585 26028 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.002s
I20260709 13:06:51.257854 25963 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:51.274547 25991 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.316984 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:51.122134 25929 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:51.330554 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 55823.6us
I20260709 13:06:51.334328 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:51.282200 25523 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:51.336513 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:51.341890 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1937) took 2485 ms (client timeout 1662 ms). Trace:
W20260709 13:06:51.342293 21929 rpcz_store.cc:269] 0709 13:06:48.856692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.856805 (+   113us) service_pool.cc:224] Handling call
0709 13:06:51.341633 (+2484828us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54003539d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:51.341802 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736401152}
I20260709 13:06:51.266618 25582 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.354866 22007 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:51.355706 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.356469 22008 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:51.348655 25523 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.321731 25953 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.331301 26007 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:51.332472 25932 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.335979 26028 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:51.318953 26005 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.280180 25938 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.364519 22002 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:51.391450 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:51.392890 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.393888 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.394483 21857 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:51.359377 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:51.333384 25930 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:51.454958 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.442284 21860 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:51.443594 21855 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:51.455858 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.456640 21857 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:51.340267 26020 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:51.383131 25978 raft_consensus.cc:993] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
W20260709 13:06:51.397115 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.467311 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:51.414314 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.400593 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.343505 25966 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:51.472685 21852 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:51.401100 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.372881 26007 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.472419 21851 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:06:51.389583 25631 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:51.368602 26000 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.668s	user 0.011s	sys 0.001s
I20260709 13:06:51.371330 25618 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.365790 25980 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:51.365614 26050 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.477226 26007 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.461824 26020 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.474869 21853 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:51.510627 25938 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.582662 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:51.513693 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 154354us
I20260709 13:06:51.520217 25929 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.338709 25991 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.570833 25618 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.363608 25325 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:51.594875 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:51.441736 25859 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.473150 25966 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.469513 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 180934us
I20260709 13:06:51.368220 25523 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.381350 26005 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.375114 25932 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.319375 25752 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.609551 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.585292 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:51.613598 21853 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:51.613745 22590 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEARNER]: Advancing to term 10
I20260709 13:06:51.516820 25631 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:51.520954 25936 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:51.368637 25963 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.583606 21851 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:51.588119 25938 raft_consensus.cc:1081] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:06:51.587365 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:51.349431 25898 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.125s
I20260709 13:06:51.590528 25991 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.570220 25939 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.592617 25582 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.581913 26020 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:51.576695 25980 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.580685 25729 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.606060 22593 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:06:51.653505 22593 pending_rounds.cc:85] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 10
I20260709 13:06:51.653888 22593 pending_rounds.cc:107] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:06:51.654218 22593 raft_consensus.cc:2889] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:51.615799 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:51.656080 21859 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:51.613047 21858 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
W20260709 13:06:51.657321 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.618149 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:51.660696 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:51.661057 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 397752us
I20260709 13:06:51.663335 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.625113 22584 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:51.598012 25325 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.665364 25325 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.667047 21853 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:51.642175 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:51.643275 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:51.645911 21856 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:51.669396 22592 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:51.603861 26028 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.670199 26028 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.605624 25859 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.657060 25939 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.619812 25936 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.599639 25618 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.678543 25752 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.672976 26028 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:51.699160 26028 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:51.677806 26062 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.184s	user 0.003s	sys 0.000s
I20260709 13:06:51.621474 26005 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.702767 26005 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.664106 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:51.663688 25980 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.606191 25966 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:51.705478 25631 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.708026 25938 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:51.710673 25932 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.650746 25991 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:51.718565 22591 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:06:51.720824 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56848.4us
W20260709 13:06:51.721431 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58170.4us
I20260709 13:06:51.724583 22590 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:51.621603 21854 raft_consensus.cc:2410] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:51.728719 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:51.735450 25732 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:51.739353 25928 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:51.691977 25936 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:51.662923 25729 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.691017 25868 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:51.740078 25928 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.675246 25859 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.739531 25732 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:51.740680 25991 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.757110 25991 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.691607 25929 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:06:51.759954 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.744714 25868 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.748518 25928 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:51.768874 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.753484 25732 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:51.771391 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.773129 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.774242 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:51.777040 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.777652 21931 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 6.
I20260709 13:06:51.778548 21934 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:51.766077 25868 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.780165 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.784118 21930 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:51.748760 25729 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:51.780628 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 428864us
I20260709 13:06:51.758121 25863 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:06:51.799908 21929 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
W20260709 13:06:51.801508 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 448669us
I20260709 13:06:51.803102 21932 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:51.803399 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.805598 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.805687 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.805989 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.806387 21930 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:51.799024 25958 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:51.809027 25931 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.835557 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.839689 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.840889 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.836633 21930 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:51.819317 25859 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:51.898850 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.809952 26072 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:06:51.906921 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.906505 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.904189 25618 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:51.874941 25931 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:51.909416 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.910193 21932 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:06:51.877545 25752 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:51.918076 21901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.014s	sys 0.018s
I20260709 13:06:51.922104 22007 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:51.930519 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.890501 25859 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.931535 22008 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:51.922573 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.888837 25846 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:51.925625 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:51.926513 22587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.020s	sys 0.013s
I20260709 13:06:51.927167 22006 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:06:51.933630 21901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.014s	sys 0.018s
I20260709 13:06:51.937120 22004 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:06:51.934011 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:51.937247 22587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.021s	sys 0.013s
I20260709 13:06:51.940172 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.939319 21972 leader_election.cc:400] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:51.941480 21972 leader_election.cc:403] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:51.941805 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.944314 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:51.945241 22002 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:51.905128 25939 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:51.947456 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 290100us
I20260709 13:06:51.947882 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.951643 21932 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:51.951732 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:51.917390 25752 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.953299 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.932195 22001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:06:51.955839 22001 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:51.915021 25991 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:51.956405 22001 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:51.931280 25849 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:51.931602 26042 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.932543 25969 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.933624 26005 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:51.936010 25859 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:51.973124 26082 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:51.968502 22004 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:51.911422 25619 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:06:51.978070 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.978713 21930 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:51.978869 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.979368 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.979605 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:51.944532 25582 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.982796 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 188731us
W20260709 13:06:51.964424 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 226196us
I20260709 13:06:51.935813 25666 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:51.974160 21935 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:51.989460 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.974126 26075 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.005587 21931 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:51.958266 25752 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:52.008730 21929 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:52.006634 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:51.969915 25849 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:52.012773 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:52.016198 21929 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:06:52.017305 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:52.019042 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.021198 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.009647 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:52.026719 21898 connection.cc:256] RPC call timeout handler was delayed by -1.6408184509999728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 13:06:52.028079 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:52.012900 22001 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.032126 25961 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:52.032666 25961 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.033128 25961 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:52.033577 25523 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:52.034323 25523 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:51.993556 26005 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:06:52.039067 21935 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:52.043057 22001 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.049535 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:52.050577 21930 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:52.009675 26074 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:06:52.055063 25931 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:06:52.055578 26068 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:06:52.055694 25931 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:52.056142 25931 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.056522 25931 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:52.008247 25857 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:52.057344 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 913) took 1137 ms. Trace:
I20260709 13:06:52.057636 25857 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.057724 22588 rpcz_store.cc:276] 0709 13:06:50.920332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.954762 (+ 34430us) service_pool.cc:224] Handling call
0709 13:06:50.977798 (+ 23036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.026349 (+ 48551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.026420 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.062919 (+ 36499us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.458639 (+395720us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:51.458679 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.458748 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.057105 (+598357us) raft_consensus.cc:1685] finished
0709 13:06:52.057127 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.057270 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47147,"raft.queue_time_us":51849,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3907,"thread_start_us":330202,"threads_started":1,"wal-append.queue_time_us":687939}
I20260709 13:06:52.055117 25666 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:52.060074 25666 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:52.060457 25666 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:51.969326 26070 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:06:51.981715 26042 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.038631 25950 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:52.020401 25849 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:51.975522 25991 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:52.089190 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 286843us
I20260709 13:06:52.072389 26072 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:52.006621 25582 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.036954 25523 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:52.042344 25729 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:51.980963 25969 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:52.164709 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:06:52.094642 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 154538us
I20260709 13:06:52.096753 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:52.165866 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:52.166627 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:52.167150 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:52.167840 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:52.104818 25868 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.038417 26082 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:52.014127 26075 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:52.152140 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 219116us
I20260709 13:06:52.109251 25619 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:52.094770 25761 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:52.120146 26072 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.090695 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:52.335254 21935 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:52.101609 25732 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:52.310626 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 358949us
I20260709 13:06:52.347319 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:52.073464 25857 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.061190 26078 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
W20260709 13:06:52.322103 21934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.002s	sys 0.054s
I20260709 13:06:52.340621 25619 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:52.342902 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:52.351984 21933 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
W20260709 13:06:52.348778 21934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.002s	sys 0.054s
W20260709 13:06:52.352644 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 346780us
W20260709 13:06:52.334473 25938 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.626s	user 0.007s	sys 0.012s
I20260709 13:06:52.163583 25929 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.368691 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:52.320281 25729 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:52.335911 25761 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:52.398661 21933 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:06:52.405201 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 361120us
I20260709 13:06:52.416437 21935 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:52.359015 25619 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:52.351609 26079 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:52.104789 25966 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.441583 21930 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:52.441589 21933 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:52.378715 25938 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:52.426308 26035 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:52.450436 26072 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.455760 25732 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:52.460346 25963 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:52.462327 26082 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:52.511528 25619 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:52.468531 25936 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.001s
I20260709 13:06:52.464010 25666 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.470201 25969 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:52.470602 25868 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:52.471740 26075 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:52.521888 26073 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.426414 25729 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:52.472517 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:52.476023 26020 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.000s
I20260709 13:06:52.476728 26086 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:52.477207 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 69151.8us
I20260709 13:06:52.478013 26069 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:52.478755 21746 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.185s	user 0.020s	sys 0.019s
W20260709 13:06:52.480330 25325 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.814s	user 0.008s	sys 0.001s
I20260709 13:06:52.403537 25991 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.425370 26088 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:52.403157 25929 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.481455 25761 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:52.482734 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.468729 25931 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:52.485776 26097 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:52.486645 25980 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:52.487769 25915 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:52.489086 25582 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.491029 26074 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:52.491818 25523 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.492303 25849 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:52.435976 25953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.037s	sys 0.051s
I20260709 13:06:52.472442 26070 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:52.425864 26007 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.827s	user 0.013s	sys 0.000s
I20260709 13:06:52.503527 26028 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.582098 25868 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:52.509824 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 341135us
W20260709 13:06:52.515200 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.010s
W20260709 13:06:52.520364 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 172481us
W20260709 13:06:52.612651 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.004s	sys 0.010s
W20260709 13:06:52.613057 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 626629us
I20260709 13:06:52.613126 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:52.613875 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.614336 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:52.499959 26042 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.465169 25343 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:52.584353 25915 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:52.548978 26073 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:52.540321 22586 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:52.521377 26079 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.472000 25857 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:52.539740 25978 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.031s	sys 0.053s
I20260709 13:06:52.510948 26084 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:06:52.524250 26050 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:52.562600 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.561712 21898 leader_election.cc:400] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
I20260709 13:06:52.579150 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:52.580001 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:52.581795 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:52.555225 25958 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.729s	user 0.007s	sys 0.004s
I20260709 13:06:52.577878 25863 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.586280 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:52.576053 25928 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.000s
I20260709 13:06:52.682042 22003 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:52.558578 26105 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.555717 26094 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:52.590467 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:52.601177 25950 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:06:52.556306 26086 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.546535 25936 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.561726 25325 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.615126 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:52.577912 26088 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:52.624948 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:52.608234 25761 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:52.608111 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:52.539009 21928 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:52.554015 26020 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:52.607043 25859 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260709 13:06:52.610739 21758 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 1 to 9, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 9 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:52.653621 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 374249us
W20260709 13:06:52.658459 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 493783us
W20260709 13:06:52.657733 22556 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.008s	sys 0.016s
W20260709 13:06:52.764717 22556 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.008s	sys 0.016s
I20260709 13:06:52.671690 21898 leader_election.cc:403] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:06:52.764988 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:52.766532 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:52.672415 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 317952us
I20260709 13:06:52.767948 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:52.674005 22587 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:52.675854 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:52.769326 22585 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:52.604967 25931 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:52.744464 25936 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.712297 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:52.644718 25915 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:52.775357 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:52.625975 26005 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
W20260709 13:06:52.776289 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 215994us
I20260709 13:06:52.776641 26005 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:52.637982 26042 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:52.777340 26042 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:52.777863 26042 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:52.779968 25631 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11.
W20260709 13:06:52.667709 25978 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.032s	sys 0.054s
I20260709 13:06:52.718148 22589 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:52.782053 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:52.720050 21932 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.018s	sys 0.067s
I20260709 13:06:52.608367 25523 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.685647 25863 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.723246 26102 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.635308 26007 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.639281 26028 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.644140 25991 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:06:52.640591 26082 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:52.724042 26105 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.744436 26079 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.730569 26086 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:52.742311 25961 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.001s
W20260709 13:06:52.690658 25752 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.663s	user 0.007s	sys 0.003s
I20260709 13:06:52.747628 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.638506 26035 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:06:52.749197 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.867807 26035 consensus_queue.cc:1243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:52.867580 25752 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.867982 21858 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:52.868448 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.750104 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:52.753160 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 138106us
I20260709 13:06:52.869707 25752 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.870168 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:52.754006 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.755707 21927 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:52.686028 25960 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.639809 25868 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.745026 26088 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.742501 25325 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.758909 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.759764 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.760238 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.670243 26070 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:06:52.720053 26078 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:52.814985 26082 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:52.770758 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:52.899669 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 129012us
W20260709 13:06:52.771685 21757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 306) took 31221 ms (client timeout 15000 ms). Trace:
W20260709 13:06:52.903399 21757 rpcz_store.cc:269] 0709 13:06:21.535390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.549168 (+ 13778us) service_pool.cc:224] Handling call
0709 13:06:52.756241 (+31207073us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:45.648311 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:45.652157 (+  3846us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:45.652307 (+   150us) write_op.cc:435] Acquired schema lock
0709 13:06:45.652344 (+    37us) tablet.cc:662] Decoding operations
0709 13:06:45.653299 (+   955us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:45.653366 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:45.653397 (+    31us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:06:45.654054 (+   657us) tablet.cc:701] Row locks acquired
0709 13:06:45.654102 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:06:45.654286 (+   184us) write_op.cc:270] Start()
0709 13:06:45.654409 (+   123us) write_op.cc:276] Timestamp: P: 1783602405654220 usec, L: 0
0709 13:06:45.654432 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:06:45.655157 (+   725us) log.cc:844] Serialized 2860 byte log entry
0709 13:06:46.326389 (+671232us) op_driver.cc:464] REPLICATION: finished
0709 13:06:46.523684 (+197295us) write_op.cc:301] APPLY: starting
0709 13:06:46.523821 (+   137us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:46.594482 (+ 70661us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:46.594646 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:46.595385 (+   739us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:46.595728 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:06:46.595786 (+    58us) write_op.cc:312] APPLY: finished
0709 13:06:46.596184 (+   398us) log.cc:844] Serialized 59 byte log entry
0709 13:06:46.596398 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:46.596524 (+   126us) write_op.cc:454] Released schema lock
0709 13:06:46.596595 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:46.628137 (+ 31542us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:46.629949 (+  1812us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1390246,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":182275,"apply.run_cpu_time_us":4091,"apply.run_wall_time_us":79436,"num_ops":7,"prepare.queue_time_us":474068,"prepare.run_cpu_time_us":26445,"prepare.run_wall_time_us":318854,"raft.queue_time_us":194253,"raft.run_cpu_time_us":22355,"raft.run_wall_time_us":183504,"replication_time_us":664628,"thread_start_us":755407,"threads_started":4,"wal-append.queue_time_us":236377,"wal-append.run_cpu_time_us":24774,"wal-append.run_wall_time_us":1669131}]]}
I20260709 13:06:52.773921 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:52.909459 22006 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:52.909758 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:52.765076 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 727557us
W20260709 13:06:52.910667 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.911301 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:52.776114 21851 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
W20260709 13:06:52.785284 21932 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.019s	sys 0.067s
I20260709 13:06:52.912747 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.786985 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:52.787025 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:06:52.947105 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 160174us
I20260709 13:06:52.947501 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.948376 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:52.789639 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 775) took 1195 ms. Trace:
I20260709 13:06:52.948989 22591 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:52.750581 26103 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:52.794257 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.959416 22007 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:06:52.710436 25928 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.791596 21758 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 7, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.910619 26082 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:52.801224 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:52.933565 26005 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.936162 25980 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.721684 26094 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.786842 26090 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:52.785952 25973 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:52.866494 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.866870 22587 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:52.749534 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 124671us
I20260709 13:06:52.865952 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.877009 22590 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:52.878621 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:52.800463 25953 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:52.797835 25929 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:52.798246 26093 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:52.772897 25859 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:52.770539 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:52.811308 25619 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.911593 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.937849 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:52.961385 26078 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:06:52.913771 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.881630 22009 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:52.945822 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.871309 25729 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:52.949153 22586 rpcz_store.cc:276] 0709 13:06:51.594356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.594548 (+   192us) service_pool.cc:224] Handling call
0709 13:06:52.788902 (+1194354us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400664254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:52.789569 (+   667us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171896704}
I20260709 13:06:52.949892 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:52.812050 26102 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.950382 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.132889 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:52.956192 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169271us
I20260709 13:06:53.134022 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.135491 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:52.897030 26079 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:52.962589 25931 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.178s
I20260709 13:06:53.002446 26005 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.140477 26079 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.039130 26093 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.141439 26093 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.142247 26005 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:53.142535 26079 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:53.143170 26114 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:53.081174 21746 maintenance_manager.cc:643] P 85f464a9537546118c8c1817bd5574c1: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 10.116s	user 0.198s	sys 0.091s Metrics: {"bytes_written":30121,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":802598,"dirs.run_cpu_time_us":721,"dirs.run_wall_time_us":211243,"drs_written":1,"lbm_read_time_us":49811,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":3,"lbm_write_time_us":318471,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":11,"lbm_writes_lt_1ms":21,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"spinlock_wait_cycles":177262464,"thread_start_us":824446,"threads_started":2,"update_count":7,"wal-append.queue_time_us":630470}
I20260709 13:06:53.083465 26028 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:52.934119 25936 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:53.131351 25582 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:52.813234 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:53.145396 25582 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.145861 25582 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:53.146083 22593 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:53.146483 21747 maintenance_manager.cc:419] P 85f464a9537546118c8c1817bd5574c1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9122 bytes on disk
I20260709 13:06:53.033773 25666 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.131237 25732 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:53.147482 25991 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:52.936028 26035 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:52.770390 25966 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:52.963207 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.962615 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.964509 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:52.880297 25961 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.016146 22585 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:06:53.016050 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:53.002791 21758 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 8, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.015911 21859 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:53.021247 21926 raft_consensus.cc:2368] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:06:52.888797 25960 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:52.940140 25752 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:06:52.958895 26103 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.035420 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 832) took 2167 ms. Trace:
I20260709 13:06:53.039157 21931 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:53.040030 21853 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:06:52.971689 25846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.012s	sys 0.024s
I20260709 13:06:52.960707 26100 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.129s
I20260709 13:06:53.039736 21858 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:53.097174 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2416) took 1116 ms. Trace:
I20260709 13:06:53.099215 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.111310 21856 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:53.023396 25980 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:53.111428 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 340975us
I20260709 13:06:53.086153 26105 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.123485 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:52.961596 26054 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:52.961534 25325 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:53.263671 21746 maintenance_manager.cc:643] P 85f464a9537546118c8c1817bd5574c1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.113s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":457,"lbm_reads_lt_1ms":4}
I20260709 13:06:53.029495 26020 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.287798 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.289474 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.290306 21859 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:53.290822 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.961699 25958 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:53.292117 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.292737 22007 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.147982 25991 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:53.149533 26073 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.130502 22006 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:53.309015 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:52.960278 26111 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.263687 26103 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:53.136780 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 186451us
I20260709 13:06:53.314898 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.136817 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:53.137780 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.003820 25950 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:53.297719 25958 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:53.080336 26112 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:06:53.031935 26092 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:53.029238 26007 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.297986 21758 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 6, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), now has a pending config: VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130), VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 6 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:06:53.217929 22558 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.20.66.190:46167, user_credentials={real_user=slave}} blocked reactor thread for 66178.2us
I20260709 13:06:53.302110 25991 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.181651 21860 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:53.262456 22003 rpcz_store.cc:276] 0709 13:06:50.868237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.946830 (+ 78593us) service_pool.cc:224] Handling call
0709 13:06:50.947077 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.682650 (+1735573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.035125 (+352475us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540042bdd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:53.035346 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3988850048}
W20260709 13:06:53.244740 21972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.642s (SENT) [suppressed 10 similar messages]
I20260709 13:06:53.082827 25993 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.033651 25863 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.038141 25868 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.267238 21928 rpcz_store.cc:276] 0709 13:06:51.980445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.981037 (+   592us) service_pool.cc:224] Handling call
0709 13:06:53.096958 (+1115921us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54004295d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:53.097104 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197633408}
W20260709 13:06:53.118530 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 180727us
I20260709 13:06:53.335260 25859 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.269690 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:53.335563 25619 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.336081 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 722) took 1549 ms. Trace:
I20260709 13:06:53.317668 26111 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:53.336262 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66669.9us
I20260709 13:06:52.960585 26106 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.267493 22008 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.337136 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.338310 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:53.338634 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:53.126897 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 716) took 1356 ms (client timeout 1575 ms). Trace:
W20260709 13:06:53.339327 21927 rpcz_store.cc:269] 0709 13:06:51.770176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.770988 (+   812us) service_pool.cc:224] Handling call
0709 13:06:53.126492 (+1355504us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:53.126804 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250619264}
I20260709 13:06:53.283092 21857 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:53.283746 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 267762us
I20260709 13:06:53.341425 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:53.341650 25932 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:53.342396 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:53.343641 26083 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:53.344056 26083 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.344390 26083 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.129444 22591 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:53.345204 25963 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.345670 25963 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.346029 25963 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:53.296840 22002 raft_consensus.cc:2410] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:06:53.340781 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.347643 21857 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:53.222072 26079 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.283104 26117 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:06:53.349705 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.349542 26079 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.348819 21857 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:53.350426 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2425) took 1073 ms. Trace:
I20260709 13:06:53.301167 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.264729 25938 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.351657 21857 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:53.352054 25938 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.151010 26101 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:53.353520 26101 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.217805 25936 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:52.962352 25849 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.316483 25958 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:53.020915 26086 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.266937 25915 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.315840 22005 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:53.385483 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.386159 22005 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.285672 26114 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:06:53.316764 21974 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:53.387216 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 70516.4us
I20260709 13:06:53.389081 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.323342 22007 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:53.288874 25523 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.285152 25325 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:53.291663 26109 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:53.150431 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.187975 21855 raft_consensus.cc:2410] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:06:53.327260 22006 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
W20260709 13:06:53.331933 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2423) took 1345 ms (client timeout 1772 ms). Trace:
W20260709 13:06:53.333688 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 196928us
I20260709 13:06:53.329547 26092 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:53.383208 26042 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 13:06:53.286710 21854 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:53.267277 25846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.307s	user 0.020s	sys 0.062s
I20260709 13:06:53.300688 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.355795 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.387058 25868 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:53.354974 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.354943 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.359688 22591 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:06:53.353914 26101 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.424770 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.331117 21758 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 7, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.451292 22009 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:53.351683 26079 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:53.362313 25993 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.364416 26035 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:53.350822 22001 rpcz_store.cc:276] 0709 13:06:52.277346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.277556 (+   210us) service_pool.cc:224] Handling call
0709 13:06:52.277995 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.326880 (+ 48885us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:52.678272 (+351392us) log.cc:844] Serialized 14 byte log entry
0709 13:06:52.971163 (+292891us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.001474 (+ 30311us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.001971 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.002104 (+   133us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:53.002133 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.002197 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.082613 (+ 80416us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b18002ead40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b746a358f 00007f3b6d6e2571 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd
0709 13:06:53.087714 (+  5101us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18002ead40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:53.088739 (+  1025us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b18002ead40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1
0709 13:06:53.218400 (+129661us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b18002ead40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:53.350220 (+131820us) raft_consensus.cc:1685] finished
0709 13:06:53.350244 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.350365 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113854,"prepare.queue_time_us":552267,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":5894,"raft.queue_time_us":263396,"raft.run_cpu_time_us":435,"raft.run_wall_time_us":66166,"spinlock_wait_cycles":136154112,"thread_start_us":549713,"threads_started":2,"wal-append.queue_time_us":349778}
I20260709 13:06:53.473579 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.406843 25931 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.025529 25973 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.373937 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:53.380509 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:53.284930 26054 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.406481 26109 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.381994 22593 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:53.382766 22008 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:06:53.383152 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:53.284017 26028 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:53.398670 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:53.482810 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109109us
I20260709 13:06:53.482811 22585 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:53.486078 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:53.390553 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.342458 25932 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:53.404006 21931 rpcz_store.cc:269] 0709 13:06:51.986550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.006378 (+ 19828us) service_pool.cc:224] Handling call
0709 13:06:53.331782 (+1325404us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002a8854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:53.331872 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2365060992}
I20260709 13:06:53.408149 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:53.489167 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81095.6us
I20260709 13:06:53.489861 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:53.338819 21851 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:53.411846 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.492653 22007 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:06:53.030488 25898 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:53.495095 22007 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:53.409552 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.497329 25752 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:53.497573 26125 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:53.498039 26094 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.498096 25752 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:53.418567 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 717) took 1646 ms. Trace:
I20260709 13:06:53.499217 21926 rpcz_store.cc:276] 0709 13:06:51.772637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.772874 (+   237us) service_pool.cc:224] Handling call
0709 13:06:53.418315 (+1645441us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002e95d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:53.418496 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2868319616}
I20260709 13:06:53.499594 25752 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:53.421380 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:53.500667 22588 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:53.356324 21934 rpcz_store.cc:276] 0709 13:06:51.787027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.787159 (+   132us) service_pool.cc:224] Handling call
0709 13:06:51.805191 (+ 18032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.353248 (+548057us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:52.353293 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:52.365228 (+ 11935us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.747714 (+382486us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:52.747748 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.747813 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.335870 (+588057us) raft_consensus.cc:1685] finished
0709 13:06:53.335891 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.336025 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":224539,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12038,"thread_start_us":836333,"threads_started":2,"wal-append.queue_time_us":471954}
W20260709 13:06:53.457145 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 102270us
I20260709 13:06:53.501395 21758 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.502462 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.502658 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:53.503374 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.504132 26080 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:53.504503 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.407773 25325 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:53.458168 22591 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:53.508376 25849 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:53.508699 25929 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:53.509271 25929 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:53.509505 25936 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.510155 25929 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.510524 26101 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.512660 26101 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.367776 25618 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.475735 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.478344 26020 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:53.478561 25868 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:53.325377 26104 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.504720 26080 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:53.557809 26093 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.559453 26089 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.560132 26093 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.559971 26089 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.562155 26089 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.562801 25973 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.505131 26007 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:53.403754 26078 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:53.505069 26028 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.506017 22006 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:53.523485 26130 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.565486 26078 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:53.565531 26130 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:53.566241 26130 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:53.566563 26078 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.419606 25846 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:53.568744 26103 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.571368 26103 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.571854 25993 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:53.572163 26020 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:53.487730 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:53.572882 26028 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.573388 26080 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:53.517200 21851 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:53.400065 26086 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.409523 26105 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.457710 25928 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.513495 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.569262 25846 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.537842 25618 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.575892 25618 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:53.576710 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 63279us
I20260709 13:06:53.510537 22005 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:06:53.548846 25960 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.518374 25523 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:53.578670 25523 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:53.579102 25523 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:53.579404 21851 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 7. Current term is 8. Ops: [5.8-7.9]
I20260709 13:06:53.580528 25950 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:53.537652 25849 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:06:53.483433 26054 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:53.555068 25980 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:53.524250 26087 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:06:53.555641 25915 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:53.604347 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 254703us
I20260709 13:06:53.488917 25931 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:53.557518 26104 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.559015 25958 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.540077 25868 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:53.564208 26007 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:53.580986 25950 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:53.538031 25619 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:06:53.503072 21758 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 10, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 10 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.583967 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2426) took 1265 ms. Trace:
I20260709 13:06:53.584607 21856 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:53.419407 26042 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.488842 26109 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.590742 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:53.591702 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:53.545905 25938 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:53.597069 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109413us
I20260709 13:06:53.600184 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:53.560771 26093 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:53.605180 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.617949 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:53.618486 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 227960us
I20260709 13:06:53.542178 25732 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:53.630455 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.573141 26020 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.623526 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.584757 25961 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.674552 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:53.639430 21854 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:53.641637 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.636590 22004 rpcz_store.cc:276] 0709 13:06:52.318844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.319472 (+   628us) service_pool.cc:224] Handling call
0709 13:06:52.340149 (+ 20677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.506856 (+166707us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:52.824299 (+317443us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.406041 (+581742us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.424112 (+ 18071us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.424523 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.424645 (+   122us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:53.424675 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.424733 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.583741 (+159008us) raft_consensus.cc:1685] finished
0709 13:06:53.583760 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.583903 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83990,"prepare.queue_time_us":456925,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":237614,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":27400,"thread_start_us":964706,"threads_started":4,"wal-append.queue_time_us":148204}
I20260709 13:06:53.593837 26080 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.675042 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.646993 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:53.695225 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48297.6us
I20260709 13:06:53.693885 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:53.696295 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 451536us
I20260709 13:06:53.655639 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.598143 26111 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:53.602723 25973 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:53.627564 26106 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:53.672670 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:53.617914 25931 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:53.675026 21758 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.677323 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:53.678593 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.679956 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:53.681157 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.717468 21932 raft_consensus.cc:2376] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10. Re-sending same reply.
I20260709 13:06:53.717623 21930 raft_consensus.cc:2376] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11. Re-sending same reply.
I20260709 13:06:53.681529 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.682719 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.674546 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:53.667662 22586 raft_consensus.cc:2368] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12.
I20260709 13:06:53.663863 25980 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.697147 26103 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:53.711223 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:53.713724 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:53.697661 26080 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.664121 25666 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:53.717422 21933 raft_consensus.cc:2376] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11. Re-sending same reply.
I20260709 13:06:53.537104 26092 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.759289 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:53.704734 22009 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:53.721979 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66427.6us
I20260709 13:06:53.674211 26073 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:53.619892 25963 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.693630 26086 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:53.727804 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:53.784792 22004 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.728526 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:53.704550 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 62998.6us
I20260709 13:06:53.693113 26093 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.729897 25863 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:53.731544 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.790851 21934 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.727599 26114 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:53.734550 25980 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.713814 25928 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:06:53.728088 25868 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.717757 26106 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:53.736608 26103 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:53.743818 25958 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:53.726426 25961 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:53.744899 25932 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:53.746263 25582 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.003s
I20260709 13:06:53.728849 21758 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 9, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 9 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.752875 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:53.723610 21935 raft_consensus.cc:2376] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9. Re-sending same reply.
I20260709 13:06:53.769829 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.822037 21928 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:53.822220 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 69478.5us
I20260709 13:06:53.770408 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:53.823217 21932 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:53.776891 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:53.778314 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:53.770090 26073 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.711210 25973 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.703742 25938 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:53.787497 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.780510 25325 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.800005 22584 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:06:53.802837 21933 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:06:53.824669 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47872.6us
I20260709 13:06:53.791049 26102 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:53.842059 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 54659.7us
I20260709 13:06:53.793416 26114 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.794094 25294 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:53.798000 26092 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:53.830070 22004 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:53.802987 26088 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:53.842306 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.879212 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.879263 21934 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:53.880131 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.818167 25961 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:06:53.887171 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.831411 25958 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:53.782330 26007 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.888111 21758 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 8, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.825805 25936 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:53.833101 26073 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:53.900245 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.002s	sys 0.009s
W20260709 13:06:53.900982 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.002s	sys 0.009s
W20260709 13:06:53.901476 21900 connection.cc:256] RPC call timeout handler was delayed by -1.4296926519999715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:06:53.902509 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.903107 22004 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:53.889734 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.824692 26138 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:53.904327 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.904584 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.905551 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.906142 21858 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:53.906981 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.907922 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.891032 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.908905 21853 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:53.909686 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.891714 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:53.831080 25973 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.895880 21932 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:06:53.895943 21758 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:06:53.848661 26114 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:53.852815 26101 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:53.881081 25857 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.912876 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:53.888327 26086 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:53.908454 21858 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:53.826100 25523 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.917789 21854 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:53.834035 25582 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.888175 25961 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:53.922322 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:53.890551 26128 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:53.892082 26094 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:53.832612 25991 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.890694 26111 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:53.925940 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.912184 26138 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:53.924002 26028 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:53.928755 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 215098us
I20260709 13:06:53.977536 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:53.902275 26083 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 13:06:53.810844 25732 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.897439 26042 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:53.895346 26100 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:53.893646 26089 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:06:53.967648 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:53.924726 25960 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:53.943663 21758 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 7, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 7 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:54.076054 21758 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:53.969444 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:53.900883 25958 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:53.912413 26105 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:53.911419 25846 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.973249 21858 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:54.014480 25969 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:53.913980 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:53.895681 25729 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:06:53.974642 21852 raft_consensus.cc:2368] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:06:53.892889 26102 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:06:53.923877 25938 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.923807 25928 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.925218 26086 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:54.006819 22556 leader_election.cc:400] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:06:54.107201 22556 leader_election.cc:403] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:54.107992 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 101449us
I20260709 13:06:54.064284 26103 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.938508 26141 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.008427 22005 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:54.005681 26083 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:53.939495 26128 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:53.938434 26094 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:53.979305 26082 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.977s	user 0.012s	sys 0.002s
I20260709 13:06:53.946743 26104 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:54.039091 26143 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:06:54.040266 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:54.041497 26140 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:06:53.974327 26111 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:54.041122 26138 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:54.042092 26139 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:06:53.929191 25857 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:53.978269 26158 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.063230 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:54.064970 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 114113us
I20260709 13:06:54.066223 21854 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:54.180634 21855 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 7.
I20260709 13:06:54.185258 21854 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:54.079443 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 317453us
I20260709 13:06:54.006451 26132 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:54.080768 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:53.973425 22588 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:54.009387 26028 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:54.096920 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:54.101266 21758 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.100087 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:53.939190 25936 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:54.075380 25960 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.938506 25993 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.082553 25752 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:54.087491 26110 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:54.091568 25963 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:54.087831 26105 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:54.098450 26089 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:54.388950 25960 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:54.122552 22556 connection.cc:256] RPC call timeout handler was delayed by -1.8871398649999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260709 13:06:54.172260 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:54.124629 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:54.126564 25991 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:54.173667 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:54.187168 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:54.395918 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 208805us
I20260709 13:06:54.397993 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:54.184362 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:54.132323 26146 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260709 13:06:54.134227 26148 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:06:54.132889 26149 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:06:54.412513 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:54.133507 26145 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:06:54.194247 21929 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14.
I20260709 13:06:54.194554 26093 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:54.284106 26154 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:54.185730 26082 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:54.256543 26157 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:06:54.389170 25993 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:54.227757 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:54.433907 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:54.434255 25993 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:54.405515 25752 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:54.115046 25929 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:06:54.213519 21858 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:54.435158 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:54.221856 26147 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 13:06:54.436018 22584 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:54.093255 25931 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:54.437400 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:54.438014 21933 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:54.438241 25931 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:54.119555 25928 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.213697 25325 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:54.287393 26083 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:54.242688 25958 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:54.134821 26141 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.114384 26101 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:54.193382 26158 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.449518 26101 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.348624 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2355) took 1048 ms. Trace:
I20260709 13:06:54.450410 26141 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:54.450624 22009 rpcz_store.cc:276] 0709 13:06:53.300093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.559661 (+259568us) service_pool.cc:224] Handling call
0709 13:06:53.559935 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.760792 (+200857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.348360 (+587568us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001973d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:54.348561 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332561152}
I20260709 13:06:54.451643 25928 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:54.452159 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:54.333720 26167 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:06:54.358640 26153 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.132s	user 0.000s	sys 0.007s
I20260709 13:06:54.125450 26103 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.455225 21933 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:06:54.413304 25932 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:54.455919 25932 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.456324 25932 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:54.450457 26158 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:54.140667 25618 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.001s
I20260709 13:06:54.340981 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:54.457726 22009 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:54.324942 25523 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:54.364974 25294 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.348145 26054 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.744s	user 0.007s	sys 0.003s
I20260709 13:06:54.369220 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2432) took 1422 ms. Trace:
I20260709 13:06:54.456250 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:54.371497 26166 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.373147 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2357) took 1057 ms. Trace:
W20260709 13:06:54.374315 25950 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.001s
I20260709 13:06:54.290942 26028 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:54.375541 21972 leader_election.cc:336] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.662s (SENT)
W20260709 13:06:54.336337 26130 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.769s	user 0.006s	sys 0.003s
I20260709 13:06:54.377568 26092 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:54.380708 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.229373 26086 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:54.340314 26109 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.000s
I20260709 13:06:54.328015 25857 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:54.347663 25898 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.853s	user 0.009s	sys 0.000s
I20260709 13:06:54.133507 26094 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:54.098959 25846 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:54.179394 26111 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:54.120529 25980 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:54.133315 26128 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:54.110584 25938 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:54.214093 26155 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 13:06:54.391155 22556 connection.cc:256] RPC call timeout handler was delayed by -1.5764208780000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 13:06:54.403987 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:54.407532 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:54.394815 26089 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.431721 26082 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:54.436908 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:54.413162 26174 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:54.383626 26114 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.388953 25936 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.409951 26073 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:54.373792 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:54.351608 21758 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 5, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 5 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.339425 21825 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:54.429746 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:54.434567 21928 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:54.407155 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:54.424528 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:54.373183 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:54.400786 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 216488us
I20260709 13:06:54.429464 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2430) took 1487 ms. Trace:
I20260709 13:06:54.285811 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:54.440361 22006 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:55.080020 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.121628 22005 rpcz_store.cc:276] 0709 13:06:53.316029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.577927 (+261898us) service_pool.cc:224] Handling call
0709 13:06:53.578210 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.113103 (+534893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.372899 (+259796us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b5400437bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:54.373088 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":962385792}
I20260709 13:06:55.123710 21932 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:54.458911 22008 rpcz_store.cc:276] 0709 13:06:52.946424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.489065 (+542641us) service_pool.cc:224] Handling call
0709 13:06:53.489374 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.628238 (+138864us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:53.628269 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.628666 (+   397us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.733054 (+104388us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b4800130158. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:06:54.081064 (+348010us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:54.081189 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.081384 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.368734 (+287350us) raft_consensus.cc:1685] finished
0709 13:06:54.368754 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.369103 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138504,"raft.queue_time_us":141724,"raft.run_cpu_time_us":2163,"raft.run_wall_time_us":96503,"spinlock_wait_cycles":13121920,"thread_start_us":451957,"threads_started":1,"wal-append.queue_time_us":639555}
I20260709 13:06:54.549747 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.066608 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.078822 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:55.126941 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.033s	sys 0.088s
W20260709 13:06:55.127530 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.034s	sys 0.088s
I20260709 13:06:55.132735 21926 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:55.104929 22000 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:55.125203 22009 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:06:55.063354 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.006s	sys 0.021s
W20260709 13:06:55.064283 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.133340 21926 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:55.133070 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2360) took 1473 ms. Trace:
W20260709 13:06:55.215441 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.012s	sys 0.022s
W20260709 13:06:55.225584 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 97911.5us
I20260709 13:06:55.280681 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2348) took 2107 ms. Trace:
I20260709 13:06:54.576292 26105 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.281975 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.227473 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 935) took 1640 ms. Trace:
I20260709 13:06:55.282877 22588 rpcz_store.cc:276] 0709 13:06:53.586970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.588079 (+  1109us) service_pool.cc:224] Handling call
0709 13:06:53.628126 (+ 40047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.196904 (+568778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.226823 (+1029919us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540076fcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:55.227253 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793636096}
I20260709 13:06:55.220636 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2301) took 1944 ms. Trace:
I20260709 13:06:55.229257 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1966) took 1220 ms. Trace:
I20260709 13:06:55.231513 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 742) took 1047 ms. Trace:
I20260709 13:06:55.232679 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1974) took 1107 ms. Trace:
I20260709 13:06:55.270991 22002 rpcz_store.cc:276] 0709 13:06:53.653343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.285554 (+632211us) service_pool.cc:224] Handling call
0709 13:06:55.126431 (+840877us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.234812 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:55.298936 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:54.908241 25973 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.973s	user 0.005s	sys 0.004s
W20260709 13:06:54.821192 25961 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.851s	user 0.010s	sys 0.002s
I20260709 13:06:55.291086 21930 rpcz_store.cc:276] 0709 13:06:54.124969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.424021 (+299052us) service_pool.cc:224] Handling call
0709 13:06:55.232453 (+808432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.289696 21935 rpcz_store.cc:276] 0709 13:06:54.008963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.388195 (+379232us) service_pool.cc:224] Handling call
0709 13:06:55.229032 (+840837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.289455 21928 rpcz_store.cc:276] 0709 13:06:54.183954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.433617 (+249663us) service_pool.cc:224] Handling call
0709 13:06:55.231300 (+797683us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.218688 26088 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.004s	sys 0.013s
I20260709 13:06:55.303519 21927 raft_consensus.cc:2368] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10.
I20260709 13:06:55.295241 25618 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.295266 26101 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:55.304047 26092 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.304441 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:55.216743 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:55.298419 22589 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:54.973810 26181 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
W20260709 13:06:54.547492 26080 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.777s	user 0.006s	sys 0.005s
W20260709 13:06:54.847551 26106 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.029s	user 0.012s	sys 0.000s
W20260709 13:06:54.899607 25960 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.003s
I20260709 13:06:55.280779 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:55.317137 26073 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.317166 25936 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.318266 25938 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.319947 26109 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.322584 26130 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:55.325465 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 918150us
I20260709 13:06:55.326489 25950 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.327972 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:55.331820 26089 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.335353 26114 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.363407 21926 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.218104 25859 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:55.281383 22007 rpcz_store.cc:276] 0709 13:06:53.173469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.495782 (+322313us) service_pool.cc:224] Handling call
0709 13:06:53.496083 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.576424 (+ 80341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.576455 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.602621 (+ 26166us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.904608 (+301987us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:53.904647 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.279588 (+1374941us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b54002152d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:55.279712 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.279785 (+    73us) raft_consensus.cc:1685] finished
0709 13:06:55.279802 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.280309 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182674688,"thread_start_us":228841,"threads_started":1,"wal-append.queue_time_us":475708}
I20260709 13:06:55.376250 22591 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:55.361043 26088 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.004s	sys 0.026s
I20260709 13:06:54.455940 26103 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.391604 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 999) took 1117 ms. Trace:
I20260709 13:06:55.393885 22593 rpcz_store.cc:276] 0709 13:06:54.273947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.715477 (+441530us) service_pool.cc:224] Handling call
0709 13:06:55.387510 (+672033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.288947 21851 rpcz_store.cc:276] 0709 13:06:53.275472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.592658 (+317186us) service_pool.cc:224] Handling call
0709 13:06:53.593058 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:53.641074 (+ 48016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.641105 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:53.667079 (+ 25974us) log.cc:844] Serialized 52 byte log entry
0709 13:06:53.889592 (+222513us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:53.912848 (+ 23256us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.921547 (+  8699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.921617 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.219880 (+1298263us) raft_consensus.cc:1685] finished
0709 13:06:55.219928 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.220281 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79016,"prepare.queue_time_us":22302,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":131361,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":61164,"thread_start_us":164676,"threads_started":1,"wal-append.queue_time_us":275950}
I20260709 13:06:55.400960 22009 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:54.517082 25294 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.383494 26101 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:55.406149 22590 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:55.409142 25980 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:55.409428 21932 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:55.218170 21758 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.407692 25973 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.411638 26080 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.413028 22587 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:55.413334 25859 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:55.414394 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2437) took 1732 ms (client timeout 1779 ms). Trace:
I20260709 13:06:55.418933 26073 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:55.419389 21931 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:55.419067 25960 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:55.419891 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 139514us
I20260709 13:06:55.417776 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:55.218231 22001 rpcz_store.cc:276] 0709 13:06:52.942089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.485032 (+542943us) service_pool.cc:224] Handling call
0709 13:06:53.485398 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.485806 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:53.541113 (+ 55307us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.324934 (+783821us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b180030f5e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:06:54.428979 (+104045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.429085 (+   106us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:54.429152 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.429240 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.429392 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55040,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":7896,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":77,"spinlock_wait_cycles":53634432,"thread_start_us":102363,"threads_started":1,"wal-append.queue_time_us":905272}
I20260709 13:06:55.412135 26101 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.356765 26166 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:55.424137 21929 rpcz_store.cc:269] 0709 13:06:53.682286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.682436 (+   150us) service_pool.cc:224] Handling call
0709 13:06:55.413814 (+1731378us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400206a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:55.414122 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083442944}
I20260709 13:06:54.516794 26054 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:06:55.428004 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 985206us
I20260709 13:06:55.449203 21926 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:55.433559 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 1.065e+06us
I20260709 13:06:55.423663 25980 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.451431 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:54.911509 25932 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.425483 26158 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:55.468158 22592 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:55.462283 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.470811 22585 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14.
I20260709 13:06:55.471698 21933 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:55.455822 25980 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:55.480906 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:55.483915 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.484970 21929 raft_consensus.cc:2368] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:55.485385 26080 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.485946 26109 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.487018 25958 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:55.487552 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:55.487840 25958 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.489677 21859 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:06:55.490139 25938 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:55.490794 25938 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:55.490952 25958 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.491071 26082 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:55.492195 26082 consensus_queue.cc:1243] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:55.492641 26130 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.492827 26158 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:55.493214 22591 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.493410 26158 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:55.493582 26103 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:55.494462 26103 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.494953 26158 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.466034 25857 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:55.483974 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:55.498364 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2431) took 1827 ms (client timeout 1581 ms). Trace:
I20260709 13:06:55.498739 25973 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.499068 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:55.498835 21927 rpcz_store.cc:269] 0709 13:06:53.671448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.674325 (+  2877us) service_pool.cc:224] Handling call
0709 13:06:55.498095 (+1823770us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540042a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:55.498285 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3624139520}
I20260709 13:06:55.500533 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.501454 26132 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:55.520280 26158 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:55.506496 26078 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.528699 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:55.529032 26078 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.530539 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2356) took 2222 ms. Trace:
I20260709 13:06:55.531046 22000 rpcz_store.cc:276] 0709 13:06:53.307864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.570117 (+262253us) service_pool.cc:224] Handling call
0709 13:06:53.570477 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.218016 (+1647539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.530203 (+312187us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540033e5d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:55.530456 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3526931200}
I20260709 13:06:55.532562 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:55.533506 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.479687 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:55.510309 26188 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
I20260709 13:06:55.510177 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.510895 25961 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.598269 25582 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.600226 25961 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.463388 25936 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:55.600720 22585 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 932) took 2014 ms (client timeout 1779 ms). Trace:
W20260709 13:06:55.601256 22585 rpcz_store.cc:269] 0709 13:06:53.585938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.591441 (+  5503us) service_pool.cc:224] Handling call
0709 13:06:55.600413 (+2008972us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54006009d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:55.600622 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4160406400}
I20260709 13:06:55.602584 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:55.485450 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1978) took 1228 ms. Trace:
I20260709 13:06:55.605144 21929 rpcz_store.cc:276] 0709 13:06:54.256706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.471351 (+1214645us) service_pool.cc:224] Handling call
0709 13:06:55.485358 (+ 14007us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.606153 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:55.606590 26105 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.606901 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 741) took 1423 ms. Trace:
W20260709 13:06:55.505874 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2006) took 2541 ms (client timeout 2785 ms). Trace:
I20260709 13:06:55.607481 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:55.607690 21852 rpcz_store.cc:269] 0709 13:06:52.965573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.150162 (+184589us) service_pool.cc:224] Handling call
0709 13:06:55.505415 (+2355253us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54000591d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:55.505674 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314115840}
I20260709 13:06:55.521275 21927 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:55.608803 26109 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:55.609396 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1968) took 1545 ms (client timeout 2008 ms). Trace:
W20260709 13:06:55.609781 21927 rpcz_store.cc:269] 0709 13:06:54.063730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.500168 (+1436438us) service_pool.cc:224] Handling call
0709 13:06:55.609301 (+109133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.610836 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.611359 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:55.611689 21927 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:55.612731 21758 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.615505 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:55.615223 21758 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 3 to 6, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 6 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.616942 26192 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:55.588428 26073 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.617991 26192 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.619984 26073 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.620654 26192 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.479348 26174 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.551213 25938 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.532724 26078 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.509387 26130 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:55.536474 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 794) took 1865 ms. Trace:
I20260709 13:06:55.539444 26080 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:55.540292 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 72038.2us
I20260709 13:06:55.520898 22588 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:55.538506 26100 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.553236 26054 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.551609 25859 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:55.537449 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 120652us
I20260709 13:06:55.402295 26106 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.556695 22593 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:55.499533 26103 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.628612 22584 rpcz_store.cc:276] 0709 13:06:53.670602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.129323 (+1458721us) service_pool.cc:224] Handling call
0709 13:06:55.130364 (+  1041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.131570 (+  1206us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:55.131612 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.132718 (+  1106us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.455979 (+323261us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:55.456024 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.456217 (+   193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.536113 (+ 79896us) raft_consensus.cc:1685] finished
0709 13:06:55.536133 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.536262 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83847,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":328,"thread_start_us":318354,"threads_started":1,"wal-append.queue_time_us":401135}
I20260709 13:06:55.546249 25846 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.521878 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:55.549286 25960 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.591831 22591 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:06:55.586071 21926 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:55.639357 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1960) took 1633 ms (client timeout 1817 ms). Trace:
W20260709 13:06:55.592960 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.21732e+06us
I20260709 13:06:55.595715 21930 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:06:55.596369 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 205871us
I20260709 13:06:55.597688 21825 leader_election.cc:400] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12.
I20260709 13:06:55.640586 21825 leader_election.cc:403] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:06:55.525482 21928 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:55.527922 22589 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:55.607332 21929 rpcz_store.cc:276] 0709 13:06:54.183803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.605844 (+1422041us) service_pool.cc:224] Handling call
0709 13:06:55.606827 (+   983us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.639742 21926 rpcz_store.cc:269] 0709 13:06:54.006804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.078091 (+1071287us) service_pool.cc:224] Handling call
0709 13:06:55.639295 (+561204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.612262 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 732) took 1430 ms. Trace:
W20260709 13:06:55.612277 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1977) took 1390 ms (client timeout 1638 ms). Trace:
I20260709 13:06:55.642382 21927 rpcz_store.cc:276] 0709 13:06:54.182348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.610418 (+1428070us) service_pool.cc:224] Handling call
0709 13:06:55.612205 (+  1787us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.642520 21935 rpcz_store.cc:269] 0709 13:06:54.221635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.611017 (+1389382us) service_pool.cc:224] Handling call
0709 13:06:55.612225 (+  1208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.609405 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1976) took 1435 ms (client timeout 1674 ms). Trace:
I20260709 13:06:55.553243 25294 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.628095 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2447) took 1141 ms. Trace:
I20260709 13:06:55.624089 26162 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:55.550311 25618 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.647792 22000 rpcz_store.cc:276] 0709 13:06:54.484696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.532171 (+1047475us) service_pool.cc:224] Handling call
0709 13:06:55.625476 (+ 93305us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.643469 21932 rpcz_store.cc:269] 0709 13:06:54.174086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.281472 (+1107386us) service_pool.cc:224] Handling call
0709 13:06:55.609171 (+327699us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540003cd54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:55.609300 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279599872}
W20260709 13:06:55.648880 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1971) took 1554 ms (client timeout 1807 ms). Trace:
W20260709 13:06:55.654469 21930 rpcz_store.cc:269] 0709 13:06:54.086532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.422095 (+1335563us) service_pool.cc:224] Handling call
0709 13:06:55.640322 (+218227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.649400 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1954) took 1668 ms (client timeout 1881 ms). Trace:
W20260709 13:06:55.656793 21928 rpcz_store.cc:269] 0709 13:06:53.977899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.403115 (+1425216us) service_pool.cc:224] Handling call
0709 13:06:55.646097 (+242982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.644397 21758 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:06:55.653355 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:55.664925 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.661588 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.662168 22002 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:06:55.672715 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 223097us
I20260709 13:06:55.673548 26092 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.673834 26110 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:55.674448 26110 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:55.676200 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:55.677273 25973 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:55.677737 25973 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.678171 25973 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:55.680599 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.697507 21926 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:55.698024 22006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 857) took 2078 ms (client timeout 1886 ms). Trace:
W20260709 13:06:55.698191 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1975) took 1524 ms (client timeout 1942 ms). Trace:
W20260709 13:06:55.698711 21926 rpcz_store.cc:269] 0709 13:06:54.173736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.653192 (+1479456us) service_pool.cc:224] Handling call
0709 13:06:55.698093 (+ 44901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.699492 22584 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:55.700148 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 798) took 2028 ms. Trace:
I20260709 13:06:55.700084 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.700744 22584 rpcz_store.cc:276] 0709 13:06:53.671790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.645043 (+1973253us) service_pool.cc:224] Handling call
0709 13:06:55.699314 (+ 54271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.699898 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.700079 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.701390 25863 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:06:55.682087 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48476.5us
I20260709 13:06:55.624092 26174 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.685129 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 939) took 2081 ms. Trace:
I20260709 13:06:55.686976 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:55.688282 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 90900.4us
I20260709 13:06:55.698853 26100 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.688871 25846 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.691447 25973 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.692814 25898 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:55.693230 26135 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:55.694046 26190 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:55.694803 26133 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.794s	user 0.000s	sys 0.012s
I20260709 13:06:55.672729 22593 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:55.659855 26130 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:55.702697 22005 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:55.703296 22589 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:55.738399 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 738) took 1555 ms. Trace:
I20260709 13:06:55.738744 22591 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:55.713325 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:55.740033 22009 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:55.728196 25898 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:06:55.741482 22009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2362) took 2081 ms (client timeout 2477 ms). Trace:
I20260709 13:06:55.742017 21930 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:55.707507 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:55.739020 21926 rpcz_store.cc:276] 0709 13:06:54.183387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.699703 (+1516316us) service_pool.cc:224] Handling call
0709 13:06:55.738305 (+ 38602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.706235 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:55.706605 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:55.732733 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 997) took 1459 ms. Trace:
W20260709 13:06:55.745141 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 737) took 1561 ms (client timeout 2013 ms). Trace:
I20260709 13:06:55.726965 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:55.745857 21930 rpcz_store.cc:269] 0709 13:06:54.183246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.660634 (+1477388us) service_pool.cc:224] Handling call
0709 13:06:55.745078 (+ 84444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.723743 21825 leader_election.cc:400] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14.
I20260709 13:06:55.747320 21825 leader_election.cc:403] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:06:55.745262 22593 rpcz_store.cc:276] 0709 13:06:54.273264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.656292 (+1383028us) service_pool.cc:224] Handling call
0709 13:06:55.670672 (+ 14380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.732365 (+ 61693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.732632 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.708024 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:55.728042 25973 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:55.803125 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 76390.7us
I20260709 13:06:55.711957 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:55.806242 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 82790.7us
I20260709 13:06:55.702967 25732 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.809191 25582 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:55.809473 25732 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.810323 25857 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:55.812139 25732 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.813537 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:55.816038 26089 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.721189 22587 rpcz_store.cc:276] 0709 13:06:53.603697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.604209 (+   512us) service_pool.cc:224] Handling call
0709 13:06:53.631611 (+ 27402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.449223 (+1817612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.684753 (+235530us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54003791d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:55.685051 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4064183040}
I20260709 13:06:55.808701 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:55.705480 26162 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:55.818033 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:55.818740 26162 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.820181 25868 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:55.822587 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:55.824052 22000 raft_consensus.cc:2410] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:55.824563 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2363) took 2162 ms. Trace:
I20260709 13:06:55.824352 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:55.698452 22006 rpcz_store.cc:269] 0709 13:06:53.619152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.630156 (+ 11004us) service_pool.cc:224] Handling call
0709 13:06:55.697651 (+2067495us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54001e1ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:55.697819 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785752960}
I20260709 13:06:55.825320 21932 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:55.825779 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 139527us
I20260709 13:06:55.826609 22584 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:55.825031 22000 rpcz_store.cc:276] 0709 13:06:53.661783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.657901 (+1996118us) service_pool.cc:224] Handling call
0709 13:06:55.824468 (+166567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.809844 25582 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:55.741956 22009 rpcz_store.cc:269] 0709 13:06:53.659685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.533104 (+1873419us) service_pool.cc:224] Handling call
0709 13:06:55.741395 (+208291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.724761 25950 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:55.815908 25898 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:55.849874 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.704605 26102 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:55.701886 25863 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:55.854924 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 150000us
I20260709 13:06:55.856855 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.856505 25582 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.857331 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.859546 26089 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.710697 25915 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:55.863466 21929 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:55.863675 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1979) took 1591 ms (client timeout 1982 ms). Trace:
W20260709 13:06:55.945271 21928 rpcz_store.cc:269] 0709 13:06:54.272209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.707578 (+1435369us) service_pool.cc:224] Handling call
0709 13:06:55.863614 (+156036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.713330 26192 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:55.724210 26100 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.713992 26166 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.721698 26174 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.748589 25936 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:55.937825 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 230544us
I20260709 13:06:55.714711 25932 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:55.948748 26092 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:55.725930 25859 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.224s
I20260709 13:06:55.854730 26132 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:55.950640 22006 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:55.705256 26105 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.246s
I20260709 13:06:55.731875 26106 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:55.957454 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:55.958107 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:55.960439 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 284581us
W20260709 13:06:55.969272 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 733) took 1786 ms (client timeout 2199 ms). Trace:
W20260709 13:06:55.969583 21926 rpcz_store.cc:269] 0709 13:06:54.182621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.832959 (+1650338us) service_pool.cc:224] Handling call
0709 13:06:55.969216 (+136257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.708477 26111 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:55.966032 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:55.725697 26054 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.212s
W20260709 13:06:55.967115 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.972198 22005 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:55.885011 22586 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
I20260709 13:06:55.736436 26130 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:55.972057 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:55.865084 26078 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:06:55.983074 21932 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
W20260709 13:06:55.983670 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 735) took 1800 ms (client timeout 2010 ms). Trace:
W20260709 13:06:55.983990 21932 rpcz_store.cc:269] 0709 13:06:54.182950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.652708 (+1469758us) service_pool.cc:224] Handling call
0709 13:06:55.983619 (+330911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.979884 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:55.980868 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:55.985208 22002 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
W20260709 13:06:55.986068 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.855255 26158 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:55.982661 22005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2367) took 1595 ms (client timeout 2113 ms). Trace:
I20260709 13:06:55.965437 25915 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:55.977833 25898 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:55.987556 22585 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:55.992317 22005 rpcz_store.cc:269] 0709 13:06:54.383097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.628640 (+1245543us) service_pool.cc:224] Handling call
0709 13:06:55.977468 (+348828us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.998955 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 133662us
W20260709 13:06:56.002285 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1973) took 1886 ms (client timeout 1711 ms). Trace:
I20260709 13:06:56.002535 21930 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:06:56.003046 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1972) took 1893 ms (client timeout 2522 ms). Trace:
I20260709 13:06:56.003686 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.004544 22005 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:06:56.005155 26094 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.002s
I20260709 13:06:56.005426 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.005586 26094 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:56.002754 21933 rpcz_store.cc:269] 0709 13:06:54.116191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.461067 (+344876us) service_pool.cc:224] Handling call
0709 13:06:55.998565 (+1537498us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54002e81d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:56.002214 (+  3649us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789926144}
I20260709 13:06:56.007014 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.007820 21933 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:56.007905 26094 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:56.008190 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1970) took 1927 ms. Trace:
I20260709 13:06:56.006036 22005 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:56.008560 21933 rpcz_store.cc:276] 0709 13:06:54.080932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.006712 (+1925780us) service_pool.cc:224] Handling call
0709 13:06:56.008136 (+  1424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.009490 22004 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:56.010707 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.012109 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.012529 26128 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.015185 26128 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:56.021127 26054 consensus_queue.cc:1243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:56.022145 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.023074 21928 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:06:56.023480 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1958) took 2044 ms (client timeout 2654 ms). Trace:
W20260709 13:06:56.023846 21928 rpcz_store.cc:269] 0709 13:06:53.979008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.959664 (+1980656us) service_pool.cc:224] Handling call
0709 13:06:56.023424 (+ 63760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.026012 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:56.021127 26020 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.027873 26020 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.028790 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:55.976365 26102 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.031527 26102 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:56.032491 26102 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:06:56.033785 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.034500 21851 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.034761 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.035459 21853 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:56.037058 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.037410 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.038787 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.038836 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.039590 21852 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:06:56.040362 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.042062 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:56.003475 21930 rpcz_store.cc:269] 0709 13:06:54.109627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.845920 (+1736293us) service_pool.cc:224] Handling call
0709 13:06:56.002993 (+157073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:55.997179 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.024838 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:55.985713 22002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2440) took 2370 ms (client timeout 2492 ms). Trace:
I20260709 13:06:56.043633 21928 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.044595 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:55.980618 26092 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:55.980731 26192 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:56.047313 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 740) took 1863 ms. Trace:
I20260709 13:06:56.047847 21932 rpcz_store.cc:276] 0709 13:06:54.183669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.984493 (+1800824us) service_pool.cc:224] Handling call
0709 13:06:56.047216 (+ 62723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.050252 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:56.032871 26102 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:56.053095 22000 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:55.968531 25980 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.054144 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:06:56.043601 22002 rpcz_store.cc:269] 0709 13:06:53.615038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.480094 (+1865056us) service_pool.cc:224] Handling call
0709 13:06:55.985655 (+505561us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.040833 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.008749 26094 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:56.058089 22006 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:06:55.993732 25936 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.058787 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2012) took 2817 ms. Trace:
I20260709 13:06:56.059172 21859 rpcz_store.cc:276] 0709 13:06:53.240955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.352241 (+111286us) service_pool.cc:224] Handling call
0709 13:06:53.352512 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.490206 (+2137694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.049524 (+559318us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b5400259c54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:56.058685 (+  9161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4876539648}
I20260709 13:06:56.060626 25936 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:56.061398 21857 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:06:56.061720 26192 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.062590 21854 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:56.064218 25980 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:56.064556 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.064670 22588 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 7. Current term is 9. Ops: []
I20260709 13:06:56.065056 26088 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:56.066190 21855 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:06:56.069432 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:56.069741 25846 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12.
I20260709 13:06:56.070343 21926 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:06:56.089394 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1969) took 2014 ms. Trace:
I20260709 13:06:56.089830 21926 rpcz_store.cc:276] 0709 13:06:54.075546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.981807 (+1906261us) service_pool.cc:224] Handling call
0709 13:06:56.089324 (+107517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.090776 26111 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:56.055091 26092 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:56.093874 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.093492 25915 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.095134 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.095592 26111 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.096200 26111 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.098717 26111 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:56.098980 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:56.100250 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:56.100503 21853 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:56.101743 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.102669 26205 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:56.102865 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.103148 26205 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:56.103524 21855 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:56.103592 26205 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.104400 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.078073 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.022157 25859 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:56.090806 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.056134 25857 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14.
W20260709 13:06:56.126287 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.127069 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.128095 21853 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:56.148535 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.154554 21930 raft_consensus.cc:2368] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:06:56.160310 21932 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:56.161545 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.164539 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.166115 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:56.177201 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.112564 25859 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:56.189527 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 854) took 2553 ms. Trace:
W20260709 13:06:56.165660 21823 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.581s (SENT)
I20260709 13:06:56.145744 26110 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:56.374933 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.474411 21928 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:06:56.258066 21926 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:56.383749 21858 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:06:56.317868 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.208240 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:56.317410 22006 raft_consensus.cc:2368] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13.
I20260709 13:06:56.394568 21854 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:56.276085 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.204612 22009 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:06:56.479033 21857 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
W20260709 13:06:56.391278 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 200983us
W20260709 13:06:56.409083 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 261606us
I20260709 13:06:56.190054 22002 rpcz_store.cc:276] 0709 13:06:53.615706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.057638 (+2441932us) service_pool.cc:224] Handling call
0709 13:06:56.114366 (+ 56728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.168974 (+ 54608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.169016 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.169054 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:56.169082 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.169132 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.169227 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.394879 21852 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:56.481575 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 412228us
I20260709 13:06:56.484966 21851 raft_consensus.cc:2410] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:56.486135 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2438) took 3087 ms. Trace:
I20260709 13:06:56.486601 22009 rpcz_store.cc:276] 0709 13:06:53.391955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.046575 (+2654620us) service_pool.cc:224] Handling call
0709 13:06:56.145353 (+ 98778us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:56.478862 (+333509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.479091 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.487674 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:56.489624 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.490255 21930 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:56.491269 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.492128 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.493299 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 936) took 2906 ms. Trace:
I20260709 13:06:56.493970 21853 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:56.493680 22590 rpcz_store.cc:276] 0709 13:06:53.587311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.588169 (+   858us) service_pool.cc:224] Handling call
0709 13:06:53.588523 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.425306 (+1836783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.483815 (+1058509us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b540060d6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:56.493222 (+  9407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4156217600}
I20260709 13:06:56.492868 22007 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:56.495769 21853 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:56.497890 25857 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:56.499444 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:56.496618 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:56.502070 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.502786 22007 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:56.503657 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.505251 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:56.505805 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.507133 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:56.507230 26020 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:56.505929 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.508255 22007 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
I20260709 13:06:56.509259 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.510483 22007 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:56.511346 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.512153 22002 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:56.513404 21853 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:06:56.518123 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.241278 26194 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:56.169528 26094 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:56.522738 21932 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:56.523204 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.300882 26073 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:56.525620 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.149250 26088 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.534744 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:56.537175 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:56.538065 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:56.540959 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:56.541385 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:56.542258 22589 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:56.542994 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:06:56.544519 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.544749 26086 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:56.543108 25857 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.394039 25863 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.523857 25846 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.522558 26194 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:56.545149 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:56.516284 25915 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:56.555655 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 401867us
I20260709 13:06:56.553987 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:56.534740 26020 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.565864 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:56.554148 25582 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.566798 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:56.568550 25857 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:56.590939 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.605062 21926 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:56.604645 22007 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:56.607208 22009 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.523628 26207 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:06:56.608233 22593 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:56.609650 22002 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:56.559506 21928 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:56.612862 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:56.617686 22587 raft_consensus.cc:2368] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:56.561730 26194 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.637590 21932 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:56.640990 22585 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:06:56.641556 26209 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:06:56.641712 22007 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:56.642813 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2442) took 2793 ms. Trace:
I20260709 13:06:56.650199 22008 rpcz_store.cc:276] 0709 13:06:53.849176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.641871 (+2792695us) service_pool.cc:224] Handling call
0709 13:06:56.642269 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.642463 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.642487 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.642533 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:56.642566 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.642626 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.642740 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.651278 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1955) took 2661 ms. Trace:
I20260709 13:06:56.647404 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1965) took 2638 ms. Trace:
I20260709 13:06:56.648105 21930 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:56.648389 26207 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:56.651782 21926 rpcz_store.cc:276] 0709 13:06:53.978197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.484075 (+2505878us) service_pool.cc:224] Handling call
0709 13:06:56.539892 (+ 55817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.639406 (+ 99514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.639630 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.652227 21928 rpcz_store.cc:276] 0709 13:06:54.008689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.484100 (+2475411us) service_pool.cc:224] Handling call
0709 13:06:56.647346 (+163246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.655359 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:56.656503 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2359) took 3003 ms. Trace:
I20260709 13:06:56.657020 22007 rpcz_store.cc:276] 0709 13:06:53.648014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.641174 (+2993160us) service_pool.cc:224] Handling call
0709 13:06:56.641537 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.651533 (+  9996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.651727 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.660046 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.660566 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1961) took 2653 ms (client timeout 2232 ms). Trace:
I20260709 13:06:56.638973 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:06:56.661216 21929 rpcz_store.cc:269] 0709 13:06:54.007496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.671891 (+1664395us) service_pool.cc:224] Handling call
0709 13:06:56.660317 (+988426us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400228654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:56.660499 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326103936}
I20260709 13:06:56.606319 25846 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.663657 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.665218 22006 pending_rounds.cc:85] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
W20260709 13:06:56.660461 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2430) took 2989 ms (client timeout 2343 ms). Trace:
I20260709 13:06:56.666771 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:56.672276 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 114935us
W20260709 13:06:56.668257 21931 rpcz_store.cc:269] 0709 13:06:53.671085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.672350 (+  1265us) service_pool.cc:224] Handling call
0709 13:06:56.660158 (+2987808us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54003552d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:56.660338 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3832600320}
I20260709 13:06:56.648190 26094 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.605949 25859 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.681833 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.668196 22006 pending_rounds.cc:107] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:06:56.684235 22006 raft_consensus.cc:2889] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:56.653352 26207 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:56.695241 22007 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:56.649178 25915 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.689218 21932 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.705713 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 859) took 2351 ms. Trace:
I20260709 13:06:56.706257 22007 rpcz_store.cc:276] 0709 13:06:54.344449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.667110 (+2322661us) service_pool.cc:224] Handling call
0709 13:06:56.695009 (+ 27899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.695857 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.696105 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.678670 25898 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.000s
I20260709 13:06:56.666978 26073 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:56.702363 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2439) took 3081 ms. Trace:
I20260709 13:06:56.702777 21931 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:56.654094 26020 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:56.703648 26213 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:06:56.674580 26130 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:56.677512 26092 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:56.721519 22590 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:06:56.725418 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 178748us
I20260709 13:06:56.718688 22000 rpcz_store.cc:276] 0709 13:06:53.614729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.057337 (+2442608us) service_pool.cc:224] Handling call
0709 13:06:56.070612 (+ 13275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.071142 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.071181 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.071779 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.090106 (+ 18327us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:56.090148 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.111210 (+ 21062us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.696302 (+585092us) raft_consensus.cc:1685] finished
0709 13:06:56.696320 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.696491 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1293,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":17928,"threads_started":1,"wal-append.queue_time_us":321430}
I20260709 13:06:56.726238 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1963) took 2714 ms. Trace:
I20260709 13:06:56.726694 21931 rpcz_store.cc:276] 0709 13:06:54.008115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.683157 (+2675042us) service_pool.cc:224] Handling call
0709 13:06:56.702597 (+ 19440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.720264 (+ 17667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.722714 (+  2450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.727411 22000 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:56.728018 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 862) took 2376 ms. Trace:
I20260709 13:06:56.728328 22000 rpcz_store.cc:276] 0709 13:06:54.351496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.726743 (+2375247us) service_pool.cc:224] Handling call
0709 13:06:56.727083 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.727759 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.727951 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.733088 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.742862 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 860) took 2398 ms. Trace:
I20260709 13:06:56.743434 22007 rpcz_store.cc:276] 0709 13:06:54.344748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.718470 (+2373722us) service_pool.cc:224] Handling call
0709 13:06:56.742217 (+ 23747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.742451 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.742487 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.742542 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:56.742584 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.742654 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.742790 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.676702 25582 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.753645 21932 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:56.731592 26092 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.730901 26110 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:56.704046 26174 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:06:56.775666 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2449) took 2427 ms. Trace:
I20260709 13:06:56.773564 26100 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:56.778964 21926 rpcz_store.cc:276] 0709 13:06:54.340045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.692464 (+2352419us) service_pool.cc:224] Handling call
0709 13:06:56.729744 (+ 37280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.767089 (+ 37345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.767140 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.767203 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:56.767251 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.767337 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.767503 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.741120 26130 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:56.756487 26088 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:56.780583 26100 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:56.792788 26100 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.793957 26100 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.797817 25846 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:56.799121 21926 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:56.801195 21926 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:06:56.783231 26207 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.802155 26207 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:56.771275 26073 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:56.806219 25582 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:56.807410 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:56.807658 26158 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:56.808372 25932 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:56.807427 26205 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.703s	user 0.010s	sys 0.003s
W20260709 13:06:56.809950 26206 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:06:56.809845 26205 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:56.811522 26212 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:06:56.813028 26205 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:56.806108 26073 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:56.813829 26073 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:56.814850 26073 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:56.773571 26092 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.816963 26073 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:56.821408 26210 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:06:56.829166 25863 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:56.831255 25915 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:56.832476 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:56.833300 22589 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:56.833398 22009 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:56.813086 21926 pending_rounds.cc:85] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 10
I20260709 13:06:56.834502 21926 pending_rounds.cc:107] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:06:56.834843 21926 raft_consensus.cc:2889] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:56.809319 26211 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:06:56.839334 21932 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:56.840740 22002 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:56.843415 22589 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:56.850267 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:56.832480 25857 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:56.879976 21856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2304) took 2988 ms (client timeout 2477 ms). Trace:
W20260709 13:06:56.880623 21856 rpcz_store.cc:269] 0709 13:06:53.891050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.891472 (+   422us) service_pool.cc:224] Handling call
0709 13:06:56.838078 (+2946606us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540018f6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:56.879570 (+ 41492us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653836032}
I20260709 13:06:56.882756 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2368) took 2495 ms. Trace:
W20260709 13:06:56.880875 21972 connection.cc:256] RPC call timeout handler was delayed by -1.7175529919999235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:06:56.883577 21972 connection.cc:256] RPC call timeout handler was delayed by -1.7166529919999221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:06:56.883932 21972 connection.cc:256] RPC call timeout handler was delayed by -1.6959529920000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
W20260709 13:06:56.884209 21972 connection.cc:256] RPC call timeout handler was delayed by -1.5861529919999384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
W20260709 13:06:56.884507 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2387498099999448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 13:06:56.884838 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2126529920000166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
W20260709 13:06:56.885180 21972 connection.cc:256] RPC call timeout handler was delayed by -1.0578498099999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 13:06:56.884603 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 379513us
I20260709 13:06:56.844064 21928 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:06:56.889367 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 238683us
I20260709 13:06:56.883229 22007 rpcz_store.cc:276] 0709 13:06:54.387664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.771907 (+2384243us) service_pool.cc:224] Handling call
0709 13:06:56.785714 (+ 13807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.843970 (+ 58256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.844001 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.882502 (+ 38501us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:56.882541 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.882598 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.882705 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.897596 22587 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:56.899196 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:56.894605 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:56.907464 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:56.900233 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 291380us
W20260709 13:06:56.909596 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 184995us
I20260709 13:06:56.907013 22585 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:56.909317 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 111421us
W20260709 13:06:56.911253 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 102479us
I20260709 13:06:56.915963 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2446) took 2456 ms. Trace:
I20260709 13:06:56.916417 22002 rpcz_store.cc:276] 0709 13:06:54.459471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.842799 (+2383328us) service_pool.cc:224] Handling call
0709 13:06:56.893680 (+ 50881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.901364 (+  7684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.901398 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.907242 (+  5844us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:56.915716 (+  8474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.915788 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.915907 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.916141 22589 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:56.920413 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.30024e+06us
W20260709 13:06:56.925473 22004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2361) took 3269 ms (client timeout 1924 ms). Trace:
W20260709 13:06:56.926041 22004 rpcz_store.cc:269] 0709 13:06:53.656152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.933935 (+277783us) service_pool.cc:224] Handling call
0709 13:06:56.925159 (+2991224us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b5400167954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:56.925363 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4441839488}
I20260709 13:06:56.928051 22004 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:56.930027 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.931207 21932 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 12 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:56.932585 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.933585 21926 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:56.935187 21926 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:56.938162 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 805) took 1655 ms. Trace:
I20260709 13:06:56.938855 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2316) took 1116 ms. Trace:
I20260709 13:06:56.939397 21860 rpcz_store.cc:276] 0709 13:06:55.822035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.822172 (+   137us) service_pool.cc:224] Handling call
0709 13:06:56.938586 (+1116414us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b540025ab54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:56.938756 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755112320}
I20260709 13:06:56.938689 22584 rpcz_store.cc:276] 0709 13:06:55.282771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.703088 (+420317us) service_pool.cc:224] Handling call
0709 13:06:55.703360 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.826998 (+123638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.937890 (+1110892us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54006630d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:56.938085 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282644736}
I20260709 13:06:56.933629 21932 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:56.928761 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 866) took 1247 ms. Trace:
I20260709 13:06:56.949028 22004 rpcz_store.cc:276] 0709 13:06:55.681884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.927191 (+1245307us) service_pool.cc:224] Handling call
0709 13:06:56.927589 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.928462 (+   873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.928697 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.949762 22007 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:56.950403 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2448) took 2405 ms. Trace:
I20260709 13:06:56.950814 22007 rpcz_store.cc:276] 0709 13:06:54.545400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.893497 (+2348097us) service_pool.cc:224] Handling call
0709 13:06:56.945036 (+ 51539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.950166 (+  5130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.950348 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.952937 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.958127 22004 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:56.961668 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2449) took 1135 ms. Trace:
W20260709 13:06:56.961720 22003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 855) took 3345 ms (client timeout 2901 ms). Trace:
I20260709 13:06:56.962637 22007 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:56.961930 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 863) took 1458 ms. Trace:
I20260709 13:06:56.963246 22004 rpcz_store.cc:276] 0709 13:06:55.826061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.957151 (+1131090us) service_pool.cc:224] Handling call
0709 13:06:56.957456 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.958780 (+  1324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.961602 (+  2822us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:56.964293 22002 rpcz_store.cc:276] 0709 13:06:55.504007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.945043 (+1441036us) service_pool.cc:224] Handling call
0709 13:06:56.961332 (+ 16289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.961625 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.961656 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.961702 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:56.961730 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.961790 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.961879 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:56.964819 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 739) took 2781 ms (client timeout 1967 ms). Trace:
W20260709 13:06:56.965546 21935 rpcz_store.cc:269] 0709 13:06:54.183523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.955806 (+2772283us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:56.964757 (+  8951us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:06:56.962451 22003 rpcz_store.cc:269] 0709 13:06:53.616005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.958000 (+3341995us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:56.961519 (+  3519us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:06:56.969265 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76088.5us
I20260709 13:06:56.968508 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:56.973145 21927 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.982223 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:56.992611 21926 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:56.901757 26130 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:56.995898 22007 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:56.997159 22002 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:56.998926 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2365) took 2623 ms. Trace:
I20260709 13:06:57.000669 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.005376 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.006299 21932 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:06:56.999317 22000 rpcz_store.cc:276] 0709 13:06:54.375313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.730755 (+2355442us) service_pool.cc:224] Handling call
0709 13:06:56.777453 (+ 46698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.790357 (+ 12904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.790402 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.843265 (+ 52863us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.960347 (+117082us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:56.960386 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.960448 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.998734 (+ 38286us) raft_consensus.cc:1685] finished
0709 13:06:56.998752 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.998870 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9728,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":80349,"threads_started":1,"wal-append.queue_time_us":112922}
W20260709 13:06:57.011323 22556 connection.cc:256] RPC call timeout handler was delayed by -1.3728488329999209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 13:06:57.017777 21935 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:57.019778 21932 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:56.979585 26020 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:06:57.010522 22592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 793) took 3331 ms (client timeout 2766 ms). Trace:
I20260709 13:06:57.021337 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:57.023401 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2444) took 2590 ms. Trace:
I20260709 13:06:57.023733 22009 rpcz_store.cc:276] 0709 13:06:54.429065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.834255 (+2405190us) service_pool.cc:224] Handling call
0709 13:06:56.882107 (+ 47852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.887683 (+  5576us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:56.919853 (+ 32170us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.019146 (+ 99293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.019199 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:57.019254 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.019322 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.019455 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5753,"prepare.queue_time_us":73484,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":30,"thread_start_us":93805,"threads_started":2}
W20260709 13:06:57.021093 22592 rpcz_store.cc:269] 0709 13:06:53.670289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.171997 (+501708us) service_pool.cc:224] Handling call
0709 13:06:56.978758 (+2806761us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54004c22d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.001216 (+ 22458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2431418624}
I20260709 13:06:56.955541 26192 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:06:57.025985 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2443) took 3014 ms. Trace:
I20260709 13:06:57.027134 21933 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:57.027442 22008 rpcz_store.cc:276] 0709 13:06:54.011168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.651366 (+2640198us) service_pool.cc:224] Handling call
0709 13:06:56.676044 (+ 24678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.691161 (+ 15117us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:56.890755 (+199594us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.025660 (+134905us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.025714 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:57.025775 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.025836 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.025910 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2124,"prepare.queue_time_us":260885,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"thread_start_us":249740,"threads_started":2}
I20260709 13:06:57.028810 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:56.975353 25863 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:06:57.017112 26105 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.032142 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.026505 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:57.039042 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 856) took 3420 ms. Trace:
I20260709 13:06:57.038698 26222 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:06:57.039793 22006 rpcz_store.cc:276] 0709 13:06:53.618844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.611624 (+2992780us) service_pool.cc:224] Handling call
0709 13:06:56.650215 (+ 38591us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:56.901179 (+250964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.901208 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:56.916515 (+ 15307us) log.cc:844] Serialized 52 byte log entry
0709 13:06:57.015447 (+ 98932us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:57.015489 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.015558 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.038752 (+ 23194us) raft_consensus.cc:1685] finished
0709 13:06:57.038777 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.038974 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80444,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":9654,"thread_start_us":224649,"threads_started":2,"wal-append.queue_time_us":79772}
I20260709 13:06:57.042922 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1962) took 3035 ms. Trace:
I20260709 13:06:57.043236 26138 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:57.043355 21930 rpcz_store.cc:276] 0709 13:06:54.007809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.492495 (+2484686us) service_pool.cc:224] Handling call
0709 13:06:56.646987 (+154492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.653022 (+  6035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.042695 (+389673us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54004295d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.042832 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354369792}
I20260709 13:06:57.045022 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 795) took 3374 ms. Trace:
I20260709 13:06:57.025322 26110 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:57.045404 22588 rpcz_store.cc:276] 0709 13:06:53.670941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.637748 (+1966807us) service_pool.cc:224] Handling call
0709 13:06:55.638200 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.081684 (+443484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.044750 (+963066us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400668354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:57.044936 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980299648}
I20260709 13:06:57.051608 25863 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:57.052191 25863 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:57.010321 25857 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:06:57.053428 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.053807 21927 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:06:57.031226 26020 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:57.053812 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.057534 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1001) took 2783 ms. Trace:
W20260709 13:06:57.059033 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.059964 22556 connection.cc:256] RPC call timeout handler was delayed by -1.114548832999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:06:57.060352 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1001488329999347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
W20260709 13:06:57.060631 22556 connection.cc:256] RPC call timeout handler was delayed by -1.1001488329999347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
W20260709 13:06:57.060881 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0920488329999216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 13:06:57.061208 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0920488329999216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
I20260709 13:06:57.057965 22591 rpcz_store.cc:276] 0709 13:06:54.274466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.645497 (+1371031us) service_pool.cc:224] Handling call
0709 13:06:55.738579 (+ 93082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.739910 (+  1331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.057307 (+1317397us) spinlock_profiling.cc:243] Waited 75.5 ms on lock 0x7b54004cca50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.057465 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173559168}
W20260709 13:06:57.058475 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2366) took 2682 ms (client timeout 1760 ms). Trace:
W20260709 13:06:57.062093 22001 rpcz_store.cc:269] 0709 13:06:54.375795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.047646 (+2671851us) service_pool.cc:211] Skipping call since client already timed out
0709 13:06:57.058404 (+ 10758us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:06:57.062894 21856 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:57.061486 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0416488329999538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 13:06:57.048571 26110 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.065693 21860 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.073104 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2312) took 1264 ms. Trace:
I20260709 13:06:57.073508 25932 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.073983 25932 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:57.075022 25932 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.075732 25859 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.076404 25932 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.077132 25857 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:57.073498 26218 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:06:57.080416 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1964) took 3072 ms (client timeout 3184 ms). Trace:
W20260709 13:06:57.080838 21933 rpcz_store.cc:269] 0709 13:06:54.008409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.010432 (+2002023us) service_pool.cc:224] Handling call
0709 13:06:57.080215 (+1069783us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002e86d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.080359 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2335590784}
W20260709 13:06:57.081456 22586 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 796) took 3410 ms (client timeout 1711 ms). Trace:
I20260709 13:06:57.082053 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.082408 26215 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:57.082736 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.084053 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.084474 25857 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.084882 25857 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:57.085901 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.085732 25857 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.086395 26092 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.087054 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.082911 26215 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:57.088521 21926 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:06:57.081882 22586 rpcz_store.cc:269] 0709 13:06:53.671244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.778037 (+106793us) service_pool.cc:224] Handling call
0709 13:06:57.081228 (+3303191us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54006248d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.081374 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4737989376}
W20260709 13:06:57.137509 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.073496 21858 rpcz_store.cc:276] 0709 13:06:55.808797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.817605 (+  8808us) service_pool.cc:224] Handling call
0709 13:06:57.072911 (+1255306us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54000596d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.073046 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837990272}
W20260709 13:06:57.138623 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.137585 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:57.139209 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.121519 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.139816 21927 raft_consensus.cc:2410] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:06:57.139902 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 111125us
W20260709 13:06:57.145576 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.157819 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.066636 26086 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:57.165380 21930 raft_consensus.cc:2376] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10. Re-sending same reply.
I20260709 13:06:57.159781 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.165759 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.088827 26110 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:57.167528 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.174901 21927 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:57.087034 25857 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.171365 21932 raft_consensus.cc:2368] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:57.173252 21855 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:57.182250 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.182803 21859 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:57.061575 25863 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.166446 25846 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.162299 26225 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:06:57.138193 26223 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:57.194181 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 172216us
I20260709 13:06:57.194674 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.185145 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.198477 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.199872 21972 leader_election.cc:400] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:06:57.200173 21972 leader_election.cc:403] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:57.200757 26207 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:57.205163 26100 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.205698 26100 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.206125 26092 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:57.206770 26092 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:06:57.208531 26207 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:57.209784 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.210930 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.211957 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:57.213773 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.216962 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.177603 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.166194 26086 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.139111 26232 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:57.189394 21852 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:57.191642 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:57.223093 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 190791us
I20260709 13:06:57.193602 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2448) took 3499 ms. Trace:
I20260709 13:06:57.065490 26205 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:06:57.221690 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2450) took 1199 ms. Trace:
I20260709 13:06:57.221961 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.222836 26086 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.227504 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.259738 21927 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:57.230645 21930 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:06:57.240957 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.258018 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.262588 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.272562 21857 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:06:57.208310 26100 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:57.275641 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.227332 26223 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.229333 21931 rpcz_store.cc:276] 0709 13:06:55.995032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.727439 (+732407us) service_pool.cc:224] Handling call
0709 13:06:56.727799 (+   360us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:56.765411 (+ 37612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.765463 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:56.837203 (+ 71740us) log.cc:844] Serialized 52 byte log entry
0709 13:06:56.965308 (+128105us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:57.004905 (+ 39597us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.005198 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.005301 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.194022 (+188721us) raft_consensus.cc:1685] finished
0709 13:06:57.194047 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.194198 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39034,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":935,"raft.queue_time_us":22288,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":22454,"thread_start_us":76270,"threads_started":1,"wal-append.queue_time_us":144537}
I20260709 13:06:57.223771 21934 rpcz_store.cc:276] 0709 13:06:53.686405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.968175 (+281770us) service_pool.cc:224] Handling call
0709 13:06:54.003210 (+ 35035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.382023 (+1378813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.386571 (+  4548us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.476246 (+ 89675us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.669364 (+193118us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:55.669412 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.171872 (+1502460us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004290d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:57.171950 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.172015 (+    65us) raft_consensus.cc:1685] finished
0709 13:06:57.172035 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.186155 (+ 14120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10735,"raft.queue_time_us":70472,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":50579,"spinlock_wait_cycles":3003225344,"thread_start_us":200181,"threads_started":2,"wal-append.queue_time_us":458796,"wal-append.run_cpu_time_us":1881,"wal-append.run_wall_time_us":1057996}
I20260709 13:06:57.273288 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.285120 21932 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:57.224071 26232 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:57.288831 22586 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:06:57.295807 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.295847 22588 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.296746 22588 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:57.297262 22592 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:06:57.279950 26194 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.004s
I20260709 13:06:57.299850 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.305685 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.312879 21935 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:57.299263 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.317019 21857 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:57.299227 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.308645 22001 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:57.309739 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.310683 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.318334 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.318804 22007 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:06:57.229046 25846 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.324926 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.325870 21927 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:57.321738 22006 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:57.326210 22589 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:06:57.322098 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:57.330613 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.333920 22002 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:57.300104 26020 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:57.337489 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.338609 22003 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:57.328549 22588 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:57.340969 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78581.7us
I20260709 13:06:57.342175 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.344291 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:06:57.345268 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1611634239999375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2522s for a response
I20260709 13:06:57.295087 26094 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:06:57.346963 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.347666 22009 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:57.296156 26223 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.348977 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 744) took 1747 ms. Trace:
I20260709 13:06:57.349972 21899 leader_election.cc:400] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13.
I20260709 13:06:57.337182 26222 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:57.370101 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.382237 22589 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:06:57.345803 26207 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:06:57.278944 26215 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.379154 22588 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.351485 26235 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.173s
I20260709 13:06:57.340734 26105 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:06:57.352193 21899 leader_election.cc:403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:06:57.351857 21928 rpcz_store.cc:276] 0709 13:06:55.602060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.682866 (+1080806us) service_pool.cc:224] Handling call
0709 13:06:56.702142 (+ 19276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.748592 (+ 46450us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:57.017478 (+268886us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.348483 (+331005us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.348661 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.348709 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:57.348751 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.348812 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.348923 (+   111us) inbound_call.cc:177] Queueing success response
0709 13:06:57.357299 (+  8376us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b18001be680. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
Metrics: {"prepare.queue_time_us":20228,"raft.queue_time_us":65715,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":25370,"spinlock_wait_cycles":4576640,"thread_start_us":276208,"threads_started":2,"wal-append.queue_time_us":303843}
I20260709 13:06:57.376974 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.389186 21859 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:06:57.391314 21857 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:57.388447 21856 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:57.390357 21860 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:06:57.394819 26073 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
I20260709 13:06:57.395715 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.397030 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.397195 21926 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:57.398080 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.398598 21857 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:57.398861 26110 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.314589 26194 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.401513 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.402352 26207 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.402716 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.402797 26207 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:57.403863 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.404641 21856 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:06:57.404028 26207 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.405527 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.353260 26028 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:06:57.405375 21851 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:06:57.406998 21851 pending_rounds.cc:85] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 10
I20260709 13:06:57.407316 21851 pending_rounds.cc:107] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:06:57.407588 21851 raft_consensus.cc:2889] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:57.408316 21856 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:57.409320 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117798us
I20260709 13:06:57.410086 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.410733 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.411417 21860 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:06:57.404898 26228 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.261s	user 0.003s	sys 0.000s
I20260709 13:06:57.412875 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.415113 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.422770 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:06:57.422796 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73007.8us
I20260709 13:06:57.402633 26194 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:57.426281 26020 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.429457 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.429689 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.429853 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:06:57.430217 26231 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:06:57.430562 22001 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 7.
I20260709 13:06:57.430694 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.430871 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.431823 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:57.429466 26245 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:57.435997 21929 raft_consensus.cc:2410] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:57.442781 22585 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:57.445080 26166 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.445631 26166 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.446615 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 807) took 1459 ms. Trace:
I20260709 13:06:57.354307 26232 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.447075 22593 rpcz_store.cc:276] 0709 13:06:55.986909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.989778 (+  2869us) service_pool.cc:224] Handling call
0709 13:06:57.446364 (+1456586us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400595454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.446533 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330657664}
W20260709 13:06:57.424642 21972 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.642s (SENT)
I20260709 13:06:57.429690 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.449396 22591 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:57.449527 21857 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:57.412310 26233 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:57.447470 26232 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:57.450930 21931 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
W20260709 13:06:57.452631 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 234149us
I20260709 13:06:57.455977 22589 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:57.457233 21857 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:57.459064 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:57.418655 26222 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:57.453325 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.461736 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.463114 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:57.471062 22001 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:06:57.485689 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.484797 21974 leader_election.cc:400] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:57.486340 21974 leader_election.cc:403] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:06:57.475953 22588 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:06:57.497617 22588 pending_rounds.cc:85] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 9
I20260709 13:06:57.497915 22588 pending_rounds.cc:107] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:06:57.498205 22588 raft_consensus.cc:2889] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:57.503551 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.479633 21857 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:57.527413 21856 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:57.453390 26110 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:57.540707 21853 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:57.546140 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 62789.6us
W20260709 13:06:57.547206 26242 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:57.555146 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:57.555701 22585 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:57.475063 26233 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:57.557304 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:06:57.547101 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 98847.5us
I20260709 13:06:57.558458 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.547719 26138 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:57.559267 26138 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.559703 26138 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.560082 26250 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:57.561249 26192 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:57.561640 26192 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:57.562592 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 657) took 1494 ms. Trace:
I20260709 13:06:57.562561 26192 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.563136 21851 rpcz_store.cc:276] 0709 13:06:56.067756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.515358 (+447602us) service_pool.cc:224] Handling call
0709 13:06:56.515664 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.417388 (+901724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.562353 (+144965us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400259754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:57.562534 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045256064}
I20260709 13:06:57.564041 26192 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.560112 26092 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.565423 26092 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:57.566277 26092 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:06:57.567425 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0029488330000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
I20260709 13:06:57.567747 26092 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:57.567904 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0002488330000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
W20260709 13:06:57.569288 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 153422us
I20260709 13:06:57.570091 26246 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.554493 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:57.554688 22589 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:57.558059 26233 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.578400 22588 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.548324 21934 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:57.579219 22592 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:06:57.581212 22585 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
W20260709 13:06:57.549669 21823 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.772s (SENT)
I20260709 13:06:57.586782 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:57.475301 26105 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:57.591097 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:57.596920 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:57.601867 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 129682us
I20260709 13:06:57.605559 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2455) took 1614 ms. Trace:
I20260709 13:06:57.608613 22005 rpcz_store.cc:276] 0709 13:06:55.991299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.011842 (+ 20543us) service_pool.cc:224] Handling call
0709 13:06:57.605294 (+1593452us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54001e1ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:57.605480 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876272000}
I20260709 13:06:57.613775 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:57.614303 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:57.615178 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56808.3us
I20260709 13:06:57.616214 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.619422 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.577553 25898 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.475569 26247 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13.
I20260709 13:06:57.626215 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.564886 26094 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:57.580070 26207 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.632005 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:57.628131 21934 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [7.5-10.7]
I20260709 13:06:57.550540 26194 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:06:57.613531 26243 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.607064 26105 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:06:57.660408 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 171428us
W20260709 13:06:57.640394 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.660624 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:57.666886 21928 raft_consensus.cc:2368] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:57.619105 25932 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.660234 21929 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:06:57.664608 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.620301 26216 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:57.591403 26250 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:06:57.669453 21933 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:06:57.674012 22588 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:06:57.675999 22001 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:57.678226 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:57.679407 26086 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.633445 26094 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:57.598536 25915 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:06:57.681775 26086 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:57.681717 26094 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.683779 21931 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:57.633292 26246 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.683120 26094 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:57.686480 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 95494.4us
I20260709 13:06:57.687451 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.688880 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.627799 26248 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:57.630224 25846 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:57.690169 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:57.625365 26174 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:06:57.683154 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:57.669027 26105 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:57.695626 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.637096 26111 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:06:57.713666 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:57.731492 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99568.2us
I20260709 13:06:57.636875 25898 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.670323 26073 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:57.688405 26255 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:06:57.733899 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.734894 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:57.689697 26248 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:57.683493 26247 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:57.696537 25857 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.006s
I20260709 13:06:57.734980 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:57.657544 26260 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:06:57.738157 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:06:57.736089 26218 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:57.703449 25915 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:57.738852 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:57.742429 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55051.9us
I20260709 13:06:57.735889 25932 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.746629 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.746955 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:06:57.747920 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 64864.6us
I20260709 13:06:57.727777 26254 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:57.751289 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.739281 26247 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:57.752336 26258 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:06:57.746093 26218 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.752192 26238 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:57.741055 26073 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:57.738965 25846 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:57.765584 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.745558 26110 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:57.740756 26248 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:57.746310 25915 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.757393 26246 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.750973 25857 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:57.738407 26028 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:06:57.761062 26216 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:57.769421 26218 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:57.770182 26238 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.781623 26254 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:57.786283 25915 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:57.795804 26086 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:57.810617 26086 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.812053 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.812615 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.763381 26247 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:57.803253 26110 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:57.804762 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.693806 26222 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.806741 26254 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.776656 26250 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:06:57.795974 26238 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.794107 26166 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:57.800797 26262 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:57.819340 25857 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.840310 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93714.3us
I20260709 13:06:57.811260 26086 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.852690 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.855432 21852 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:57.822579 26239 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.858904 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.865458 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:57.866325 21855 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.869974 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:57.870824 21851 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
W20260709 13:06:57.872078 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67461.7us
W20260709 13:06:57.872751 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.879956 21852 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:06:57.893569 26264 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:06:57.894189 26086 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:57.897728 22003 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:57.900871 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.862407 26073 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.920959 26239 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.921716 26138 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:57.880312 26216 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:06:57.886646 26256 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:06:57.856849 25857 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:57.896536 26105 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:57.940148 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.937790 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49660us
I20260709 13:06:57.952340 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:57.952872 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:57.961278 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.870225 26262 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
W20260709 13:06:57.960947 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78177.4us
I20260709 13:06:57.953239 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.887712 26100 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:06:57.963804 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:57.983381 26105 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:57.951941 26265 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:57.982671 26243 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.934983 26020 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:06:57.985114 26267 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.011390 22005 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:57.961656 26086 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:06:57.993183 26111 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.026046 26278 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:57.923064 26110 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.007354 26233 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.952713 26239 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 13:06:58.032866 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 79728.8us
I20260709 13:06:57.992919 26262 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:58.005440 26100 consensus_queue.cc:1243] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:57.964428 26166 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:06:57.979212 26271 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:06:58.033710 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.011627 26235 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.010787 25846 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:58.072386 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 119571us
I20260709 13:06:58.076386 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:58.010812 25932 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:06:58.036520 26262 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.035617 26239 consensus_queue.cc:1243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:58.084118 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.034581 26105 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:58.099568 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47441.8us
I20260709 13:06:58.101094 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.103365 26269 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:58.100716 26273 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:58.106601 25857 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:58.084457 26232 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.003s
I20260709 13:06:58.009410 26243 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.103519 21972 leader_election.cc:400] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:58.129648 21972 leader_election.cc:403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:58.051929 26223 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.083706 26262 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.133608 21855 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:06:58.135406 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:06:58.142059 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 86783.6us
I20260709 13:06:58.090871 26105 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:58.150934 22584 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:58.032286 26216 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.159520 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:58.156438 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:58.084288 26233 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:06:58.082927 25846 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.164489 22586 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:58.146358 25857 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.146546 26174 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:06:58.171221 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 124691us
I20260709 13:06:58.142577 26273 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12.
I20260709 13:06:58.159890 26232 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:58.168002 26073 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.169680 25932 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:58.194166 26192 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:58.195277 26234 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:06:58.196777 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93370.5us
I20260709 13:06:58.205274 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.180477 25857 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:58.214972 26279 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:06:58.216120 25857 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.217156 25932 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:58.187578 26174 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.186317 26232 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:58.218302 25932 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.219053 25846 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.219929 26174 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.187547 26105 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.221464 26105 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:58.222106 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.219939 26283 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.223278 26283 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:58.204109 26275 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:06:58.225211 26028 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:58.196877 26248 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.226344 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.226564 26250 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:06:58.227752 26248 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.228343 26273 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:58.229219 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.229126 26282 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:06:58.230319 26282 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:58.230592 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:58.231410 26247 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.232405 26243 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.205312 26073 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.234520 26283 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.235313 26073 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.235725 26261 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:58.210032 26271 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:06:58.237560 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:58.203261 25898 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.204602 26234 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:58.244728 26234 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.245893 26247 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.246122 25915 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.218837 26232 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:58.248785 25915 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.230649 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.251737 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 1984) took 1172 ms. Trace:
I20260709 13:06:58.251806 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:58.252290 21933 rpcz_store.cc:276] 0709 13:06:57.079131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.081739 (+  2608us) service_pool.cc:224] Handling call
0709 13:06:58.251559 (+1169820us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54003070d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:06:58.251671 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871157504}
W20260709 13:06:58.236090 26094 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.005s
I20260709 13:06:58.230710 26282 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:58.265933 22591 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:58.270138 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:58.270097 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:06:58.277890 22593 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.270053 26273 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:58.270362 26282 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.271621 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:58.276856 26223 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:58.262993 26238 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.283905 26223 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:58.205087 26192 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:58.285020 26223 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.285786 26192 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.286742 26223 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:58.287006 26192 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:58.273906 26280 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:06:58.289640 26282 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.290815 26238 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.293989 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:58.295218 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:58.295770 22006 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:58.274287 26277 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:58.265978 26269 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:58.281837 21855 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.267352 26028 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:06:58.306450 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:58.305516 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:58.308498 22009 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:06:58.310029 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.310876 21926 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:58.310727 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:58.322433 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:58.323297 22005 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:58.323387 21854 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:58.325161 26248 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:06:58.328231 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.329295 22593 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:58.330855 21855 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:58.333719 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:58.336329 22002 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:58.300916 26277 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:58.347271 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82695.9us
W20260709 13:06:58.622030 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.390018 25898 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.391039 26232 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:58.603040 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.338651 26262 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.371191 26277 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.529196 21927 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:58.539551 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.490810 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:58.634536 26285 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.355s	user 0.002s	sys 0.000s
I20260709 13:06:58.296991 26271 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:06:58.528844 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.651863 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.652527 21928 raft_consensus.cc:2376] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11. Re-sending same reply.
W20260709 13:06:58.652808 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.653621 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.510905 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:06:58.518882 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.407003 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 196217us
I20260709 13:06:58.655799 21935 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:58.656143 26292 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:58.464818 26222 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.608129 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.656782 26292 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.657455 26222 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.519048 21934 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:58.497237 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.378386 21854 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:58.618161 26248 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:06:58.661556 26298 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:06:58.608438 26260 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.664016 26260 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.638629 21972 leader_election.cc:400] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:58.667769 26260 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:58.530262 26218 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.001s
I20260709 13:06:58.657583 21932 raft_consensus.cc:2376] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11. Re-sending same reply.
I20260709 13:06:58.668560 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:58.507470 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.669691 26105 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:58.661170 26248 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.670288 26105 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:58.670787 26105 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:58.671391 26276 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.590s	user 0.006s	sys 0.016s
I20260709 13:06:58.672906 26248 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.674820 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:58.645395 21926 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:58.678994 21935 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:58.680882 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.682828 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
W20260709 13:06:58.684533 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.689610 26216 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:58.689844 26028 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.690392 26216 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.690766 26028 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.695569 26028 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:58.698190 26246 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:58.698693 26246 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:58.699641 26246 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.700778 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.682178 26207 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:06:58.700212 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.706041 26207 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:58.706591 26207 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:58.661489 26232 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:58.647156 25898 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:58.709729 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53463.2us
I20260709 13:06:58.711030 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:06:58.711980 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 678324us
I20260709 13:06:58.708035 26138 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.707667 26207 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:58.724539 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48249.1us
I20260709 13:06:58.733263 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.743237 21900 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:58.742833 26105 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.715297 25898 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.729403 26246 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:58.744768 26288 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:58.745177 26207 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.745865 26105 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:58.743958 26282 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.746920 26138 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:58.747478 26138 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:58.747931 26268 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.749138 21854 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:06:58.748554 26138 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.749186 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:58.751744 21851 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:58.753989 25898 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.715581 26261 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:58.754633 25898 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.718120 26234 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.692935 26216 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.715664 26114 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:58.764783 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.766662 21926 raft_consensus.cc:2368] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:58.775179 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.779242 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.779384 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:06:58.686074 26218 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.764488 26282 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.765961 26114 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:06:58.790464 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77520.9us
I20260709 13:06:58.793069 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.748555 26268 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.781021 26288 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:58.801723 22587 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:06:58.803807 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1716918350000469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260709 13:06:58.806496 21972 connection.cc:256] RPC call timeout handler was delayed by -1.0879918350000253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:06:58.806885 21972 connection.cc:256] RPC call timeout handler was delayed by -1.0555918349999729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
I20260709 13:06:58.810894 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:58.812008 22008 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:58.804162 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:58.814528 21854 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:58.805593 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:58.815490 21927 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:58.818506 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.777987 26295 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:06:58.821595 26114 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:58.822886 26114 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.825237 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.818516 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.826040 21932 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:58.826736 21926 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:58.827248 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.827950 21932 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:58.825416 26100 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:58.830677 26100 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:58.831718 26100 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.797809 26223 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:58.833349 26223 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.833781 26223 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:58.837492 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.838337 21935 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:58.840751 26288 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.843389 26288 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.844650 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.789999 25898 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.792549 26248 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:58.795291 26192 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:58.853799 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 74637.5us
I20260709 13:06:58.858614 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:58.853084 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:58.804589 26233 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:58.859951 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52381.1us
I20260709 13:06:58.898499 22001 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:58.900168 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.900230 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.902160 22008 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:58.795847 26282 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:58.921265 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.922235 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:58.924763 22006 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:58.926185 21932 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:06:58.922640 21927 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:58.859414 26100 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.934689 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.945768 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.937593 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:58.905360 26261 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:58.959239 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.960044 22006 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:58.936589 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.961546 21927 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:06:58.965188 21935 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:58.966531 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.968019 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:58.922959 26250 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.975123 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.976728 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:58.979315 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 183662us
I20260709 13:06:58.927687 26216 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:58.863128 26248 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:58.986637 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49147.2us
I20260709 13:06:58.932551 26105 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.956148 26192 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:58.993348 26028 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.995076 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.981644 26111 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:58.997735 26273 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.001188 26292 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:58.971833 26110 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.008193 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 820) took 1184 ms. Trace:
I20260709 13:06:59.009133 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:06:59.060395 22586 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
W20260709 13:06:59.010641 26305 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:59.015260 26222 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:58.923046 26295 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:06:58.957257 25846 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:58.965893 26294 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.343s	user 0.000s	sys 0.002s
I20260709 13:06:58.988798 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.052866 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:59.057487 21935 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:06:59.058208 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63184.5us
I20260709 13:06:58.984398 26261 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:59.076287 21927 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:06:59.076818 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:58.976364 26174 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:59.059743 22587 rpcz_store.cc:276] 0709 13:06:57.823480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.823649 (+   169us) service_pool.cc:224] Handling call
0709 13:06:57.841618 (+ 17969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.824679 (+983061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.007777 (+183098us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54004d12d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:59.008100 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205801344}
I20260709 13:06:59.070300 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.083446 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:59.072347 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.073253 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.054172 26262 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:59.089118 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.088948 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.094041 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.094252 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.094846 21927 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:59.067902 26192 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:59.071640 26111 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:59.102105 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:59.105633 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.090842 26110 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.113967 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:59.115785 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:59.057713 26105 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.138309 26235 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:59.138742 26105 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.138814 26235 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:59.095430 25582 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:59.140385 25582 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:59.140033 26235 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.140834 25582 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.125957 25859 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:59.141590 25859 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.141952 25859 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.097973 25898 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:59.103581 26094 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:59.097427 26250 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:06:59.100888 26174 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:59.127813 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.151412 26261 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.152352 26105 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.151863 25582 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.129969 26223 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.122375 26218 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.133325 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:59.154618 21927 raft_consensus.cc:2376] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9. Re-sending same reply.
I20260709 13:06:59.103366 26271 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:59.154783 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.135823 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.160043 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.161926 21933 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:59.162753 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2480) took 1277 ms. Trace:
I20260709 13:06:59.163303 22000 rpcz_store.cc:276] 0709 13:06:57.885372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.921470 (+ 36098us) service_pool.cc:224] Handling call
0709 13:06:57.921784 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:57.935081 (+ 13297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:57.935116 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:57.961229 (+ 26113us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.330496 (+369267us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:58.330549 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.330634 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:58.375202 (+ 44568us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4800130158. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:06:59.140910 (+765708us) raft_consensus.cc:1685] finished
0709 13:06:59.140933 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.162663 (+ 21730us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12873,"raft.queue_time_us":13178,"raft.run_cpu_time_us":3587,"raft.run_wall_time_us":437716,"spinlock_wait_cycles":27244288,"thread_start_us":345144,"threads_started":1,"wal-append.queue_time_us":526964}
I20260709 13:06:59.121554 26088 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:59.153748 26105 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:06:59.169253 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.169538 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53846.9us
I20260709 13:06:59.177582 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.178264 21927 raft_consensus.cc:2376] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10. Re-sending same reply.
I20260709 13:06:59.179469 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.180361 21933 raft_consensus.cc:2368] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13.
I20260709 13:06:59.181354 26110 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:59.181718 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:59.182013 26110 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:59.183179 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.183063 26110 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.183946 21927 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:06:59.184805 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.185915 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:59.161969 26207 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:06:59.185793 26192 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:59.161536 25898 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:59.188162 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.189651 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.189940 26268 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.157271 26260 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:59.190783 26232 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.191135 26260 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.191622 26260 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:59.193495 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:59.166558 26235 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:59.194298 26282 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.195508 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.197384 26162 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.198253 21926 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:06:59.199426 25898 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:59.201998 21972 leader_election.cc:400] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:06:59.202451 25582 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.203063 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:59.203627 26241 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.932s	user 0.006s	sys 0.006s
I20260709 13:06:59.206017 26130 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:59.213634 22556 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Timed out: Call waited in the queue past client deadline [suppressed 45 similar messages]
I20260709 13:06:59.192600 26248 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.214644 26218 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:59.218885 26218 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:06:59.219965 26218 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.170359 26088 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.221978 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.222944 21933 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:06:59.222139 26088 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:59.224436 26105 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:59.238205 22007 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:06:59.238626 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:59.239380 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.239945 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.240535 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.235597 26111 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.241408 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.241554 26111 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.242703 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.243580 22008 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:59.243971 26111 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.244642 26094 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.230486 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:59.246505 22001 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:59.245383 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.247354 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.245761 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.225855 26223 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.249289 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.250056 22009 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:06:59.250095 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:59.251492 22009 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:06:59.251852 22009 pending_rounds.cc:85] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:59.252100 22009 pending_rounds.cc:107] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:59.252343 22009 raft_consensus.cc:2889] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:59.230846 21928 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:06:59.216697 25582 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.253343 22007 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:59.261860 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:59.262773 21933 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:59.225909 26248 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.264240 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.265250 22009 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:06:59.265859 22009 pending_rounds.cc:85] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:06:59.266207 22009 pending_rounds.cc:107] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:06:59.266546 22009 raft_consensus.cc:2889] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:59.265196 21928 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:06:59.263825 26273 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:59.288816 26273 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:06:59.289378 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.290006 26273 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.244472 26260 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.291201 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.291862 22003 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:59.292734 26262 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:59.263641 25582 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.294422 22001 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:59.294770 26260 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.295477 26207 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:06:59.296068 26207 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:59.296669 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.271911 26028 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:59.274979 26105 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.280275 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:59.295509 25582 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.280836 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:06:59.304040 22002 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:59.292924 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.306951 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:59.311728 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:06:59.284013 26246 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.000s
I20260709 13:06:59.270687 26248 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.300987 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:59.351145 22005 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:59.329526 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 1998) took 2048 ms. Trace:
W20260709 13:06:59.369918 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 58277.9us
I20260709 13:06:59.373883 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:59.381904 21854 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:59.383689 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.393075 21854 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:59.300338 26207 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.320863 26262 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.277359 25859 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:59.404596 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53794.9us
I20260709 13:06:59.350083 26138 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:59.322898 26247 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.322563 26282 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:59.362660 21930 rpcz_store.cc:276] 0709 13:06:57.270773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.277949 (+  7176us) service_pool.cc:224] Handling call
0709 13:06:57.293297 (+ 15348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.455752 (+162455us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:57.550891 (+ 95139us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.948635 (+397744us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.948693 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:57.948741 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.318606 (+1369865us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54002e86d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:06:59.318690 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.318905 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64467,"prepare.queue_time_us":85645,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":88146,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":12685,"spinlock_wait_cycles":275950336,"thread_start_us":306973,"threads_started":2,"wal-append.queue_time_us":59875,"wal-append.run_cpu_time_us":1675,"wal-append.run_wall_time_us":1016443}
I20260709 13:06:59.415578 21858 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:06:59.403506 21859 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:06:59.403961 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 145039us
W20260709 13:06:59.405203 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 104335us
I20260709 13:06:59.300634 26105 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:06:59.301800 26265 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:06:59.407331 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.377058 21933 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:06:59.382233 25915 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.418587 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44820.9us
I20260709 13:06:59.400333 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.421180 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.423997 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.424626 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.393508 26310 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:59.430846 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.382660 26292 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:06:59.435935 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:06:59.445254 21851 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:06:59.442540 22000 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:59.442296 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.445585 21853 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:59.439076 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.447795 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.365669 26088 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.451308 21933 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:59.451475 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.459543 21930 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:06:59.420543 26260 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:59.423506 26265 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:59.468873 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.471520 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.472389 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:06:59.474337 21854 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:59.415603 26114 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:06:59.444821 26245 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.532887 21860 raft_consensus.cc:2368] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:06:59.418582 26247 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.442271 26207 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.558830 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:06:59.470405 26260 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.467792 26288 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:59.467535 26166 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.469429 26100 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.433930 25915 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:59.574666 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 135488us
I20260709 13:06:59.478528 26267 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.582949 21933 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:59.584952 21853 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:06:59.475384 26254 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:06:59.464146 26234 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:06:59.587107 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.588239 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.590283 22008 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:06:59.590590 21859 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:59.480839 26283 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:06:59.478149 26282 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:06:59.563581 26322 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:59.585884 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169196us
I20260709 13:06:59.583804 26260 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:59.578903 26222 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:59.581672 26288 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.580690 26166 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:06:59.645995 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.629451 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50035us
I20260709 13:06:59.633493 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:59.683923 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.683090 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:06:59.687927 21926 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:06:59.689108 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:06:59.583863 26100 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.686820 26338 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:59.645437 25582 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.690961 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:59.691376 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.712589 21857 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:59.604828 25932 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:59.690292 21926 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:59.708668 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:06:59.632639 26247 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.634268 26105 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:59.641140 26271 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:59.633455 25859 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.635507 26088 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.609604 26310 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:06:59.638350 25915 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.624007 26248 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:59.709525 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 495897us
W20260709 13:06:59.858505 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.742854 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52032.1us
I20260709 13:06:59.759347 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:06:59.780757 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:06:59.767593 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45710us
I20260709 13:06:59.888181 21972 leader_election.cc:400] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13.
I20260709 13:06:59.888620 21972 leader_election.cc:403] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:59.750151 21932 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:06:59.883975 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:59.886806 22001 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:59.891652 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:06:59.700079 26295 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.894042 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:06:59.887166 26271 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:06:59.896751 22006 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:59.896162 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.742974 26338 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:59.876133 26273 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.905706 26273 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.908054 26254 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:59.908375 26273 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.908634 26254 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:06:59.910126 26254 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.911520 26254 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:59.747439 26166 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.720830 25582 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.901219 26295 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:06:59.781950 26331 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:06:59.915122 26295 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:06:59.916172 26299 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.297s	user 0.006s	sys 0.004s
I20260709 13:06:59.915122 25582 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:06:59.919269 26265 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.827520 26207 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:59.920646 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1017) took 1675 ms. Trace:
I20260709 13:06:59.920465 26207 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:06:59.714120 26267 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.921609 26267 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.923051 26207 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:06:59.846828 26262 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.923856 26262 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.924268 26267 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.926294 26262 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:06:59.921165 22584 rpcz_store.cc:276] 0709 13:06:58.245523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.267529 (+ 22006us) service_pool.cc:224] Handling call
0709 13:06:58.268023 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.277710 (+  9687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.277744 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.278273 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.489384 (+211111us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:58.489452 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.489565 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.915462 (+1425897us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:59.920291 (+  4829us) raft_consensus.cc:1685] finished
0709 13:06:59.920316 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.920577 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2175,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":1552,"spinlock_wait_cycles":5788544,"thread_start_us":196655,"threads_started":1,"wal-append.queue_time_us":337131}
I20260709 13:06:59.787199 26215 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:59.752151 26339 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:06:59.795241 26114 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:06:59.904709 26235 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:59.900943 26218 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:59.828137 26111 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:06:59.699206 26322 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:59.937635 22001 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:06:59.940953 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:06:59.942343 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:00.494936 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 552676us
W20260709 13:06:59.946997 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.742354 26100 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:00.497802 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.497884 26100 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:00.499569 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:06:59.854995 26288 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:00.500685 26354 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:00.501289 26354 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.503652 26354 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:00.504241 26354 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:00.010195 26207 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.513232 26207 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.012523 26273 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:59.989246 26105 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:59.930593 26330 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 13:06:59.886401 25932 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:06:59.928964 26346 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
I20260709 13:06:59.921092 26338 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.901643 25915 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:59.929193 26260 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:06:59.966356 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:00.011633 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 70077.5us
I20260709 13:07:00.026046 21926 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:00.680531 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2498) took 1228 ms. Trace:
I20260709 13:07:00.106375 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:00.683200 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:00.684034 22001 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:00.152438 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:00.160370 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:00.164626 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2016) took 1264 ms. Trace:
I20260709 13:07:00.169337 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:00.171504 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.689093 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 528788us
I20260709 13:07:00.689996 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:00.688827 21935 rpcz_store.cc:276] 0709 13:06:58.900616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.967716 (+ 67100us) service_pool.cc:224] Handling call
0709 13:07:00.164336 (+1196620us) spinlock_profiling.cc:243] Waited 88.7 ms on lock 0x7b54003066d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:00.164531 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203921280}
W20260709 13:07:00.219384 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 323264us
I20260709 13:07:00.688824 22004 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:00.346356 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2491) took 1536 ms. Trace:
I20260709 13:07:00.027367 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:00.429569 22007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2402) took 1189 ms (client timeout 1565 ms). Trace:
W20260709 13:07:00.430290 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2048) took 1627 ms (client timeout 1502 ms). Trace:
I20260709 13:07:00.437852 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2353) took 1031 ms. Trace:
I20260709 13:07:00.467777 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2398) took 1229 ms. Trace:
W20260709 13:07:00.468055 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.474704 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.692462 21926 rpcz_store.cc:276] 0709 13:06:59.452492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.688737 (+236245us) service_pool.cc:224] Handling call
0709 13:07:00.680438 (+991701us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:00.507659 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.520246 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:00.715054 21854 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:00.529387 21852 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:00.529460 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2389) took 1301 ms. Trace:
I20260709 13:07:00.717083 22000 rpcz_store.cc:276] 0709 13:06:59.227966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.228900 (+   934us) service_pool.cc:224] Handling call
0709 13:06:59.230055 (+  1155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.446285 (+216230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.529131 (+1082846us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540038b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:00.529382 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468211584}
W20260709 13:07:00.547721 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.557034 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2403) took 1317 ms. Trace:
W20260709 13:07:00.319854 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.753275 26354 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.160305 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.079113 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 138291us
I20260709 13:07:00.563944 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2492) took 1285 ms. Trace:
I20260709 13:07:00.455020 25859 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.860579 25859 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.863368 25859 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:00.294991 25582 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.517561 25915 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.912264 25915 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:00.518546 25932 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.053750 25898 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:00.939512 25898 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:00.940963 25898 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.518160 26215 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.343976 26110 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:00.978097 26110 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.037191 26111 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.983830 26111 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:00.349043 26138 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:00.405128 26222 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:00.992028 26222 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.992731 25898 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:00.994728 26222 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:00.119036 26174 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.938253 26205 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.996352 26174 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.997022 26205 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.021157 26218 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:00.993379 25898 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.092082 26265 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:00.020625 26114 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.264684 26250 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:01.011272 26250 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.012033 26114 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:00.232488 26234 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.015566 26234 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.016495 25898 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:00.231047 26223 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.019023 26223 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.036108 26235 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:01.023286 26235 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.523530 26260 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:00.550587 26216 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:01.028424 26216 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:00.522055 26207 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:00.072913 26339 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:07:00.353240 26347 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.609s	user 0.000s	sys 0.006s
I20260709 13:07:00.036966 26269 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13.
I20260709 13:07:01.047891 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2495) took 1596 ms. Trace:
I20260709 13:07:01.048391 21928 rpcz_store.cc:276] 0709 13:06:59.451294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.451828 (+   534us) service_pool.cc:224] Handling call
0709 13:06:59.474403 (+ 22575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.565067 (+ 90664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.565098 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:59.570299 (+  5201us) log.cc:844] Serialized 26 byte log entry
0709 13:06:59.712619 (+142320us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:59.712697 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.712817 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:00.352134 (+639317us) spinlock_profiling.cc:243] Waited 9.35 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:01.047468 (+695334us) raft_consensus.cc:1685] finished
0709 13:07:01.047495 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.047811 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93029,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":11112,"spinlock_wait_cycles":21512576,"thread_start_us":105924,"threads_started":2,"wal-append.queue_time_us":138261}
I20260709 13:07:00.032044 26271 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.487927 26277 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:00.033209 26322 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.060401 26322 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:00.529498 26288 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.066116 26288 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.517465 26338 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:00.664134 26273 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.072288 26273 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:00.676137 26351 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:07:00.262930 26352 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:00.691906 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:00.489739 21934 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:07:01.077644 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2474) took 2527 ms. Trace:
I20260709 13:07:01.077895 26273 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.078050 21934 rpcz_store.cc:276] 0709 13:06:58.549802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.658759 (+108957us) service_pool.cc:224] Handling call
0709 13:07:00.489157 (+1830398us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:00.489452 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.077273 (+587821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.077572 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38229248}
I20260709 13:07:01.078768 26273 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.691658 21927 rpcz_store.cc:276] 0709 13:06:58.809485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.186926 (+377441us) service_pool.cc:224] Handling call
0709 13:07:00.344502 (+1157576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.344855 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.344876 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:00.344916 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:00.344981 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.345130 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.346155 (+  1025us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003066d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:00.346286 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2661091712}
W20260709 13:07:00.692329 21851 rpcz_store.cc:269] 0709 13:06:58.802815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.803001 (+   186us) service_pool.cc:224] Handling call
0709 13:07:00.430055 (+1627054us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54000fcb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:00.430181 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087613696}
I20260709 13:07:00.694214 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:00.692530 21860 rpcz_store.cc:276] 0709 13:06:59.406861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.407027 (+   166us) service_pool.cc:224] Handling call
0709 13:07:00.437525 (+1030498us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:00.437732 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257222016}
I20260709 13:07:00.695427 22005 rpcz_store.cc:276] 0709 13:06:59.238541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.251161 (+ 12620us) service_pool.cc:224] Handling call
0709 13:06:59.251487 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.365286 (+113799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.467374 (+1102088us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b5400166f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:00.467682 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228704128}
I20260709 13:07:00.696918 26105 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:00.697078 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:07:00.692265 22007 rpcz_store.cc:269] 0709 13:06:59.239691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.240292 (+   601us) service_pool.cc:224] Handling call
0709 13:07:00.429312 (+1189020us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b54002710d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:00.429482 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17312000}
I20260709 13:07:00.701261 26254 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:01.088425 26254 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:00.703794 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 204277us
I20260709 13:07:01.090986 25898 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:00.709954 26088 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.076905 26352 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.721381 22006 rpcz_store.cc:276] 0709 13:06:59.239988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.240934 (+   946us) service_pool.cc:224] Handling call
0709 13:07:00.556561 (+1315627us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b540038aed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:00.556901 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491043712}
I20260709 13:07:00.729916 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:00.700672 22034 heartbeater.cc:646] Failed to heartbeat to 127.20.66.190:46167 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.20.66.190:46167 timed out after 15.000s (SENT)
I20260709 13:07:00.754009 26354 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.095925 26352 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:01.094949 22000 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:00.757864 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:01.097321 26354 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:00.757107 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:00.760859 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:00.761345 22004 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:07:01.104761 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.767423 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 77504.6us
I20260709 13:07:00.778692 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.105398 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2502) took 1170 ms. Trace:
I20260709 13:07:01.106370 22004 rpcz_store.cc:276] 0709 13:06:59.934913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.152094 (+217181us) service_pool.cc:224] Handling call
0709 13:07:01.105303 (+953209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:00.778699 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.778988 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.109771 22591 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
W20260709 13:07:00.784140 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 92944.6us
I20260709 13:07:00.788630 26365 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:00.793114 26366 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:00.793754 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:00.811612 22586 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:07:00.812109 26364 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.814607 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2358) took 1387 ms. Trace:
I20260709 13:07:00.845327 22009 rpcz_store.cc:276] 0709 13:06:59.278379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.925425 (+647046us) service_pool.cc:224] Handling call
0709 13:06:59.925734 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.925970 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:59.926311 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:07:00.435498 (+509187us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b1800138b20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:00.553933 (+118435us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b1800138b20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a
0709 13:07:00.563278 (+  9345us) log.cc:844] Serialized 14 byte log entry
0709 13:07:00.563532 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:00.563590 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:00.563646 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.563717 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.563869 (+   152us) inbound_call.cc:177] Queueing success response
0709 13:07:00.762038 (+198169us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800138b20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
Metrics: {"mutex_wait_us":9839,"prepare.queue_time_us":8810,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":100,"raft.queue_time_us":4082,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":6603,"spinlock_wait_cycles":893919744,"thread_start_us":98677,"threads_started":1,"wal-append.queue_time_us":99071}
W20260709 13:07:00.851215 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 55622.2us
I20260709 13:07:01.143537 21852 rpcz_store.cc:276] 0709 13:06:59.427246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.427400 (+   154us) service_pool.cc:224] Handling call
0709 13:06:59.455203 (+ 27803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.727957 (+1272754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.814134 (+ 86177us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540008a7d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:00.814494 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2469595008}
I20260709 13:07:00.812412 26371 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:00.857901 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:00.305946 25846 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.303900 25857 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:00.894954 26376 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:01.227797 25857 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.910250 25582 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.933684 25932 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:01.229648 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2490) took 1951 ms. Trace:
I20260709 13:07:01.230670 25582 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.230108 22003 rpcz_store.cc:276] 0709 13:06:59.277845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.292480 (+ 14635us) service_pool.cc:224] Handling call
0709 13:07:00.557711 (+1265231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.096631 (+538920us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:01.096929 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.214685 (+117756us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.215087 (+   402us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:01.215180 (+    93us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:01.215268 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.227201 (+ 11933us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18000a1fe0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419
0709 13:07:01.228629 (+  1428us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b540038aed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:01.228684 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.229426 (+   742us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540038aed0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.229576 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11367,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":1626,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":11015,"spinlock_wait_cycles":4170384512,"thread_start_us":91879,"threads_started":1}
I20260709 13:07:00.956019 26378 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.232728 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:01.233373 26378 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.234221 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.235596 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.235958 26378 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:00.955726 25915 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.236572 22003 raft_consensus.cc:2410] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:07:01.237550 21758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 191) took 23272 ms (client timeout 15000 ms). Trace:
I20260709 13:07:01.237618 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:00.971496 22584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:01.230743 25857 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:00.976802 26215 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.986004 26138 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.240963 26215 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:00.987931 26110 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.241364 26138 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.000145 26174 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.001588 26218 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:01.006206 26205 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.030296 26250 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.029541 26216 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.039481 26223 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.054386 26271 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:01.085526 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.245801 26216 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:01.246243 22002 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:01.247440 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.248376 22002 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:07:01.036825 26234 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.092734 26088 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.756248 21854 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:01.094533 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.079358 26273 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:00.756981 26357 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.093819 25898 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:07:00.764127 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.087783 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.121701 22000 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:01.128731 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:01.129972 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:01.135110 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.136535 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.140367 22592 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:07:01.140920 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 443895us
I20260709 13:07:01.142280 22589 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:07:01.148041 26366 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:01.088928 26254 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:01.154335 26288 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.155119 21929 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:01.140460 26114 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:01.155529 26364 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.156243 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.140177 26260 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.147157 26352 raft_consensus.cc:2764] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:01.157496 26371 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.182741 26387 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:01.184079 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 326229us
I20260709 13:07:01.185192 26322 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.187754 21756 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 6 to 7, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 7 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:01.196768 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.204854 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 94922.1us
I20260709 13:07:01.200322 26235 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.217554 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 125722us
I20260709 13:07:01.217841 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2500) took 1797 ms. Trace:
I20260709 13:07:01.227958 26376 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:00.893260 26374 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.383227 22586 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:01.385317 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.386377 22587 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:01.087759 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.249300 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.387324 26234 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:01.387610 22002 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:07:01.387882 26234 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:01.388902 26234 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.203939 25846 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.238819 22584 raft_consensus.cc:606] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:01.385207 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:01.388979 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:01.393535 22002 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:01.394912 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.396588 21756 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:01.435005 26110 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:01.435787 25932 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.435349 21756 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 5 to 8, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 8 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:01.381348 22004 rpcz_store.cc:276] 0709 13:06:59.419974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.107223 (+1687249us) service_pool.cc:224] Handling call
0709 13:07:01.107609 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.107882 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:01.108387 (+   505us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.217084 (+108697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:01.217206 (+   122us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:01.217464 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.217546 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.217766 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":941,"prepare.queue_time_us":3346,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":7005,"raft.queue_time_us":3144,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":852,"thread_start_us":107112,"threads_started":1}
I20260709 13:07:01.398912 26312 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.436852 26312 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.437564 22004 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:01.438298 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2501) took 1503 ms. Trace:
I20260709 13:07:01.438618 22004 rpcz_store.cc:276] 0709 13:06:59.934523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.437019 (+1502496us) service_pool.cc:224] Handling call
0709 13:07:01.437376 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.438020 (+   644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.438234 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:01.439734 26223 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:01.391405 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 245795us
I20260709 13:07:01.440346 26223 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.397398 26114 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.440950 21900 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:01.442523 26223 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.402570 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.402977 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:01.444547 26271 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:01.445194 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.445631 26271 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:01.441804 26312 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.446199 26271 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:01.446975 26312 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.447144 26271 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.404340 22009 raft_consensus.cc:2410] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:07:01.448951 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.450946 22009 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:07:01.451088 26088 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.451330 22009 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:07:01.451625 22009 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:07:01.404938 26250 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:01.436378 25932 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.405524 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.238389 22003 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:01.452891 22006 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
W20260709 13:07:01.407974 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 279398us
I20260709 13:07:01.453963 26110 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:01.454285 22006 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:01.454963 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 885) took 1021 ms. Trace:
I20260709 13:07:01.455312 22006 rpcz_store.cc:276] 0709 13:07:00.433416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.453651 (+1020235us) service_pool.cc:224] Handling call
0709 13:07:01.453955 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.454697 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.454900 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:01.455093 26110 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.456470 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:01.457647 22006 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:01.409507 22588 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:01.405099 25846 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.411484 26376 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.459442 26376 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:01.460114 26376 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.411782 26260 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.412505 26288 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.473026 22592 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:01.473340 26288 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.473819 26376 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:01.474464 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:01.475572 26288 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.448511 26271 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:01.477118 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.413897 26366 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.238422 25915 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.419378 26364 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.421394 22008 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:01.423339 26371 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:01.425598 26273 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:01.426666 26322 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:01.427668 22001 raft_consensus.cc:2410] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:07:01.514139 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:01.433889 21854 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:01.397722 26235 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:01.400907 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2011) took 2595 ms (client timeout 1671 ms). Trace:
W20260709 13:07:01.515313 21929 rpcz_store.cc:269] 0709 13:06:58.805262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.806621 (+  1359us) service_pool.cc:224] Handling call
0709 13:07:01.400691 (+2594070us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b54002a8354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.400828 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5371568512}
I20260709 13:07:01.442654 26138 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:01.442916 26114 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.392169 22584 raft_consensus.cc:614] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:01.443007 26365 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:01.445468 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.447482 26312 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.451778 26254 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.381485 26374 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.457654 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:01.541436 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 83849.6us
I20260709 13:07:01.540338 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.542462 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:01.542887 21933 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:01.451844 22009 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:01.465919 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:01.467283 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.545763 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.546795 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.513643 26130 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.470798 25846 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.556217 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.557413 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2507) took 1258 ms. Trace:
I20260709 13:07:01.558076 22009 rpcz_store.cc:276] 0709 13:07:00.299144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.450232 (+1151088us) service_pool.cc:224] Handling call
0709 13:07:01.450585 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.556854 (+106269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.557311 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:01.471771 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.471880 25932 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.472203 26260 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:01.472491 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77673.9us
I20260709 13:07:01.335405 26338 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.416334 22002 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:01.568320 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.432700 26357 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.501178 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.527529 26288 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.504369 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.498078 22006 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:01.508198 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.508468 21935 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:01.577591 21930 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:01.578130 22007 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:01.510840 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.536926 26138 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.518020 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:01.519153 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:01.499614 26387 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:01.522930 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2406) took 2167 ms. Trace:
I20260709 13:07:01.523080 22004 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:01.524777 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.529810 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 888) took 1095 ms. Trace:
I20260709 13:07:01.532496 22591 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:01.589727 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.589999 22008 rpcz_store.cc:276] 0709 13:06:59.355860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.672119 (+316259us) service_pool.cc:224] Handling call
0709 13:07:01.522680 (+1850561us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400166554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.522861 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3532510464}
I20260709 13:07:01.531162 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.478741 25898 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:01.538388 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.594097 21927 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 7.
I20260709 13:07:01.594521 21929 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:07:01.404176 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 274274us
I20260709 13:07:01.596654 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.465695 22000 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:07:01.597376 22593 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:01.544490 22587 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:01.598754 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.551609 26114 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:01.608445 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.548473 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:01.561589 26234 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.566993 26088 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:01.583030 25932 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.535260 26364 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:01.512159 21932 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:01.642728 22593 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:07:01.579519 21931 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:01.600756 26387 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.580557 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.652415 21854 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:01.578671 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:01.574877 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.653558 21935 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:01.600119 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.601578 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:01.654302 21852 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 7.
W20260709 13:07:01.654579 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53149.7us
I20260709 13:07:01.590680 22001 rpcz_store.cc:276] 0709 13:07:00.434267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.684846 (+250579us) service_pool.cc:224] Handling call
0709 13:07:01.529432 (+844586us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54001e1ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.529699 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788815616}
I20260709 13:07:01.610141 26114 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.589305 21930 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:01.605759 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.556066 25915 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:01.606559 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2505) took 1670 ms. Trace:
I20260709 13:07:01.606812 22009 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:07:01.606961 22003 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:01.608376 21857 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:01.513803 26265 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:01.567092 26250 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:01.679394 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.610327 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.671511 22591 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:01.610639 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.680790 22588 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:01.570595 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.681926 22584 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:07:01.611451 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 63162.5us
W20260709 13:07:01.238113 21758 rpcz_store.cc:269] 0709 13:06:37.964820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.968239 (+  3419us) service_pool.cc:224] Handling call
0709 13:07:01.237374 (+23269135us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:56.071666 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:56.072380 (+   714us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:56.072437 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:06:56.072473 (+    36us) tablet.cc:662] Decoding operations
0709 13:06:56.478054 (+405581us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:56.487275 (+  9221us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:56.514244 (+ 26969us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:06:56.915310 (+401066us) tablet.cc:701] Row locks acquired
0709 13:06:56.953380 (+ 38070us) write_op.cc:260] PREPARE: finished
0709 13:06:56.974022 (+ 20642us) write_op.cc:270] Start()
0709 13:06:57.016129 (+ 42107us) write_op.cc:276] Timestamp: P: 1783602416963328 usec, L: 0
0709 13:06:57.017402 (+  1273us) op_driver.cc:348] REPLICATION: starting
0709 13:06:57.096638 (+ 79236us) log.cc:844] Serialized 8250 byte log entry
0709 13:06:58.270616 (+1173978us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b18000c1cc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:06:58.721200 (+450584us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:06:59.636093 (+914893us) op_driver.cc:464] REPLICATION: finished
0709 13:06:59.855246 (+219153us) write_op.cc:301] APPLY: starting
0709 13:06:59.855445 (+   199us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:00.134591 (+279146us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:00.134666 (+    75us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:00.155192 (+ 20526us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:00.155666 (+   474us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:07:00.155737 (+    71us) write_op.cc:312] APPLY: finished
0709 13:07:00.156439 (+   702us) log.cc:844] Serialized 180 byte log entry
0709 13:07:00.156913 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:00.157237 (+   324us) write_op.cc:454] Released schema lock
0709 13:07:00.157345 (+   108us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:00.157938 (+   593us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:00.157977 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7126577,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":169686,"apply.run_cpu_time_us":40051,"apply.run_wall_time_us":309223,"cfile_cache_hit":78,"cfile_cache_hit_bytes":81627,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6220,"cfile_init":2,"lbm_read_time_us":31514,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":9,"num_ops":20,"prepare.queue_time_us":192491,"prepare.run_cpu_time_us":25767,"prepare.run_wall_time_us":1260932,"raft.queue_time_us":430646,"raft.run_cpu_time_us":10894,"raft.run_wall_time_us":258608,"replication_time_us":2618491,"spinlock_wait_cycles":20762624,"thread_start_us":767511,"threads_started":4,"wal-append.queue_time_us":160204,"wal-append.run_cpu_time_us":47177,"wal-append.run_wall_time_us":2887024}]]}
I20260709 13:07:01.614442 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:01.482085 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.555456 26130 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.570926 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:01.613288 21756 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 7 to 10, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 10 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:01.684017 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 113255us
I20260709 13:07:01.684446 22004 rpcz_store.cc:276] 0709 13:06:59.936170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.439375 (+1503205us) service_pool.cc:224] Handling call
0709 13:07:01.522867 (+ 83492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.590202 (+ 67335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.606330 (+ 16128us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b5400166550. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.606491 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191214976}
I20260709 13:07:01.617779 22590 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:01.616950 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.632265 26223 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:01.617609 26174 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:01.590147 26138 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:01.431460 26218 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.644970 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.642931 22587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4021839a030d44ab886833307930ad33"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:01.650569 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:01.591171 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66497.7us
I20260709 13:07:01.716886 22587 raft_consensus.cc:606] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:01.702543 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.583801 26288 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.686187 21858 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:07:01.617805 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.778293 21853 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:01.683634 22592 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:01.644346 26260 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.642717 26088 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.697208 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:01.789034 21857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2365) took 2030 ms (client timeout 2088 ms). Trace:
I20260709 13:07:01.612453 26205 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 13:07:01.789806 21857 rpcz_store.cc:269] 0709 13:06:59.758619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.758916 (+   297us) service_pool.cc:224] Handling call
0709 13:07:01.788772 (+2029856us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.788939 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3959958784}
I20260709 13:07:01.711433 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.715939 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.716095 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:01.794279 22030 debug-util.cc:398] Leaking SignalData structure 0x7b080032d380 after lost signal to thread 20749
I20260709 13:07:01.794806 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.795789 26130 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.577744 25846 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.796139 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.718704 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.797129 26254 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:01.798420 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 79790.5us
I20260709 13:07:01.798606 25846 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.661662 26215 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:01.799405 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:01.678687 26114 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:01.774981 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.635776 26234 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.803421 26234 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.803611 21934 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:01.793051 22584 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:01.805433 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.806126 26215 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:01.806486 22593 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:01.806628 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:01.807457 22584 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:01.644191 26364 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:01.807307 26215 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.808027 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.768405 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.780824 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:01.808933 22588 raft_consensus.cc:2368] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9.
I20260709 13:07:01.694219 26110 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:01.809968 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.810336 26110 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.810564 26234 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:01.810834 26110 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:01.717633 26174 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.811645 26174 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:01.687983 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:01.813481 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 125635us
I20260709 13:07:01.782948 21898 leader_election.cc:400] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:07:01.814169 26234 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:01.814941 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.815459 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:01.808310 26364 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:01.681445 26250 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:01.817240 26250 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:01.818742 26376 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:01.818859 26398 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:01.819406 26376 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:01.819988 26376 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:01.654938 25932 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:01.784020 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.716929 26370 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:01.786824 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.821936 26234 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:01.787072 22585 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:01.813293 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.823127 25932 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.823351 26235 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:01.825505 26088 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.693286 25915 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.775080 21935 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:01.789150 21926 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:01.704711 21930 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:01.791482 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.829154 21935 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:01.830873 21858 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:07:01.833446 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.787424 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 884) took 1354 ms. Trace:
I20260709 13:07:01.834766 22003 rpcz_store.cc:276] 0709 13:07:00.433064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.453165 (+1020101us) service_pool.cc:224] Handling call
0709 13:07:01.584751 (+131586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.687255 (+102504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.787185 (+ 99930us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54001982d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:01.787358 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301724544}
I20260709 13:07:01.694779 26265 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.846225 21852 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:01.846722 25915 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.847792 26265 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.848069 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.848405 26338 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.848959 21853 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:01.849681 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.851150 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.851850 21855 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:01.851912 22584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:01.852597 22584 raft_consensus.cc:606] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:01.848177 26364 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.799902 26288 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.853031 22584 raft_consensus.cc:614] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:01.800278 26222 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:01.805541 25846 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:01.809777 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:01.813657 26371 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:01.683894 26277 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:01.863298 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 53593.8us
W20260709 13:07:01.795898 22030 debug-util.cc:398] Leaking SignalData structure 0x7b0800341c60 after lost signal to thread 21792
W20260709 13:07:01.866693 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08002dec60 after lost signal to thread 21884
W20260709 13:07:01.868294 22030 debug-util.cc:398] Leaking SignalData structure 0x7b080039d760 after lost signal to thread 21959
I20260709 13:07:01.837563 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:01.618685 26312 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.839107 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.790252 26260 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:01.881584 26400 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:07:01.881729 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.882696 26364 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:01.882855 26383 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.883484 26383 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.842773 21858 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:01.882699 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:01.885484 21853 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:01.822521 26234 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:01.852975 21860 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:01.834206 22592 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:07:01.891682 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:01.856778 22593 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:01.658104 26387 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:01.854662 25846 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:01.876088 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08003115a0 after lost signal to thread 22033
I20260709 13:07:01.878015 21928 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:02.196681 26271 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.009s
I20260709 13:07:01.847805 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.247815 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.249297 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:02.250061 26403 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:07:02.251058 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:01.897490 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.251906 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2418) took 1075 ms. Trace:
I20260709 13:07:02.252599 21851 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
W20260709 13:07:02.253494 26420 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:07:02.253958 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:02.255154 26417 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.003s
I20260709 13:07:01.896476 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:01.858587 26398 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:01.884450 26390 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:07:01.946434 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:01.854859 26357 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:01.879231 26322 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.882699 26260 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:01.977572 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 153445us
I20260709 13:07:02.273419 26322 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:01.880038 26265 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:07:01.885408 26277 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.280755 26277 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:01.991783 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 308560us
I20260709 13:07:02.281320 26398 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:01.949468 26399 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:02.283200 26399 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.283672 26399 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:02.284024 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:02.285943 26414 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:07:02.041505 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:02.289312 26410 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.114s	user 0.000s	sys 0.002s
W20260709 13:07:02.153159 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.289983 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.290532 21860 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:02.291435 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.292064 21860 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:02.165714 22586 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:02.167057 21931 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:07:02.293920 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.295185 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.170531 26138 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.173358 26374 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.174465 25846 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.297014 25846 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.297581 26138 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.298560 26404 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:02.177525 26383 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:02.298935 26138 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.178404 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:02.299819 22591 raft_consensus.cc:606] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:02.300899 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.301314 21932 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:02.180846 26234 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:01.885255 26222 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.183960 21756 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 5 to 7, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 7 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.182228 21934 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:02.304668 21756 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.306691 21756 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 2 to 5, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 5 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.184185 22592 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:02.184809 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.180428 26376 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.309334 21933 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:02.299556 26138 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.310642 26376 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:02.189152 26250 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.310989 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.191531 26114 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.190596 26174 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.312682 26250 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.190788 26419 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:02.313813 26174 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.195094 26418 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.314451 21855 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.314604 26418 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.316025 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.316674 21855 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:02.202277 26110 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.205163 26371 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:02.317294 21856 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.317802 26371 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.318533 26110 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.318724 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.320375 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.207123 21928 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:02.205240 26273 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:02.209545 22030 debug-util.cc:398] Leaking SignalData structure 0x7b080039bf80 after lost signal to thread 22617
I20260709 13:07:02.321298 26273 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:02.322250 22030 debug-util.cc:398] Leaking SignalData structure 0x7b080014b560 after lost signal to thread 26314
I20260709 13:07:02.323498 26273 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:02.209312 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 344577us
I20260709 13:07:02.324743 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.325399 21855 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:02.326072 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:02.327368 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:02.327322 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.328469 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:02.211033 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:02.328886 21855 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:02.329826 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.213817 26215 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.214100 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:02.216104 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.349327 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.311914 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.223057 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.350920 22588 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:02.224624 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.226246 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.221371 26387 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.229537 22009 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:07:02.359495 21930 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:02.360848 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:02.232982 26416 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:07:02.233136 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:02.364964 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:02.366026 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 132976us
I20260709 13:07:02.236048 26422 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:02.239464 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:02.240384 26394 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 13:07:02.311888 26138 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:02.243129 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.247128 26235 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:02.252302 22007 rpcz_store.cc:276] 0709 13:07:01.176958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.585915 (+408957us) service_pool.cc:224] Handling call
0709 13:07:01.586198 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.611397 (+ 25199us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:01.642529 (+ 31132us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.189219 (+546690us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.189789 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.190127 (+   338us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:02.190174 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.190415 (+   241us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.215017 (+ 24602us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b18001f47a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:02.249396 (+ 34379us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:02.251665 (+  2269us) raft_consensus.cc:1685] finished
0709 13:07:02.251688 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.251840 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15734,"prepare.queue_time_us":37515,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":133948,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":27779,"spinlock_wait_cycles":43316352,"thread_start_us":502429,"threads_started":2,"wal-append.queue_time_us":512612}
I20260709 13:07:02.256639 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.377789 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.378397 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.378602 22586 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:01.903507 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.258474 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.263139 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.264380 21852 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:02.288884 26412 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.117s	user 0.002s	sys 0.000s
I20260709 13:07:02.382833 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.289852 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.166814 26364 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.385272 26364 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.366029 26277 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:02.295571 26409 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:02.298593 26205 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:02.178333 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.372603 26273 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.377338 26250 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:02.299513 25582 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:02.311794 26383 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:02.408955 26383 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:02.409489 26383 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.313501 26419 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.414779 26419 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:02.190971 21972 connection.cc:256] RPC call timeout handler was delayed by -1.735576437000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 13:07:02.415359 21972 connection.cc:256] RPC call timeout handler was delayed by -1.7337764370000741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
W20260709 13:07:02.416131 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1434255249999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:07:02.417032 21853 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:02.417714 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:02.418723 25582 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:02.419215 25582 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:02.419549 25582 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:02.420320 25582 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.211150 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.334141 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:02.426092 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 92039.6us
I20260709 13:07:02.392843 26277 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:02.387046 26222 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:02.427408 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:02.428664 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:02.330224 21859 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:02.292909 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.337765 21856 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.332315 26114 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:02.426548 26138 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:02.431567 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2408) took 2977 ms. Trace:
I20260709 13:07:02.431361 26422 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:02.308302 21756 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.341120 21855 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:07:02.430799 21854 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.431367 26138 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.433089 21855 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:02.433434 21854 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:02.432934 21756 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.434315 21855 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:02.343662 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.434401 21756 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 2 to 7, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 7 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:02.431908 22005 rpcz_store.cc:276] 0709 13:06:59.454395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.408821 (+1954426us) service_pool.cc:224] Handling call
0709 13:07:01.409212 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.468492 (+ 59280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.468528 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.473113 (+  4585us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.577563 (+104450us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:01.577612 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.578751 (+  1139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.239677 (+660926us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:02.428105 (+188428us) raft_consensus.cc:1685] finished
0709 13:07:02.428129 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.431501 (+  3372us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2884,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":38274816,"thread_start_us":104021,"threads_started":1,"wal-append.queue_time_us":388226}
I20260709 13:07:02.349325 22585 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:02.436190 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.436100 21756 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.228549 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.436918 21928 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:02.437144 26277 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.437440 22004 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.437884 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.431862 21860 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:02.438270 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.438458 21928 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:02.437611 21756 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:02.348639 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:02.357872 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.439945 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:02.276754 26322 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.441334 26322 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:02.440291 21756 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 4 to 5, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 5 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:02.359447 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.362505 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.443118 26387 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:02.443354 22587 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:02.436957 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.444180 22585 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:02.444351 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.364857 22003 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:02.233675 26216 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:02.234534 26423 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:02.367115 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.367707 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.369287 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.369865 21932 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:02.369750 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:02.371620 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.372534 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.243083 25898 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:02.374446 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.374636 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.372534 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.375986 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.380004 21857 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.380476 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.381079 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.382139 21858 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.375633 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.383415 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:02.382411 21825 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.384203 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.170259 26218 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:02.366230 26377 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:02.367992 26288 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:02.388942 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.392366 26364 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:02.387420 26174 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:07:02.444837 26387 raft_consensus.cc:993] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:07:02.439386 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.446375 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:02.495512 22004 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.376624 26427 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:02.495597 21929 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:02.431910 21856 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 8. Current term is 10. Ops: []
I20260709 13:07:02.496258 26427 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.496732 26427 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.445514 21859 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:02.436184 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.498283 22005 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:02.498625 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.499745 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.500461 22589 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:07:02.501329 22000 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:02.445093 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.501863 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.502180 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.443420 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.432055 26422 consensus_queue.cc:1243] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:02.396055 26205 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.503925 26205 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.299309 26404 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:02.445930 21852 raft_consensus.cc:1217] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[10.9-14.11]   Dedup: 10.9->[12.10-14.11]
I20260709 13:07:02.505170 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:02.505906 21852 pending_rounds.cc:85] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 9
I20260709 13:07:02.506307 21852 pending_rounds.cc:107] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:07:02.505793 26404 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:02.506547 21852 raft_consensus.cc:2889] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:02.417570 26357 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:02.508550 26357 raft_consensus.cc:1081] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
W20260709 13:07:01.886838 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49398.1us
I20260709 13:07:02.508658 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.509277 21931 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:02.509512 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:02.509956 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.510224 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.511318 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.503003 22585 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEARNER]: Advancing to term 11
I20260709 13:07:02.513790 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.514370 22006 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 8.
I20260709 13:07:02.515146 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.515686 22006 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:02.516160 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.516419 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.516770 21926 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:02.511515 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:02.517474 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88902.7us
I20260709 13:07:02.517627 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.447515 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.383486 26418 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.504886 21930 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:02.519829 21927 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:02.439327 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.520445 21928 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:02.520830 22588 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:02.521525 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 151841us
I20260709 13:07:02.521672 21928 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:07:02.522410 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:02.494850 22590 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:02.468779 26271 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:02.444873 22587 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.424034 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:02.401602 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.524593 22593 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:01.798424 26254 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:02.518137 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.510879 22007 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:02.365779 26215 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.440737 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.538792 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.393473 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.532317 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:02.503158 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.530313 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.540565 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.541172 22007 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.493321 22008 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:07:02.442420 21756 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 4 to 7, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 7 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.543766 21756 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 4 to 9, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:02.533404 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 151050us
I20260709 13:07:02.502983 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.545890 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:02.546200 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 122243us
I20260709 13:07:02.511449 26218 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:02.512681 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.551910 21931 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:02.542302 22000 raft_consensus.cc:2368] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:07:02.544665 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:02.552276 21756 catalog_manager.cc:5697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:02.550563 21933 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.554747 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:02.533980 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:02.546456 21930 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:02.562369 21935 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:02.564898 22005 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:02.572078 22005 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:02.592810 21929 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:02.548228 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.603454 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:02.604019 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55836.9us
I20260709 13:07:02.605700 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:02.608333 22004 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:02.609397 22008 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:02.609324 21927 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:02.615247 21933 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:02.621157 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.623248 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.624318 22592 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:02.625496 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:02.627035 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.627832 22588 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:02.629549 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.629905 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:02.630991 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:02.468472 25582 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.447088 26273 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:02.426662 26374 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.531428 26271 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:02.640718 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.520077 26174 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:02.645829 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.646245 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:02.457885 25846 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:02.647328 21851 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:02.647696 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:02.649137 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43509.3us
I20260709 13:07:02.519073 26288 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.545547 26088 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:02.545534 26364 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:02.434443 26138 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:02.602910 26352 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:02.656086 21756 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 6 to 9, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 9 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:02.552696 26218 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:02.538700 26110 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:02.538753 25915 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:02.539297 26222 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:02.708411 21932 raft_consensus.cc:2376] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11. Re-sending same reply.
I20260709 13:07:02.552117 26418 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:02.649753 26408 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:02.655405 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.660082 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.761746 22009 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:02.762410 22006 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:02.711129 25846 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.504853 26427 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:02.715380 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69242.4us
I20260709 13:07:02.714495 26174 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:02.764299 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:02.711413 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:02.722548 22585 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:07:02.709489 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:02.725973 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 171305us
I20260709 13:07:02.731271 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:02.638870 26216 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:07:02.736871 22593 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:02.571897 26383 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.774950 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:02.604593 26365 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:02.741268 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 207362us
I20260709 13:07:02.765841 22585 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 6
I20260709 13:07:02.790326 22585 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:07:02.790596 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.790850 22585 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:02.791718 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.711982 26288 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.711249 26271 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:02.720691 26364 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.714571 26425 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:02.712292 26088 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:02.758386 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 359444us
I20260709 13:07:02.811998 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.812789 21856 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:02.759495 21926 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:02.816434 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:02.546298 26398 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:02.650835 26376 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.718021 26352 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:02.862251 26352 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:02.659443 26371 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.604745 26419 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.202s
I20260709 13:07:02.728511 26110 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.869508 26110 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.666412 22588 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:02.655432 26277 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.875764 26277 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:02.790223 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.877488 21858 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:02.783175 26387 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:02.741778 25915 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:02.753110 26222 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.807408 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:02.892443 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 85117us
I20260709 13:07:02.893298 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:02.895596 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:02.637395 26415 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 13:07:02.810029 21851 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:02.757864 26138 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:02.822304 26088 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.811240 26425 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:02.823972 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.951200 21853 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:02.753425 26418 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:02.712713 26374 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:02.827623 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 288854us
I20260709 13:07:02.829519 22590 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:02.831238 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:02.757993 26223 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:02.772166 25846 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:02.795548 26402 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:07:02.834115 26439 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:02.835460 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:02.825146 26436 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:02.839956 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:02.806380 26364 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:02.841984 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.842330 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.851450 26114 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:02.878583 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:02.752930 26260 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:02.755182 26409 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:02.907845 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:02.909765 22586 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
I20260709 13:07:03.060909 22586 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:03.061297 21932 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:02.912065 21851 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:07:02.915747 22007 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:02.822386 26398 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.885161 26376 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:02.972219 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:03.079885 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 107808us
I20260709 13:07:02.977856 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.979058 21900 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:02.811638 26322 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 13:07:03.033895 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.028805 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:03.025043 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 260823us
I20260709 13:07:03.029378 21929 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:02.938881 26277 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:02.976152 26425 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:03.039327 26439 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.910176 26222 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:02.997383 26250 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:02.979933 26273 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:03.003656 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.068148 26114 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:02.891978 25915 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.935683 26138 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.041839 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:02.911324 26218 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.042578 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.043412 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:02.970886 26205 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.093250 26444 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:02.937455 26423 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.043264 26130 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.045178 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.046592 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:03.048092 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.049893 21933 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:02.880002 26357 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:07:02.879055 25898 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:07:03.058012 22002 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:03.058272 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.186827 21927 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:02.821101 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:03.067252 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.187086 21930 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:03.195964 22586 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:02.982112 26418 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:03.069123 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 161433us
I20260709 13:07:03.197605 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.067025 26419 raft_consensus.cc:2764] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:03.070730 26383 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:03.203148 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 153588us
I20260709 13:07:03.204008 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:03.071301 26428 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:07:03.072419 26446 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:03.073107 26374 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:03.077246 26215 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.078248 26427 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:03.229372 26446 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:02.884321 26174 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.230958 26174 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.057512 26260 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.233363 26260 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.094403 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.099974 22590 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:07:03.100317 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 390861us
I20260709 13:07:03.238161 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:03.049916 26457 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:03.068367 26409 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.078828 26398 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:03.239482 26409 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:02.983683 22006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:32904
I20260709 13:07:03.024698 26223 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.241539 21935 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.242882 26223 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:03.168368 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 189402us
I20260709 13:07:03.152273 21929 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:07:03.044055 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 208684us
I20260709 13:07:03.104388 26277 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:03.246527 26277 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.249339 26409 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.250336 21935 raft_consensus.cc:2410] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:07:03.175715 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 94547.7us
I20260709 13:07:03.254812 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.255795 22586 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:03.255764 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:03.171475 26138 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.221953 21856 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:03.210309 22587 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
I20260709 13:07:03.257289 22587 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:03.258764 26409 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.261529 22591 raft_consensus.cc:606] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.262090 22591 raft_consensus.cc:614] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:03.239369 26457 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.263057 26457 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.263686 26398 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.266691 26398 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.069530 26436 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:03.220937 26430 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:03.185592 26445 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.185530 26444 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:03.195335 26423 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.221517 26357 consensus_queue.cc:1243] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:07:03.222837 26435 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:07:03.315018 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59369.4us
W20260709 13:07:03.315935 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 221592us
I20260709 13:07:03.197736 26439 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.317442 22593 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
W20260709 13:07:03.318925 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80898.2us
W20260709 13:07:03.319495 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 73621.2us
W20260709 13:07:03.313380 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 109450us
I20260709 13:07:03.295190 26408 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:03.194756 26235 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
W20260709 13:07:03.334923 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 291606us
W20260709 13:07:03.324904 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 127400us
W20260709 13:07:03.331030 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 84751.3us
I20260709 13:07:03.333096 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.333848 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2425) took 1042 ms. Trace:
I20260709 13:07:03.386607 22002 rpcz_store.cc:276] 0709 13:07:02.291802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.387232 (+ 95430us) service_pool.cc:224] Handling call
0709 13:07:03.333682 (+946450us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b5400167454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:03.333789 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191714560}
I20260709 13:07:03.321797 26223 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:07:03.391824 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59639.1us
I20260709 13:07:03.356065 22590 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:03.374958 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:03.377525 21933 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:03.378803 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.383260 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:03.385288 21927 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:03.387784 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.393072 22590 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:03.396567 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.398196 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.390663 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.391132 21930 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:03.395092 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 62099.4us
I20260709 13:07:03.371407 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.327258 26423 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.394853 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:03.336452 26407 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.399405 21931 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.381358 26383 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.392576 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:03.402037 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:03.400911 21935 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:03.400887 21899 leader_election.cc:400] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
I20260709 13:07:03.404815 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:03.403625 22590 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.426704 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.419987 21899 leader_election.cc:403] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:07:03.418345 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47081.9us
I20260709 13:07:03.399827 26374 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:07:03.430989 26460 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.475135 26460 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:03.474858 26277 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.475559 26460 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:03.476612 26110 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.003s
I20260709 13:07:03.327380 26445 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.480230 21927 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:03.433952 26454 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10.
W20260709 13:07:03.434365 25582 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.000s
I20260709 13:07:03.435046 26376 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.483006 26376 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.434890 26446 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.484211 26446 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.484241 26445 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:03.441395 26442 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 13:07:03.325750 26265 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:07:03.478339 26444 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.486656 26444 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.486785 26265 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:03.487311 26265 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:03.394501 26408 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:03.417397 26456 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:03.401944 26436 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:03.450098 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 669) took 1286 ms. Trace:
W20260709 13:07:03.450294 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57807.2us
I20260709 13:07:03.387095 26427 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:03.452034 26205 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.356065 26273 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.412197 26114 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.452023 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.454021 26422 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:03.492434 26088 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.507s	user 0.003s	sys 0.006s
I20260709 13:07:03.401815 26387 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:07:03.455016 26130 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:03.455140 26271 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.001s
I20260709 13:07:03.399257 26451 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:03.402030 26322 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:03.455821 26174 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:03.455900 26439 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:03.457846 26437 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:03.435073 25932 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:07:03.320721 26430 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.388157 26235 consensus_queue.cc:1243] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:03.426728 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:03.421813 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:03.530794 21929 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:03.530872 21856 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:03.490485 26110 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.474107 21928 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:03.317281 26458 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:03.432240 26452 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:07:03.477142 26399 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.193s	user 0.012s	sys 0.000s
W20260709 13:07:03.477583 26288 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.652s	user 0.007s	sys 0.003s
W20260709 13:07:03.402974 26461 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
I20260709 13:07:03.374280 26357 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:07:03.433223 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:03.481277 21857 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:07:03.619153 26453 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.549s	user 0.005s	sys 0.004s
I20260709 13:07:03.488279 26265 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.491200 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:03.492192 21931 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
W20260709 13:07:03.489775 26371 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.000s
W20260709 13:07:03.491709 26352 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.004s
I20260709 13:07:03.493103 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:03.391753 26415 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:03.643184 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 150177us
I20260709 13:07:03.413291 26260 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.492689 21852 rpcz_store.cc:276] 0709 13:07:02.163892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.387967 (+224075us) service_pool.cc:224] Handling call
0709 13:07:02.388320 (+   353us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:02.507713 (+119393us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:02.508079 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 13:07:03.084227 (+576148us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:03.097986 (+ 13759us) log.cc:844] Serialized 52 byte log entry
0709 13:07:03.098223 (+   237us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:03.098274 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.163149 (+ 64875us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b5400259754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:03.163198 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.449749 (+286551us) raft_consensus.cc:1685] finished
0709 13:07:03.449878 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.450039 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9662,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":118719,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":4941,"spinlock_wait_cycles":131906816,"thread_start_us":575654,"threads_started":2,"wal-append.queue_time_us":370733}
I20260709 13:07:03.510057 21859 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
W20260709 13:07:03.512177 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60214.8us
I20260709 13:07:03.444820 26138 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.502595 26422 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:03.513662 26130 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.489979 25846 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.503435 26277 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.532351 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.514729 26322 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:03.532370 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:03.524377 25915 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.535897 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.725239 21855 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:03.725513 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 191252us
I20260709 13:07:03.503643 26250 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:03.536703 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 138636us
I20260709 13:07:03.518383 26205 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.502681 26425 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.726922 26265 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.511624 26088 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.738524 26088 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.532426 26174 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:03.739801 26174 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.742870 26479 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:03.534943 26439 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:03.743492 26479 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.744148 26439 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.514784 26271 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.745816 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2436) took 1166 ms. Trace:
I20260709 13:07:03.745952 26479 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.539951 26474 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.543516 26399 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.576480 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.747475 21935 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.748594 21926 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:03.585038 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:03.518105 26408 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.597079 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:03.749334 21933 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:03.590379 21930 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:03.599152 25898 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:03.604529 26218 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.542850 26457 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.599164 26430 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.617131 22585 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:03.760738 22585 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:03.761562 22585 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.762005 26457 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.762706 22585 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.762933 21854 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:03.525743 26437 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:03.544327 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 150520us
I20260709 13:07:03.490290 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:03.765286 26437 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.766039 21851 raft_consensus.cc:2410] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:03.746917 26478 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:03.628263 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 195129us
I20260709 13:07:03.766798 26478 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:03.767220 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.767738 26110 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.752044 25898 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:03.770542 25898 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.557389 26288 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.771723 25898 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.634107 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:03.638468 22590 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:03.559978 26423 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.641359 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:03.645121 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.656100 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.779122 22585 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
W20260709 13:07:03.680375 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.687523 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43634us
W20260709 13:07:03.627135 26364 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:07:03.689481 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:03.781700 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 92667.5us
W20260709 13:07:03.632316 26461 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:07:03.636703 26407 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.696637 26130 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.701504 26138 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.707415 26480 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.690564 26415 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.715585 26465 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:03.717882 26322 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.646065 26371 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:03.727102 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:03.767221 26478 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:03.727501 26418 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.835845 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:03.728142 21856 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:03.726421 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:03.794670 26377 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:03.734694 26260 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.692984 26422 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.734839 25846 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.746317 22009 rpcz_store.cc:276] 0709 13:07:02.579092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.762501 (+183409us) service_pool.cc:224] Handling call
0709 13:07:02.762932 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.794095 (+ 31163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.794148 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.834338 (+ 40190us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.097483 (+263145us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:03.097607 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.097788 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.568450 (+470662us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:03.745486 (+177036us) raft_consensus.cc:1685] finished
0709 13:07:03.745509 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.745748 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8441,"raft.queue_time_us":116065,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":9511,"spinlock_wait_cycles":21940096,"thread_start_us":262542,"threads_started":1,"wal-append.queue_time_us":235495}
I20260709 13:07:03.601414 26387 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:03.754171 26250 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.755200 26399 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.776240 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.852120 22589 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:03.776578 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.780313 21858 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:03.784797 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.633940 26451 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:03.789645 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.645701 26352 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.791849 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:03.858628 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73921.8us
I20260709 13:07:03.861056 21926 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:03.793388 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.794145 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:03.861534 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.801206 21932 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:03.801520 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:03.746749 26271 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.748968 26425 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:03.805270 22593 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:03.805737 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:03.875721 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:03.849922 26130 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:03.807885 22585 pending_rounds.cc:85] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 8
I20260709 13:07:03.814575 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:03.815716 21935 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
W20260709 13:07:03.790932 26222 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.003s
I20260709 13:07:03.499707 26427 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.822238 21927 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:03.777721 26365 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.868151 26277 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.790576 26436 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:03.899888 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:03.734424 25915 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.903781 22593 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:03.904584 26415 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:03.904744 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.800237 26407 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.905702 26407 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.905895 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:03.906467 22593 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.796938 26457 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:03.846642 22004 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:03.908089 26457 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:03.846928 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 119905us
I20260709 13:07:03.909698 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 903) took 1094 ms. Trace:
I20260709 13:07:03.909732 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.911891 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:03.840372 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 114036us
I20260709 13:07:03.913177 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.913370 26425 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:03.914069 26425 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:03.914407 26425 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:03.914767 26425 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.916051 26352 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.746806 26474 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.871758 26454 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:07:03.853201 22588 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:03.918542 26430 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.918736 26474 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.897961 26222 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.919401 26430 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:03.920468 26430 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:03.921355 26222 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.922114 26457 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:03.857451 22003 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:03.923997 26427 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.913177 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.925220 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2536) took 1394 ms. Trace:
I20260709 13:07:03.925719 26371 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:03.926167 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.901203 26277 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.926573 26449 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:03.927088 26277 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.927207 26449 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:03.929999 26483 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:03.931439 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.931689 22002 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:03.910090 22004 rpcz_store.cc:276] 0709 13:07:02.815768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.817450 (+  1682us) service_pool.cc:224] Handling call
0709 13:07:02.817699 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.908070 (+1090371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.909484 (+  1414us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540042bdd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:03.909640 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2365849984}
I20260709 13:07:03.933200 26371 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:03.933672 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:03.933877 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:03.934463 22004 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.935045 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:03.936410 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:03.849733 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:03.920864 26430 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:03.938890 21852 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:03.939121 26430 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.940038 26407 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.798715 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:03.940611 26430 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.869208 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:03.862502 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:03.869362 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.817637 26480 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.941167 22584 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
W20260709 13:07:03.948580 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 79580.7us
I20260709 13:07:03.861748 21931 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:03.949198 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.950230 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:03.840902 26465 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:03.850955 26110 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.806923 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:03.987125 21933 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:03.987090 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117808us
I20260709 13:07:03.884302 22585 pending_rounds.cc:107] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:07:03.988341 22585 raft_consensus.cc:2889] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:03.886026 22006 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:03.989506 22006 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.884028 26130 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.859267 26451 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:03.858352 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 133793us
I20260709 13:07:03.828572 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.004734 22000 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.004781 26480 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.000679 26365 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.895568 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:03.851137 26387 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:03.875146 26223 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:03.904964 26490 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:07:03.900722 26460 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.916736 26352 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:03.938046 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.018024 26223 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.034931 26223 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:03.795243 26364 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.036731 21934 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:04.037142 26364 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:03.951910 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:04.037822 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2530) took 1607 ms (client timeout 1756 ms). Trace:
W20260709 13:07:04.038226 21934 rpcz_store.cc:269] 0709 13:07:02.544303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.544437 (+   134us) service_pool.cc:224] Handling call
0709 13:07:04.037649 (+1493212us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002a8354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:04.037768 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411394304}
I20260709 13:07:04.038609 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.039762 22009 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:03.862896 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.039767 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.041282 21858 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:03.947861 21928 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:03.957085 22593 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:03.914654 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:04.043431 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 128838us
I20260709 13:07:03.869573 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:03.925581 22003 rpcz_store.cc:276] 0709 13:07:02.531208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.736268 (+205060us) service_pool.cc:224] Handling call
0709 13:07:02.736591 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.924407 (+1187816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.924973 (+   566us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003895d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:03.925163 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577228544}
I20260709 13:07:04.046116 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:03.968926 22002 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:04.046574 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.047737 22000 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:04.047912 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:03.980901 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:03.871025 21926 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:03.847836 26377 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:03.998224 21855 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:04.051628 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.051796 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.052603 22009 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:03.999408 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49245.1us
I20260709 13:07:03.982462 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:03.914638 26418 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.040498 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:04.063062 22003 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:03.986912 26110 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:04.006302 26404 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:04.030716 26455 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:07:04.019004 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.033396 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:04.001474 26130 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.085326 22006 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:03.986816 26479 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:04.090428 22005 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 16. Ops: []
I20260709 13:07:03.995057 21931 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:04.032222 26407 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:04.059242 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.064040 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:04.063555 26364 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.015756 22004 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:04.065794 22009 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:04.067397 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.075793 21858 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:04.077215 26383 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:04.112767 26383 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.115114 26383 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:04.032183 26488 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:07:03.918609 26423 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:04.081547 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 100739us
I20260709 13:07:03.986572 26374 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:07:04.087007 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53716us
I20260709 13:07:04.118453 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:04.120358 22000 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:07:04.088459 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 106227us
I20260709 13:07:04.121825 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2391) took 1294 ms. Trace:
I20260709 13:07:04.122310 21860 rpcz_store.cc:276] 0709 13:07:02.827426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.827640 (+   214us) service_pool.cc:224] Handling call
0709 13:07:04.121561 (+1293921us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:04.121755 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2203279744}
I20260709 13:07:04.122905 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:04.083770 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 64838.1us
I20260709 13:07:04.084744 26404 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.093218 26419 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.986357 26216 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:04.052623 22002 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:07:03.941672 26471 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.271s
I20260709 13:07:04.065135 26427 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:04.011422 21859 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:04.084338 26457 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:07:04.066355 25915 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:04.082111 26110 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.062656 26480 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:04.006019 26478 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.124819 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.164132 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:04.006088 26288 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:04.112655 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 246731us
I20260709 13:07:04.018438 26408 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:07:04.128520 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:04.033316 26365 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.098901 26399 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.098381 26479 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.099283 26250 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:04.101154 26352 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:04.092764 26446 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:04.105370 22590 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 13:07:04.124119 26436 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:04.134389 26404 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.078037 26474 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:07:04.033859 26387 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.200047 26277 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.134970 26088 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:04.031289 26465 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.171933 22009 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:07:04.176421 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47970.8us
I20260709 13:07:04.176617 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.141247 26271 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.148165 26449 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.151382 26423 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:04.123922 26409 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:07:04.033771 26451 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.149329 26357 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.183991 22590 pending_rounds.cc:85] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 9
I20260709 13:07:04.262840 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.263164 22590 pending_rounds.cc:107] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:07:04.263640 21928 raft_consensus.cc:2368] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:07:04.264564 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.150943 26216 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:04.191990 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.141758 26415 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:07:04.149106 26371 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:07:04.187055 26138 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.195415 26447 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 13:07:04.199134 26474 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:04.416754 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.417732 21927 raft_consensus.cc:2376] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12. Re-sending same reply.
I20260709 13:07:04.417824 26474 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.418550 26474 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.419162 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.200384 26454 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.421662 26457 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:04.421490 26454 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.424216 26454 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.159235 21853 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:07:04.204885 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 131972us
I20260709 13:07:04.427255 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:04.206154 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.420742 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.151141 25846 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:07:04.238658 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.188190 26352 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.242133 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.440111 21926 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:04.242976 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.441466 21931 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:04.243921 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.185582 26479 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:04.259994 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:04.263561 22590 raft_consensus.cc:2889] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:04.443143 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1050) took 1190 ms. Trace:
I20260709 13:07:04.443684 22590 rpcz_store.cc:276] 0709 13:07:03.253053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.253245 (+   192us) service_pool.cc:224] Handling call
0709 13:07:03.253637 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.355951 (+102314us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b5400670aa0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:07:03.637863 (+281912us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b580032f6d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7072b0aa 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4
0709 13:07:04.442791 (+804928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.443038 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609606400}
I20260709 13:07:04.130779 26425 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.186877 26374 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:04.195919 26489 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:07:04.259999 21929 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:04.270169 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.261762 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.270501 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.389792 26277 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.459415 21935 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 8: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:04.459429 21860 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.275738 21932 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:04.276546 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:04.277324 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109617us
W20260709 13:07:04.461135 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 184703us
I20260709 13:07:04.461973 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.277621 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.464180 21855 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 8: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:04.278488 22006 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.204675 26477 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.194044 26288 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:04.195524 26436 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:04.422183 26457 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:04.254622 26456 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.205633 26383 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:07:04.183681 26365 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.198076 26478 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.262338 26265 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.184492 26174 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:07:04.382745 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.451668 26374 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:04.191540 26446 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:04.261641 26453 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:07:04.469203 26377 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:04.366726 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 243861us
W20260709 13:07:04.522845 26431 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:07:04.411355 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.524256 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:04.524438 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 780) took 1033 ms. Trace:
I20260709 13:07:04.430778 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:04.524619 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.524878 21929 rpcz_store.cc:276] 0709 13:07:03.491633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.531583 (+ 39950us) service_pool.cc:224] Handling call
0709 13:07:04.524288 (+992705us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400228b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:04.524388 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123778816}
I20260709 13:07:04.367419 26433 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:04.529549 22589 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:04.450938 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:04.454200 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:04.454730 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.273808 22002 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:04.382380 26218 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:07:04.537068 22590 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:04.463210 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:04.185145 26250 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:04.478848 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 218903us
I20260709 13:07:04.537653 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.479838 22007 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:04.436621 26474 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.448401 26352 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:04.457342 26437 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.002s
I20260709 13:07:04.554000 26437 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:04.556773 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93636.3us
I20260709 13:07:04.389312 26130 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.511483 21854 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:04.182965 26480 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.438374 26427 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:07:04.451237 26235 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:07:04.468497 26460 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:04.367930 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.479147 26454 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.481570 26138 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:04.482313 26088 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:04.548594 26444 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.269184 26271 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.538295 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:04.538914 21860 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:04.519527 26265 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:04.619251 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81047us
I20260709 13:07:04.541312 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.507982 26415 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.547569 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:04.511138 26288 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:04.517644 26374 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:04.553254 26507 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.546502 26377 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:07:04.484892 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:32904 (request call id 905) took 1649 ms. Trace:
W20260709 13:07:04.557410 26497 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.385s	user 0.001s	sys 0.004s
I20260709 13:07:04.554953 26504 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.555364 26463 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.151s
W20260709 13:07:04.524531 26422 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 13:07:04.654241 21928 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
W20260709 13:07:04.576035 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 147315us
I20260709 13:07:04.576831 22590 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:04.525262 22001 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:04.478555 26216 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.545786 26250 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.549540 26419 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.618382 22592 raft_consensus.cc:2368] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:04.507846 26456 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.663908 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.630798 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.636974 21858 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:04.640429 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.641986 26454 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:04.641866 21854 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:04.553395 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:04.609474 26235 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:04.645561 22006 rpcz_store.cc:276] 0709 13:07:02.829540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.983516 (+153976us) service_pool.cc:224] Handling call
0709 13:07:04.249947 (+1266431us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400425ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707c6a3d 00007f3b75796cd0 00007f3b6ed1b254 00007f3b6ed1b1e2 00007f3b6ed1b111 00007f3b6ed1b08c 00007f3b6ed1a342 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:04.465421 (+215474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484200320}
I20260709 13:07:04.650243 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.651194 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.670493 22586 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.674021 21852 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:04.652078 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.669412 21859 raft_consensus.cc:2368] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:07:04.622288 26433 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.677757 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.657125 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.678401 21854 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:04.655596 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.679498 22590 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:07:04.664559 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:04.483558 26477 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.665045 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.653579 21930 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:04.682742 22585 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:07:04.682896 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:04.677541 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.682757 21932 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:04.684135 26504 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.684392 25898 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:04.672624 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.684738 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.684907 25898 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:04.685875 25898 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.664763 26422 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:07:04.689568 26500 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:07:04.690217 26422 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:04.690733 26495 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:04.550041 26457 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.669144 21860 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:04.669833 22584 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:07:04.670537 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 209565us
I20260709 13:07:04.672818 26419 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.665241 26218 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:07:04.683158 22585 pending_rounds.cc:85] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 8
I20260709 13:07:04.622550 26460 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:07:04.691680 22003 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:04.629498 25846 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:07:04.694500 22585 pending_rounds.cc:107] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:07:04.693922 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:04.695254 22585 raft_consensus.cc:2889] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:04.624528 26444 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.662148 26463 consensus_queue.cc:1243] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:04.698072 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.663890 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:04.666800 26371 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:04.708817 21929 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:04.713137 26454 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:04.694059 26433 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.690904 26422 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:07:04.716104 26457 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.666631 26509 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.090s
W20260709 13:07:04.759809 26404 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.000s
I20260709 13:07:04.760309 26404 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.716980 26288 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.718156 26271 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.718711 21852 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:04.719405 26130 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:04.666512 26377 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.722987 21859 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:04.722216 21856 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:07:04.737316 26322 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.896s	user 0.012s	sys 0.000s
W20260709 13:07:04.667076 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 105040us
I20260709 13:07:04.724103 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.726557 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:04.808387 26522 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:04.808872 26522 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:04.809190 26399 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.622s	user 0.007s	sys 0.004s
I20260709 13:07:04.809610 26399 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.809770 26522 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.811565 26522 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.761291 26425 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:04.648545 26507 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.815747 26387 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.612623 26436 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.728076 26088 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.817251 26524 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.818342 26524 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.818670 26322 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.762831 26265 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.763435 26365 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.763988 26456 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.848867 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.854568 21926 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:04.767189 26419 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:04.768180 26223 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.733s	user 0.006s	sys 0.005s
I20260709 13:07:04.769152 26437 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:04.769353 21744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 34f3b83da84448bf8aac62b37a1486eb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:04.716579 26446 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.771471 26433 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:04.771992 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.867362 22589 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
W20260709 13:07:04.773300 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 109494us
W20260709 13:07:04.861107 21744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 91820.9us
I20260709 13:07:04.773908 26449 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:04.774752 26138 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.769222 21854 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:04.780624 22586 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:04.780725 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:04.802747 26371 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:04.877295 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.878028 22590 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:04.799801 26460 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.781493 26427 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:07:04.724071 26440 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:07:04.837441 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 109629us
W20260709 13:07:04.838241 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 152308us
I20260709 13:07:04.847949 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.855705 21927 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:04.857511 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 131025us
I20260709 13:07:04.861406 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.864598 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.865814 21933 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:04.869390 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:04.853722 26399 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:07:04.798967 26423 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:04.788574 26523 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:04.879436 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:04.879719 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99274.4us
I20260709 13:07:04.886062 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.860903 26265 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.885360 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:04.887238 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:04.887238 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.887708 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:04.862061 26465 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:04.912469 21856 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.907802 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.894867 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:04.915511 21931 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:04.860342 26223 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.904680 21927 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:07:04.904523 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:04.923328 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.923894 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2400) took 1118 ms. Trace:
I20260709 13:07:04.905556 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.875439 26444 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:04.929016 26527 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:04.901856 26449 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:04.929636 26527 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:04.930094 26427 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:04.930394 26449 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.930680 26527 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.932272 26439 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:04.932657 26439 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:04.858314 26271 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:07:04.910157 26322 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:04.933393 26423 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.934514 26322 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.927709 26399 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.935212 26322 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:04.881120 26138 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.935875 26322 consensus_queue.cc:1243] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:04.936481 26322 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:04.930678 26427 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:04.937395 26138 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:04.938105 26427 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.939596 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.940379 22005 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:04.941279 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.942332 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:04.944584 26419 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:04.944985 26419 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.837913 26377 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:04.917495 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53262.7us
I20260709 13:07:04.918211 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:04.958283 21972 leader_election.cc:400] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:07:04.924642 22590 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:04.958743 21972 leader_election.cc:403] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:07:04.925388 21930 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:04.922050 26110 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.008s
I20260709 13:07:04.927680 26523 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:04.928485 26404 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.929328 26444 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:04.924238 21853 rpcz_store.cc:276] 0709 13:07:03.805781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.805910 (+   129us) service_pool.cc:224] Handling call
0709 13:07:04.156236 (+350326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.424886 (+268650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.923711 (+498825us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540018ecd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:04.923844 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723147648}
I20260709 13:07:04.933452 26439 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.897257 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.884460 26446 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:04.946408 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61145.2us
I20260709 13:07:04.988423 21823 leader_election.cc:400] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:07:04.988791 21823 leader_election.cc:403] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:07:04.948042 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.903681 21926 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:07:04.952062 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.958220 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:04.922590 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.965346 26480 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.964975 21856 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:04.990890 26446 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:04.996800 26446 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:04.997311 26456 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:04.997386 26415 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.998443 26535 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:07:04.999574 26535 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:04.999979 26535 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:04.973637 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:04.970134 26425 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 9. Current term is 10.
I20260709 13:07:04.936829 26322 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:04.902488 26457 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:07:04.976662 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:04.980896 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:04.981627 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.922675 26520 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:04.887990 26460 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:04.940579 26374 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.929332 26524 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:04.916100 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:04.993132 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:04.983629 21858 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:04.945371 26419 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.970501 26138 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:04.929644 26371 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.009944 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:05.001881 21859 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:07:04.995764 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:05.004948 26535 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:04.922878 26265 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:05.002445 26430 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.006s	user 0.009s	sys 0.006s
I20260709 13:07:04.901402 26454 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.027539 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:05.009030 26404 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:05.032648 26507 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:05.030162 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53586us
I20260709 13:07:05.031241 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:04.970613 26110 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.014223 26480 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:05.066253 25582 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:05.004288 22002 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:05.062651 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66967.9us
I20260709 13:07:05.064853 22009 pending_rounds.cc:85] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
W20260709 13:07:05.078521 26479 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.000s
I20260709 13:07:05.035565 26425 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:05.035255 26520 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:05.143725 26520 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.075548 22591 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:04.970373 26447 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:07:05.057188 26138 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.090349 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:05.061025 26524 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:05.117249 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.049074 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56037.1us
I20260709 13:07:05.125625 26110 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.127830 22004 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:05.129238 22009 pending_rounds.cc:107] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:05.154929 22009 raft_consensus.cc:2889] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:05.066035 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:05.132489 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:05.129640 26535 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.134393 26377 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:05.141875 26540 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:07:05.152711 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:05.130827 26532 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:07:05.156211 22006 raft_consensus.cc:2410] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:07:05.156275 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66065.1us
I20260709 13:07:05.069809 26352 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.160542 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:05.148839 26425 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:05.148048 26479 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.079612 26250 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.121361 26480 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.239240 26352 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.239758 26480 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.239883 26352 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.169534 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:05.241822 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 72412.7us
I20260709 13:07:05.154102 26138 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.072312 26322 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.172559 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:05.179616 26425 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.219635 26088 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:07:05.237057 26527 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.237576 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:05.246073 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:05.164983 26444 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:05.170347 26260 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:05.257423 22000 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:05.164763 26216 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:07:05.240620 26352 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.252982 26377 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:05.266158 26536 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 13:07:05.267222 26534 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:07:05.268033 26479 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.153710 26533 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.281s
I20260709 13:07:05.178645 26526 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:05.177552 26364 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:05.276974 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:05.072356 26523 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.260545 26480 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:05.352909 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 76050.9us
I20260709 13:07:05.256910 26447 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:07:05.251701 26530 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:07:05.288897 26546 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:05.339344 26377 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:05.396433 26537 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:07:05.399889 26541 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:07:05.357489 26430 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:05.286252 26527 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.361258 26364 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:05.374927 26460 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.426402 26374 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.439666 26265 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.499480 26250 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.456010 26377 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:05.521198 26522 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.003s
I20260709 13:07:05.492913 26364 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:05.259227 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.687558 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.251828 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 79408.3us
W20260709 13:07:05.567531 26387 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.008s
I20260709 13:07:05.564508 26404 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:07:05.576776 26418 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:05.575078 26419 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
W20260709 13:07:05.723872 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.723814 26418 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:05.724566 26419 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.726665 26418 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.587004 26425 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.591869 26439 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:05.576710 26446 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 13:07:05.566570 26512 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:05.744284 26512 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:05.744693 25898 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.745918 25898 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.746249 26512 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:05.595167 26511 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.136s	user 0.003s	sys 0.005s
W20260709 13:07:05.601207 26517 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.991s	user 0.005s	sys 0.018s
I20260709 13:07:05.569576 26527 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:05.563848 26552 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:07:05.557508 26352 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.752398 25898 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.755020 26523 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.756793 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:05.759341 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:05.721243 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:05.733423 26478 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.733788 26425 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.734434 26504 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.739267 26431 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:07:05.743072 26446 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.728646 26419 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.756186 26523 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.772336 26454 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.774237 26446 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.773718 26504 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.777957 22005 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:05.773763 26454 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:05.781834 26454 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.781978 26504 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.779393 26431 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:05.782624 26433 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.782806 26431 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:05.783720 26433 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:05.789500 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2545) took 1547 ms (client timeout 1793 ms). Trace:
W20260709 13:07:05.790194 21934 rpcz_store.cc:269] 0709 13:07:04.242108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.243632 (+  1524us) service_pool.cc:224] Handling call
0709 13:07:05.788939 (+1545307us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540017e154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:05.789240 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817310592}
I20260709 13:07:05.784873 26431 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:05.802680 22000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2452) took 1632 ms (client timeout 1660 ms). Trace:
W20260709 13:07:05.808410 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70551.1us
I20260709 13:07:05.810106 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:05.812633 26371 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:05.813282 26478 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.814019 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2547) took 2028 ms. Trace:
I20260709 13:07:05.814301 26371 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.814445 22003 rpcz_store.cc:276] 0709 13:07:03.785920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.100957 (+315037us) service_pool.cc:224] Handling call
0709 13:07:04.101324 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.696946 (+595622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.813513 (+1116567us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b540038a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.813830 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357141376}
I20260709 13:07:05.816627 26478 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.812526 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 910) took 2012 ms. Trace:
W20260709 13:07:05.803530 22000 rpcz_store.cc:269] 0709 13:07:04.170123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.170280 (+   157us) service_pool.cc:224] Handling call
0709 13:07:05.802190 (+1631910us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54002152d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:05.802433 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2291298176}
I20260709 13:07:05.808491 26110 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:05.824155 26110 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.827359 26110 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.828558 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2541) took 2033 ms. Trace:
I20260709 13:07:05.829510 26454 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.831198 26523 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.832086 26265 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.833586 26431 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.834908 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:05.838537 26265 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.843336 26371 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.839793 26454 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.840101 21927 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:05.835183 22007 rpcz_store.cc:276] 0709 13:07:03.784256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.057914 (+273658us) service_pool.cc:224] Handling call
0709 13:07:04.058216 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.538586 (+480370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.817567 (+1278981us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400169254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.817987 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969049856}
I20260709 13:07:05.852772 26273 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:05.853330 21934 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:05.820731 26512 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.855253 26130 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:05.855767 26130 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:05.857349 26265 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.853188 26454 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.821973 22004 rpcz_store.cc:276] 0709 13:07:03.799838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.109611 (+309773us) service_pool.cc:224] Handling call
0709 13:07:04.109920 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.154859 (+1044939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.811738 (+656879us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540042b3d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.812139 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340579584}
I20260709 13:07:05.877475 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:05.878650 21927 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:05.842444 26433 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.853468 26273 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:05.846698 26223 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.862528 26523 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.890273 26523 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.886742 26130 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.896332 26130 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:05.897243 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:05.899132 26433 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:05.899268 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:05.899611 26433 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.900039 26433 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.900601 26504 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:05.902387 26504 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:05.902792 26504 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.903216 26530 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.904003 26530 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.907028 26530 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.909579 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1008) took 1629 ms. Trace:
I20260709 13:07:05.910207 26273 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.911710 22007 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:07:05.911896 26273 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.910175 22593 rpcz_store.cc:276] 0709 13:07:04.280615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.280874 (+   259us) service_pool.cc:224] Handling call
0709 13:07:04.389216 (+108342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.413513 (+ 24297us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:04.446167 (+ 32654us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.609113 (+162946us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.616750 (+  7637us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.616933 (+   183us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:04.621880 (+  4947us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.622456 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.622528 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.568636 (+946108us) spinlock_profiling.cc:243] Waited 9.34 ms on lock 0x7b180031dbc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b756f8328 00007f3b756ef901 00007f3b756ef8a7 00007f3b757e7288 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508
0709 13:07:05.898290 (+329654us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b180031dbc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:05.909031 (+ 10741us) raft_consensus.cc:1685] finished
0709 13:07:05.909052 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.909305 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25697,"prepare.queue_time_us":37888,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":15040,"raft.queue_time_us":44275,"raft.run_cpu_time_us":100456,"raft.run_wall_time_us":974593,"spinlock_wait_cycles":56413312,"thread_start_us":161480,"threads_started":1,"wal-append.queue_time_us":205632}
I20260709 13:07:05.913506 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:05.918532 22007 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:07:05.917183 26404 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:05.919617 26404 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.920181 26404 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:05.921176 26110 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.921926 26110 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:05.923309 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2546) took 1680 ms (client timeout 1504 ms). Trace:
W20260709 13:07:05.923748 21933 rpcz_store.cc:269] 0709 13:07:04.242429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.261471 (+ 19042us) service_pool.cc:224] Handling call
0709 13:07:05.922895 (+1661424us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54002e8bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:05.923121 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814250752}
I20260709 13:07:05.917249 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:05.914582 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:05.925951 22004 raft_consensus.cc:2368] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16.
I20260709 13:07:05.926096 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2059) took 1279 ms. Trace:
I20260709 13:07:05.926594 21929 rpcz_store.cc:276] 0709 13:07:04.646937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.656735 (+  9798us) service_pool.cc:224] Handling call
0709 13:07:05.925723 (+1268988us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b540003cd54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:05.925919 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46454144}
I20260709 13:07:05.928467 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 782) took 1084 ms. Trace:
I20260709 13:07:05.934593 26422 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:05.934996 26288 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.935216 26422 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.935681 26288 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.937289 26371 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:05.938588 26371 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:05.939090 26371 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:05.942241 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2555) took 1525 ms. Trace:
I20260709 13:07:05.942543 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:05.945516 26404 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.946743 26265 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.947017 26138 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.688s	user 0.006s	sys 0.013s
I20260709 13:07:05.947646 26265 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.941653 26433 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.950059 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1057) took 1520 ms. Trace:
I20260709 13:07:05.950493 22585 rpcz_store.cc:276] 0709 13:07:04.429675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.429863 (+   188us) service_pool.cc:224] Handling call
0709 13:07:04.446341 (+ 16478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.713564 (+267223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.949579 (+1236015us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54006b3ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.949889 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542741376}
I20260709 13:07:05.951586 26433 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.928932 21932 rpcz_store.cc:276] 0709 13:07:04.843566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.843773 (+   207us) service_pool.cc:224] Handling call
0709 13:07:04.867467 (+ 23694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.867668 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:04.880534 (+ 12866us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.920858 (+1040324us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b180049b100. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a
0709 13:07:05.927843 (+  6985us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:05.927924 (+    81us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:05.928022 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.928125 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.928262 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16787,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":1417,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"spinlock_wait_cycles":17352960,"thread_start_us":31390,"threads_started":1,"wal-append.queue_time_us":180906}
I20260709 13:07:05.952901 22588 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:05.953410 22588 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:05.954785 26404 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.954495 22588 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:05.955618 22588 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:05.942689 22008 rpcz_store.cc:276] 0709 13:07:04.416275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.597171 (+180896us) service_pool.cc:224] Handling call
0709 13:07:04.603116 (+  5945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.973853 (+370737us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:05.018063 (+ 44210us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.125263 (+107200us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.125632 (+   369us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.126168 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.126347 (+   179us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:05.126381 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.941622 (+815241us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54000ca8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.941699 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.941845 (+   146us) raft_consensus.cc:1685] finished
0709 13:07:05.941860 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.942040 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34262,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":69,"raft.queue_time_us":40059,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":216,"spinlock_wait_cycles":396424960,"thread_start_us":279689,"threads_started":2,"wal-append.queue_time_us":98517}
W20260709 13:07:05.962905 26494 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.573s	user 0.000s	sys 0.009s
I20260709 13:07:05.964803 21860 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:05.965533 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 851) took 2190 ms. Trace:
I20260709 13:07:05.966774 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:05.967343 26503 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.557s	user 0.000s	sys 0.007s
I20260709 13:07:05.968459 26422 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.965961 22587 rpcz_store.cc:276] 0709 13:07:03.775032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.775154 (+   122us) service_pool.cc:224] Handling call
0709 13:07:03.803384 (+ 28230us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:03.803809 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.803857 (+    48us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:03.824120 (+ 20263us) log.cc:844] Serialized 78 byte log entry
0709 13:07:04.147418 (+323298us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:04.147456 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.147538 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.961945 (+1814407us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:05.964973 (+  3028us) raft_consensus.cc:1685] finished
0709 13:07:05.965039 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.965335 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4167,"raft.queue_time_us":101,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":8752,"spinlock_wait_cycles":315192576,"thread_start_us":287543,"threads_started":1,"wal-append.queue_time_us":343316}
I20260709 13:07:05.903224 26504 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.970629 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:05.940481 26371 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.924345 26110 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:05.920742 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:06.036548 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:05.993518 26415 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:06.000538 22001 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:07:06.045454 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 124999us
I20260709 13:07:06.045672 26415 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:05.976071 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:05.986678 22004 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:05.988682 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 914) took 1598 ms. Trace:
I20260709 13:07:05.892758 26523 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.019695 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.015805 22000 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:05.990975 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 861) took 1467 ms. Trace:
I20260709 13:07:06.036087 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.022504 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:05.985209 26477 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:05.979611 26288 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:05.978223 26451 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.051173 21851 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:06.004626 26277 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:07:06.016674 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2412) took 1126 ms. Trace:
I20260709 13:07:06.051719 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:06.003463 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.052119 21852 rpcz_store.cc:276] 0709 13:07:04.890180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.895426 (+  5246us) service_pool.cc:224] Handling call
0709 13:07:04.895673 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.880236 (+984563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.880296 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:05.880345 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:05.880382 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.016234 (+135852us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:06.016292 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.016474 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187720448}
I20260709 13:07:06.016214 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:05.989810 22592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1004) took 2187 ms (client timeout 1793 ms). Trace:
I20260709 13:07:05.979962 26407 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.975277 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:05.997232 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.053910 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.047209 26415 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:05.949978 26265 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.054770 22006 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:05.996188 26418 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.008159 26404 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:06.001710 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:05.979442 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2061) took 1330 ms. Trace:
I20260709 13:07:06.053643 22007 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:06.048887 26371 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:05.972911 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.009405 26322 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:05.973872 26508 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.450s	user 0.009s	sys 0.000s
I20260709 13:07:05.987735 22008 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 16. Ops: []
I20260709 13:07:06.055817 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.056568 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.056716 21935 rpcz_store.cc:276] 0709 13:07:04.649065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.650067 (+  1002us) service_pool.cc:224] Handling call
0709 13:07:05.979130 (+1329063us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54002e7cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:05.979281 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2281340288}
I20260709 13:07:06.083817 26216 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:06.084118 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.057799 22585 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:06.085605 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.086165 22587 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 10: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:06.058485 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.087121 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.055372 26415 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.058393 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.089534 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.049278 22584 rpcz_store.cc:276] 0709 13:07:04.523769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.530253 (+  6484us) service_pool.cc:224] Handling call
0709 13:07:04.530522 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.692176 (+161654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.990414 (+1298238us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54005631d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.990763 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315513216}
I20260709 13:07:06.056571 22009 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:06.082643 22593 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.059237 21858 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:06.066169 21852 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:06.060065 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:05.947568 26138 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:06.053498 22592 rpcz_store.cc:269] 0709 13:07:03.804019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.854349 (+ 50330us) service_pool.cc:224] Handling call
0709 13:07:05.989325 (+2134976us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54007300d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:05.989575 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694002560}
I20260709 13:07:06.065330 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.078274 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:06.071250 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61427.4us
I20260709 13:07:06.065176 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.056013 26404 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.099288 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.099397 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:06.099956 21853 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:06.100121 22586 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:06.047761 22002 rpcz_store.cc:276] 0709 13:07:04.390104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.548299 (+158195us) service_pool.cc:224] Handling call
0709 13:07:04.549227 (+   928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.128655 (+579428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.988062 (+859407us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b5400166a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:05.988471 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1045914496}
I20260709 13:07:06.100863 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.104490 26404 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:06.060614 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.115950 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.059854 26523 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.123380 21853 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:06.121264 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.126452 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.100943 26110 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:06.127878 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76228.2us
I20260709 13:07:06.136451 21928 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:06.128069 21854 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:06.145704 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:06.130944 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.148696 21935 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.149559 21860 raft_consensus.cc:2368] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:07:06.160008 22588 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:06.156855 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:06.161707 21851 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.135778 21856 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:06.163470 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.167099 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:06.184357 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.185025 21855 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:06.181567 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.055693 26407 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.182952 21857 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:06.189261 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 860) took 1661 ms. Trace:
I20260709 13:07:06.157325 26454 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.186316 22007 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:06.193133 21860 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:07:06.189718 22588 rpcz_store.cc:276] 0709 13:07:04.523471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.547292 (+ 23821us) service_pool.cc:224] Handling call
0709 13:07:05.952544 (+1405252us) spinlock_profiling.cc:243] Waited 8.74 ms on lock 0x7b54004955a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d2e18 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:06.184357 (+231813us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20101376}
I20260709 13:07:06.164028 26371 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.223678 26371 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.226362 26371 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:06.205520 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.206696 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.230114 22004 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:06.233369 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:06.234261 22007 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.237126 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:06.238346 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55625.6us
I20260709 13:07:06.239130 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:06.239863 26371 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:06.240180 26407 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.240424 26371 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:06.241531 26371 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.242419 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.242730 26322 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:06.243577 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.244673 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.208560 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.246081 26138 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:06.246905 21856 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:07:06.247797 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:06.250270 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:06.252947 26439 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:07:06.189841 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.257176 21857 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:06.259557 25915 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:06.260553 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:06.220120 26510 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:07:06.266357 21852 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:06.222160 21854 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.243309 26322 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:06.164901 26523 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.250891 26512 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:06.251413 26205 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:06.252209 26174 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:06.253181 26454 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:06.253763 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71417.7us
I20260709 13:07:06.254397 26451 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:06.219486 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1063) took 1343 ms. Trace:
I20260709 13:07:06.210098 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:06.209129 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44853us
I20260709 13:07:06.199697 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:06.207683 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:06.265283 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.270977 21935 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:06.275066 26322 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.276526 26523 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.280457 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.293103 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.295806 22003 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:07:06.296583 22003 pending_rounds.cc:85] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 11
I20260709 13:07:06.292464 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:06.301468 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.278470 26512 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.296967 22003 pending_rounds.cc:107] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:07:06.316506 22003 raft_consensus.cc:2889] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:06.316839 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.323348 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.318830 22000 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:06.292572 22589 rpcz_store.cc:276] 0709 13:07:04.875567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.880188 (+  4621us) service_pool.cc:224] Handling call
0709 13:07:04.881859 (+  1671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.882712 (+   853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:04.882743 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.884088 (+  1345us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.834322 (+950234us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b1800399220. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:05.948479 (+114157us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:05.948548 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.948757 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.219049 (+270292us) raft_consensus.cc:1685] finished
0709 13:07:06.219068 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.219301 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":679575,"raft.queue_time_us":107,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":684456,"spinlock_wait_cycles":5604736,"thread_start_us":51071,"threads_started":1,"wal-append.queue_time_us":1332930}
I20260709 13:07:06.331081 21851 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:06.334044 21855 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.316941 26512 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:06.335179 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.336817 22005 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:06.335291 21859 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:06.339023 21860 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:06.339360 21824 leader_election.cc:400] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16.
I20260709 13:07:06.339761 21824 leader_election.cc:403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:06.338941 21856 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:06.345036 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.345573 21857 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:06.345880 21933 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.327245 26474 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:06.331920 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.611s (SENT) [suppressed 18 similar messages]
I20260709 13:07:06.361368 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.365587 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.366438 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.367053 21935 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:06.341264 26273 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.342522 26138 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.371006 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.386704 21934 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:06.387003 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.387768 21857 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:06.388175 21854 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:06.380033 22593 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:06.348119 26474 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:06.390822 21852 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:06.398442 21932 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:06.399766 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.400413 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.400990 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.401906 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.402206 21932 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:06.376355 26174 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16.
I20260709 13:07:06.408027 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.408420 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:06.409407 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 98402.6us
I20260709 13:07:06.413523 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.415421 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.417946 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.420672 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.421890 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:06.391853 26418 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:07:06.426285 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.386703 26273 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:06.434322 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.435289 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.436185 26273 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:06.438886 21855 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:07:06.424943 26427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.170s	user 0.004s	sys 0.014s
I20260709 13:07:06.451706 21935 raft_consensus.cc:2368] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:07:06.443902 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:06.452703 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.454660 21857 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 13:07:06.459182 22592 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:07:06.442929 26427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.311s	user 0.004s	sys 0.015s
I20260709 13:07:06.411309 26138 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.421983 26504 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.458586 21860 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:06.470705 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.479730 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.474050 22007 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:06.482283 21928 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:06.453804 26439 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:07:06.483414 25915 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:06.505910 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.507465 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.509945 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:06.509977 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.511992 26418 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:06.512840 26371 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:06.513384 26371 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.513863 26371 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:06.517752 26565 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:06.512121 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.519210 21927 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:06.520470 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.520944 26504 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:06.521283 21927 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:06.522487 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.523353 21927 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:06.537622 26138 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:06.537865 22592 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:06.538160 21927 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:07:06.538565 21927 pending_rounds.cc:85] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 6
I20260709 13:07:06.538837 21927 pending_rounds.cc:107] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:07:06.539114 21927 raft_consensus.cc:2889] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:06.539172 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:06.539896 22586 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:06.533147 26523 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:06.533653 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.540732 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.533926 26235 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:06.534643 26383 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:06.533695 26477 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:07:06.535840 26427 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:06.462105 26474 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:07:06.529284 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 85469.5us
I20260709 13:07:06.543993 21929 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:06.544615 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:06.545748 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:06.580963 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:06.582055 26371 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.584069 22588 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.584280 26371 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:06.584976 22586 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:06.586766 26510 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:06.587201 26510 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.587541 26510 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.589191 21935 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:06.581034 26504 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:06.589723 26504 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:06.590591 26504 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.591254 26205 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:06.591662 26205 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:06.591928 26504 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:06.594059 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:06.595504 21928 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:07:06.595711 21929 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:06.557631 26523 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.599073 26523 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:06.605578 21929 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:06.540841 26512 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.605913 22588 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:06.610462 21933 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:06.560987 26415 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:07:06.613292 21928 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:06.606582 26439 raft_consensus.cc:2764] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:07:06.633149 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:06.615248 26415 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:06.616555 26512 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:06.620393 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.622066 21927 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:06.628108 26474 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.637713 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2465) took 1615 ms. Trace:
I20260709 13:07:06.638263 22003 rpcz_store.cc:276] 0709 13:07:05.022148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.816536 (+794388us) service_pool.cc:224] Handling call
0709 13:07:05.849224 (+ 32688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.322106 (+472882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.581558 (+259452us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b54002d1f20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d2a02 00007f3b707cedd6 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:06.635364 (+ 53806us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b5400437bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:06.637642 (+  2278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000745984}
I20260709 13:07:06.647075 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:06.648590 26451 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:06.649317 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:06.649878 26273 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:06.650315 26273 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:06.650502 26277 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.651062 26277 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.651569 21927 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:07:06.651283 26273 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:06.656559 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49005us
I20260709 13:07:06.644660 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:06.592630 26205 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:06.658866 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 326950us
I20260709 13:07:06.660964 21932 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:06.652678 26273 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:06.684953 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.685343 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 353643us
I20260709 13:07:06.687075 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:06.649353 26451 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.695293 26418 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:06.695773 21929 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:06.637030 26415 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:06.696457 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:06.702589 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.702920 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:06.703631 21857 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:06.703634 21855 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.708057 21854 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:06.699518 21972 leader_election.cc:400] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:07:06.710136 21972 leader_election.cc:403] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:07:06.698889 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:06.712265 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:06.695693 26277 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.735013 21855 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:06.696671 26512 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.695868 26418 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:06.736222 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45887.1us
I20260709 13:07:06.683219 26407 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:07:06.741317 22003 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:06.743861 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99379.2us
I20260709 13:07:06.713706 26235 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:06.757361 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:06.759110 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.746737 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 201087us
I20260709 13:07:06.743541 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:06.752619 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:06.712849 26439 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:06.772046 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:06.746182 26418 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.739118 26110 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.769794 26277 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:06.745486 26512 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.790959 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:06.788839 26277 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.789098 26371 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:06.793253 26364 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:06.788056 26110 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.804451 26277 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:06.787758 26439 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.826464 26454 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.827786 26454 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:06.829051 26447 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11.
I20260709 13:07:06.811408 26364 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.831090 26364 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.833516 26404 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:06.836848 26110 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:06.838371 26512 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.814638 26383 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:06.826557 25915 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:06.843848 25915 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.844241 25915 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.802166 26463 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.123s
I20260709 13:07:06.798550 26138 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:07:06.885587 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 235412us
I20260709 13:07:06.949501 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:06.956243 26477 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:06.886533 26451 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:06.952518 26585 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:06.961825 26477 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.986354 26530 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.986720 26205 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.987097 26530 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:06.971158 26569 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:06.992339 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:06.997687 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 245638us
I20260709 13:07:06.998615 26415 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:06.999754 26415 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:07.001065 26581 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.098s	user 0.004s	sys 0.000s
W20260709 13:07:07.002455 26580 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:07:07.010654 26530 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:07.013099 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.014086 22593 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:07:06.988654 26477 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:07.034307 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 290866us
W20260709 13:07:07.036825 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 230741us
W20260709 13:07:07.038437 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.040414 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 250712us
W20260709 13:07:07.044438 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 239768us
I20260709 13:07:07.045917 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:07.046133 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.993189 22588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
W20260709 13:07:07.071475 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 318988us
I20260709 13:07:07.042284 26569 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:07.073127 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.040331 26322 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:07:07.071473 22588 raft_consensus.cc:606] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:07.079362 22588 raft_consensus.cc:614] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:07.079751 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:07.092607 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 283719us
I20260709 13:07:07.107951 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:07.068749 26474 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.112962 21928 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:07.064679 26415 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:07:07.067054 26235 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:07.130980 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:07.127791 21934 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:07.130505 26585 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.134991 26585 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.130537 26463 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:07.137820 26463 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.138605 26463 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:07.125854 26504 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:07:07.141387 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:07.141961 26504 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:07.142385 26583 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.128324 26474 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.142477 26504 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:07.145026 26504 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.145973 26474 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:07.146433 26504 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.143199 26583 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.142532 22593 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:07.157181 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 111351us
I20260709 13:07:07.158595 22589 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:07.159610 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 81553.1us
I20260709 13:07:07.160346 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:07.170028 22590 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:07.179169 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:07.133767 26371 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:07.181803 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:07.186524 26591 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:07:07.187173 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.187044 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.179088 21928 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:07:07.189132 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 131730us
I20260709 13:07:07.189553 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:07.194221 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.194831 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.186627 26590 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:07.186524 26377 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:07.203926 22588 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:07.183908 26447 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:07.208303 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.208463 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:07.180677 26597 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:07:07.223762 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.216847 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.243703 21854 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:07.230641 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.246053 21857 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:07.208290 26447 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.211823 26585 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:07.216135 26322 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:07.257584 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 136842us
I20260709 13:07:07.262108 26439 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:07.263132 26322 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.263909 26322 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.256836 26604 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.267136 26604 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:07.247398 26510 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.001s
W20260709 13:07:07.273751 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70146.3us
I20260709 13:07:07.214536 26530 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:07.277026 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.217123 26606 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:07.281145 22003 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:07.281396 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.282368 22002 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:07.282842 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.283890 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.284555 22007 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:07.254782 26447 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.286329 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:07.289635 22003 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:07:07.290042 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.266348 26322 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:07.290966 22005 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
W20260709 13:07:07.287421 26273 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.000s
I20260709 13:07:07.305421 22007 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:07.193497 26569 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.248056 26364 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.313125 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:07.313565 26371 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.287245 26523 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:07.315425 26418 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 13:07:07.277312 26510 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.320436 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:07.312851 26595 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:07:07.322265 21930 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:07.270980 26235 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.342013 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:07.342648 21929 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:07.358412 22007 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:07.340935 26606 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:07.340744 26523 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:07.345093 26530 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:07.346397 26110 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.345844 26597 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:07.372012 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:07.375603 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:07.377728 26583 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:07.356891 26504 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:07.381922 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.385982 26614 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:07:07.388581 26597 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.389978 22002 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:07.392830 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:07.397951 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 112382us
I20260709 13:07:07.341725 26418 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:07.380057 26586 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:07:07.411038 22590 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:07:07.412554 26383 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.412640 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.413411 22009 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:07.413676 22006 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:07.413558 26383 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.416023 22593 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:07:07.416607 22593 pending_rounds.cc:85] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 11
I20260709 13:07:07.417064 22593 pending_rounds.cc:107] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:07:07.417621 22593 raft_consensus.cc:2889] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:07.416834 26383 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:07.407629 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:07.394699 25915 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.003s
W20260709 13:07:07.417958 26612 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:07:07.427405 26591 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.428227 26591 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.418123 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:07.429858 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.431237 26510 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:07.431700 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.431939 26602 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.432183 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.433221 21851 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:07.432715 26602 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.407719 26583 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.431367 26377 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.435318 26583 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.435782 26377 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.427253 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.437783 26606 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.403064 26585 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:07.439272 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:07.439473 26585 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:07.441427 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48716.3us
I20260709 13:07:07.442171 26415 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:07.442391 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.442641 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:07.434732 21851 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:07:07.447340 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65500.1us
I20260709 13:07:07.446154 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.451933 22006 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:07.451996 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:07.452185 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:07.444494 22591 raft_consensus.cc:2368] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:07.461465 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:07.445636 26364 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:07.470813 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.394984 26110 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.468820 26601 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:07.498072 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.498072 22009 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:07:07.511366 22009 pending_rounds.cc:85] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:07:07.511729 22009 pending_rounds.cc:107] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:07:07.512018 22009 raft_consensus.cc:2889] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:07.501780 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:07.513476 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.502116 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:07.504457 21853 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:07:07.505227 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45500.4us
I20260709 13:07:07.515273 22004 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:07.466673 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:07.507146 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.461506 26587 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:07.506675 22001 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
W20260709 13:07:07.516985 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50471.7us
I20260709 13:07:07.451781 25915 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:07.535166 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72863.1us
I20260709 13:07:07.504493 26510 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:07:07.539716 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 97433.9us
I20260709 13:07:07.537235 22002 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:07.544294 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.545452 21860 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:07.509586 26614 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:07.549368 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.556399 21852 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:07.530857 26590 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:07.534631 26364 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.513384 26439 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:07:07.568543 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1077) took 1006 ms. Trace:
I20260709 13:07:07.521157 26415 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:07:07.548198 26273 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.556109 26510 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:07.565750 26590 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.551614 26619 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.556900 26620 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:07.591692 26599 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 13:07:07.553786 26604 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:07.584188 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 129450us
I20260709 13:07:07.576908 22592 rpcz_store.cc:276] 0709 13:07:06.562130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.562400 (+   270us) service_pool.cc:224] Handling call
0709 13:07:06.586122 (+ 23722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.586596 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.586633 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:06.587091 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:07:06.918597 (+331506us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:06.918646 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.918711 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:07.568218 (+649507us) raft_consensus.cc:1685] finished
0709 13:07:07.568244 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.568461 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10404,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":2831,"thread_start_us":303348,"threads_started":1,"wal-append.queue_time_us":488476}
I20260709 13:07:07.586319 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:07.534584 26271 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:07:07.503525 26523 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:07.606561 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48812.8us
W20260709 13:07:07.604722 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 244596us
I20260709 13:07:07.553851 26474 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:07:07.615996 26590 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:07.637776 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:07.647701 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 87695.7us
I20260709 13:07:07.647786 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:07.614382 26585 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:07.557919 26614 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:07.655680 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69453.8us
I20260709 13:07:07.585618 26205 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:07:07.606037 26273 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.647367 26439 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:07.638520 26174 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.664402 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.672482 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.618036 26613 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:07.679163 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.680574 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:07.660709 26606 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.650278 26599 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:07:07.694705 22004 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:07.671231 26418 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:07.707814 26110 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:07.709240 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:07.638938 26619 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.707918 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:07.673822 26439 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:07.711577 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.682927 26377 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:07.685083 26415 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:07.684679 25915 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:07.713757 21825 leader_election.cc:400] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:07.721050 21825 leader_election.cc:403] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:07:07.713937 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.712919 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55659us
I20260709 13:07:07.716969 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.685956 26523 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:07.732750 21857 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:07.734603 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.738896 21858 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:07.719278 26383 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:07:07.736603 26622 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:07:07.704212 26174 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.763161 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:07.755898 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44413.6us
I20260709 13:07:07.764765 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.738140 26606 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:07.689486 26510 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.715879 26601 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:07:07.763984 21860 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:07.713964 26631 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:07.737599 26603 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:07.767969 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:07.768664 26598 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.735637 26205 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:07.775164 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.757288 26130 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:07.777710 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.765427 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:07.800379 21857 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:07:07.763724 26569 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:07.802575 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.792358 26631 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.814432 26631 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:07.761763 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:07.822674 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.823386 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.797442 26407 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.797574 26205 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:07.799799 26130 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:07.807523 25915 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.809794 26616 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:07.808966 26569 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:07.812384 26383 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:07.797894 26601 consensus_queue.cc:1243] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:07:07.861137 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 58726.8us
W20260709 13:07:07.862085 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 100456us
I20260709 13:07:07.816826 26427 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:07.773787 26439 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.865017 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:07.868990 21758 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:07.870937 26599 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:07.871363 26623 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.205s	user 0.001s	sys 0.001s
I20260709 13:07:07.843076 26637 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:07.854090 26606 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:07.839579 26585 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:07:07.858767 26569 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:07.887863 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:07.891042 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:07.905416 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.905932 22001 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:07:07.906411 22001 pending_rounds.cc:85] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:07:07.909356 22001 pending_rounds.cc:107] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:07:07.875641 26415 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.810146 26598 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.909634 22001 raft_consensus.cc:2889] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:07.878079 26273 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:07.921353 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:07.923779 22586 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:07.931450 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:07.932127 22586 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.854341 25915 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.936609 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.919535 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:07.880128 26407 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:07.941129 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.949693 22007 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:07.917080 26418 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.918735 26614 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:07.923326 26616 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:07.958895 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:07.926669 26439 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:07.921520 26637 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:07.884313 26427 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:07.947968 26624 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 13:07:07.933806 26138 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:07:07.907428 26636 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.212s
I20260709 13:07:08.019366 26646 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.020197 26646 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.967270 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:08.025774 21928 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:08.026467 26644 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:07:08.027477 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:08.029239 21933 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:08.030768 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:07.967839 26383 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.032640 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:07.947502 26601 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.034106 26601 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:07.954048 26635 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.951597 26273 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:07.927592 26415 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:07.942279 26643 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:07.936156 26634 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:07.954213 26418 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.058806 22009 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:07.880565 26599 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.147266 22584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:08.066355 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:08.068437 26633 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:07:08.148382 22000 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.069752 26504 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:08.070959 26602 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:07.963279 26619 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:08.088161 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.923167 26130 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.126593 22007 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
W20260709 13:07:08.136883 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.929029 26205 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.121544 26643 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.065521 26407 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.149765 22584 raft_consensus.cc:606] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:08.150911 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 231443us
I20260709 13:07:08.056615 26620 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:08.126689 26640 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.312s	user 0.000s	sys 0.002s
I20260709 13:07:08.161696 22002 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:08.164749 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:08.158751 26504 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:08.173942 22584 raft_consensus.cc:614] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:08.185334 21934 raft_consensus.cc:2376] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13. Re-sending same reply.
I20260709 13:07:08.174417 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.161428 22007 pending_rounds.cc:85] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:07:08.191517 22007 pending_rounds.cc:107] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:07:08.191942 22007 raft_consensus.cc:2889] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:08.195675 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.202044 26273 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:08.226058 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 303311us
I20260709 13:07:08.218861 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.146166 26635 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.242439 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.185441 26427 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.244365 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:08.180526 26174 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.245330 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:08.121340 26637 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:08.252863 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 387937us
I20260709 13:07:08.254107 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.190690 26504 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:08.189685 26418 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:08.195279 26205 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.231186 26322 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.182968 26407 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:08.256546 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 501529us
I20260709 13:07:08.261121 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.202742 26619 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:08.255215 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:08.266727 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.246986 26614 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:08.256148 26277 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.415s	user 0.015s	sys 0.013s
I20260709 13:07:08.298660 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.287879 22002 pending_rounds.cc:85] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 5
I20260709 13:07:08.300288 22002 pending_rounds.cc:107] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:07:08.300581 22002 raft_consensus.cc:2889] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:08.188056 26620 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:08.309978 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 57290.4us
I20260709 13:07:08.283758 25915 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.279019 26619 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:08.280040 26418 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:08.322533 26597 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:08.323658 26427 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.285329 26646 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.296643 26322 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:08.326189 26447 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.003s	user 0.013s	sys 0.012s
W20260709 13:07:08.329774 26463 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.190s	user 0.020s	sys 0.007s
I20260709 13:07:08.331534 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:08.333607 26647 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:07:08.336412 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:08.377699 22009 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:08.284574 26651 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:08.341637 26619 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:08.342365 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.344750 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.350397 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:08.309942 26639 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
W20260709 13:07:08.324762 26477 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.281s	user 0.021s	sys 0.003s
I20260709 13:07:08.286005 26614 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:08.354900 21860 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.546s	user 0.018s	sys 0.004s
I20260709 13:07:08.432941 21860 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:08.310338 26510 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:08.352486 26601 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.371886 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 118617us
I20260709 13:07:08.374526 22000 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:08.317911 26636 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:08.430680 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.296805 26277 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.372695 22586 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:08.349179 26418 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:08.365630 26451 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:08.375057 25915 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.481220 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:08.481932 22007 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:08.435460 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.374996 26447 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.484529 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:08.434108 26510 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.436807 26654 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.437160 26407 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.437597 26583 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.376849 26427 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.387598 26463 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.365522 26663 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:08.518504 26663 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.443121 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:08.503445 26454 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.472077 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:08.529188 21930 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.474018 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:08.530021 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 86988.6us
I20260709 13:07:08.474984 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.478180 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:08.478437 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:08.433902 26614 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:08.490576 26661 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.481683 26451 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.442947 26477 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.491823 26322 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:08.537350 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59307.3us
I20260709 13:07:08.491979 21929 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:07:08.537966 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:08.493144 25915 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.494278 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:08.481499 26418 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:08.497157 26636 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.499816 26277 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.502420 22007 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:08.431427 26646 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.502089 26447 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.567565 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:07:08.568572 21932 raft_consensus.cc:606] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:08.569592 22590 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:08.503739 21931 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:08.466670 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:08.443497 21934 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:08.502323 26510 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:08.500274 26643 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.536212 26523 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.800s	user 0.009s	sys 0.005s
I20260709 13:07:08.578660 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:08.570363 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82099.8us
I20260709 13:07:08.574117 22009 raft_consensus.cc:2368] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15.
I20260709 13:07:08.536764 26583 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:08.533627 26451 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.580892 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.612198 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:08.566781 26603 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:08.630897 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.628557 22557 leader_election.cc:400] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15.
I20260709 13:07:08.631729 22557 leader_election.cc:403] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:08.631136 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:08.632405 21859 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.628795 22007 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.559284 26613 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:08.636655 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54997.8us
I20260709 13:07:08.543139 26601 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:07:08.641602 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.575812 26205 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.645493 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.651530 21934 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 8.
I20260709 13:07:08.652653 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:08.653529 21934 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:08.583528 26643 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:08.630904 26647 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:07:08.626921 26322 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:08.566397 26454 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.537390 26661 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.648401 26523 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.649380 26613 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.520553 26654 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:08.647258 26666 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:08.614002 26585 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:07:08.613564 26504 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.727638 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 883) took 1023 ms. Trace:
W20260709 13:07:08.673413 26439 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.697s	user 0.017s	sys 0.006s
I20260709 13:07:08.627924 26646 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.735262 21933 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:08.741416 22586 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:08.698027 26637 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:08.734989 26377 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.018s	user 0.011s	sys 0.019s
I20260709 13:07:08.768544 22007 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
W20260709 13:07:08.774061 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 139135us
I20260709 13:07:08.784426 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:08.743185 22592 rpcz_store.cc:276] 0709 13:07:07.704701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.704836 (+   135us) service_pool.cc:224] Handling call
0709 13:07:07.707392 (+  2556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.710265 (+  2873us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:07.716673 (+  6408us) log.cc:844] Serialized 14 byte log entry
0709 13:07:07.812053 (+ 95380us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:07.832807 (+ 20754us) log.cc:844] Serialized 26 byte log entry
0709 13:07:07.832958 (+   151us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:07.833023 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.833099 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.327542 (+494443us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4800241458. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:07:08.699311 (+371769us) raft_consensus.cc:1685] finished
0709 13:07:08.726024 (+ 26713us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.727485 (+  1461us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2389,"prepare.queue_time_us":2700,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":5249,"raft.run_cpu_time_us":10878,"raft.run_wall_time_us":629038,"spinlock_wait_cycles":4311296,"thread_start_us":77745,"threads_started":1,"wal-append.queue_time_us":130733}
I20260709 13:07:08.676144 26643 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.728930 26322 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:08.677644 26205 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.739449 26620 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.743108 26657 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:08.787981 26616 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.826s	user 0.012s	sys 0.008s
I20260709 13:07:08.754830 26599 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.806975 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 152792us
I20260709 13:07:08.791374 22006 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:08.813941 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:08.788237 26377 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:08.744333 26656 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:07:08.787002 26591 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.835664 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 465226us
I20260709 13:07:08.797230 26676 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.804692 26594 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.819334 26616 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:08.742688 26666 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.826572 26603 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:08.884536 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.897320 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:08.826409 26407 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:07:08.922775 26683 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:08.900822 26652 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:08.934018 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:08.932156 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:08.913301 26655 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:07:08.846721 26591 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:07:08.843984 26174 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:07:08.934927 22587 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:08.912734 25915 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:08.948302 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55681.9us
I20260709 13:07:08.878095 26594 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:08.952003 26671 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:08.849203 26676 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.955060 26676 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:08.955144 26504 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:08.930933 26652 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.955787 26652 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.956382 26205 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.911082 26616 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:07:08.958017 26616 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:08.945125 26322 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:08.958384 26616 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:08.959242 26616 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.961040 26629 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:07:08.961818 26643 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.965390 26454 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.967585 26694 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.968192 26694 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.969228 26523 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15.
I20260709 13:07:08.970755 26694 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.971625 26694 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:08.972190 26694 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.939690 26601 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:08.976308 26661 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.941041 26463 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:08.977622 26654 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.978688 26670 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:07:08.980218 26658 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
W20260709 13:07:08.943519 26590 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.281s	user 0.012s	sys 0.003s
I20260709 13:07:08.980886 26477 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.982013 26674 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:07:08.932354 26663 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:08.986029 26569 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.068s	user 0.011s	sys 0.010s
I20260709 13:07:08.986951 26663 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.987516 26663 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.981473 26590 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.990114 26663 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:08.988588 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.961951 26629 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:08.992003 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:08.950297 26631 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.135s	user 0.024s	sys 0.000s
I20260709 13:07:08.993381 26631 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.995216 21930 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:08.995517 26631 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.996479 26646 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:08.997627 26669 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:07:08.996486 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:08.838367 26657 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.000063 26646 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:09.000459 26657 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.000509 26646 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.013615 26646 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:09.014953 26666 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:08.997429 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:09.018386 22001 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:08.978811 26660 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.536s	user 0.001s	sys 0.000s
W20260709 13:07:09.001690 26680 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:07:09.002168 26688 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:09.003527 21758 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 10, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 10 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:09.003908 26474 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:09.006117 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2092) took 1697 ms (client timeout 2253 ms). Trace:
I20260709 13:07:09.007479 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1019) took 1594 ms. Trace:
I20260709 13:07:08.996961 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:09.009337 21852 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:08.991616 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:08.956563 21935 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:09.012431 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:08.955885 26418 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:08.952768 26594 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.019340 22584 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:09.050391 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:09.049476 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:09.049947 21859 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:09.056142 22007 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:09.055485 22586 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:09.058976 22002 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:09.057385 22008 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:09.054428 21931 rpcz_store.cc:269] 0709 13:07:07.308278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.308408 (+   130us) service_pool.cc:224] Handling call
0709 13:07:09.005912 (+1697504us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003066d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:09.006055 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2574845312}
W20260709 13:07:09.054781 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 235977us
I20260709 13:07:09.047465 22590 rpcz_store.cc:276] 0709 13:07:07.420639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.426742 (+  6103us) service_pool.cc:224] Handling call
0709 13:07:07.427045 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.570082 (+1143037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.007216 (+437134us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400531954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:09.007416 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2626996992}
I20260709 13:07:09.075399 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:08.986527 26569 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:08.990346 26590 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:09.116098 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169448us
I20260709 13:07:09.113124 26635 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.078338 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:09.010658 26629 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:09.058553 26668 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 13:07:09.076514 26504 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.083424 26594 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
W20260709 13:07:09.089272 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 91907.2us
I20260709 13:07:09.090917 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:09.121621 26689 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:09.094424 26451 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.101131 26663 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:09.105475 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:09.082144 26620 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:09.126540 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:09.125402 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47134.8us
I20260709 13:07:09.094259 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:09.131521 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 304159us
W20260709 13:07:09.133694 22004 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.009s	sys 0.011s
W20260709 13:07:09.141114 22004 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.010s	sys 0.011s
I20260709 13:07:09.141781 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:09.140991 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:09.134003 26614 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.574s	user 0.004s	sys 0.007s
W20260709 13:07:09.146245 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52066.7us
I20260709 13:07:09.153090 22584 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:09.148290 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:09.156546 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.157538 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:09.157894 26530 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.158456 26530 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.159902 26463 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:09.160516 26463 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.160943 21859 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:09.168251 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.013552 26603 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.144124 26601 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.162392 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:09.177937 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.180063 21855 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:09.160957 26463 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:09.181464 21853 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:09.182713 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.184022 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.162407 26685 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:09.185410 26685 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:09.185081 26629 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.186570 26629 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:09.186700 26685 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.135247 26504 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.164347 26673 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:09.162779 26407 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:09.189595 26673 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.190068 26673 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.190013 26407 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.140656 26447 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:07:09.134704 26613 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.192454 26407 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.164908 26654 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:09.193621 26613 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.163285 26614 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.194047 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.194185 26694 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.194295 26601 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.164731 26697 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:07:09.196148 26606 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.196731 26606 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.197937 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.194696 21854 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:09.199230 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.162708 26322 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:09.209628 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50528.9us
I20260709 13:07:09.210861 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:09.211293 22008 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:09.165731 26591 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:09.212623 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50320.9us
I20260709 13:07:09.213798 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:09.216257 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:09.215128 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.217245 26694 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:09.217679 21858 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:09.217988 26694 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:09.218938 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.219678 21858 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.219190 26694 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.223073 21854 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:09.233681 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.223549 26699 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:09.224015 26672 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:07:09.224309 26530 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.168147 26597 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:07:09.225106 26652 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.247987 26652 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:09.227316 26619 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.761s	user 0.016s	sys 0.000s
I20260709 13:07:09.227169 26463 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.229794 26569 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.234617 21857 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:07:09.232072 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.527s (SENT)
I20260709 13:07:09.232043 21853 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:09.162010 26700 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:09.242811 21855 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:09.226197 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:09.213317 26510 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.003s
I20260709 13:07:09.264026 26510 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.264323 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:09.250330 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:09.253152 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.265276 22588 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:09.254350 21930 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:07:09.266234 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.233247 25915 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:09.266487 26510 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.210511 26322 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:09.266889 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:09.266952 25915 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.260594 26657 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:09.261297 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:09.243072 26699 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.270481 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.269682 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:09.248658 26594 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:09.242874 26606 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.268940 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.277132 21926 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:09.267609 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.271080 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:09.271729 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.248425 26614 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.279237 22003 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:09.276381 21927 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:07:09.280848 21854 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:09.246423 26620 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:09.263039 26700 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.260447 26656 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:09.322403 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 90352us
I20260709 13:07:09.327323 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.259142 26530 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:07:09.322706 26447 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:09.276507 26613 raft_consensus.cc:2764] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:09.269846 26322 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.270103 25915 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:09.271209 26110 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:09.331002 21934 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:09.275003 26591 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.262344 26619 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.270251 26657 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:09.279065 26614 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:07:09.274607 26699 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:09.343375 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67905.1us
I20260709 13:07:09.345198 26463 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.348534 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:09.332760 26666 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:09.330891 26700 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:09.341908 26599 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:07:09.360913 26672 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:09.368242 22005 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:09.364346 26601 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.319226 26606 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:07:09.351766 26657 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:09.341406 26656 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.381925 26620 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.378861 26673 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.384589 26656 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.380429 26418 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.388029 26619 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.389582 26601 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.392338 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.366489 26451 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.384758 26673 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.350946 26591 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.390086 26601 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.369510 26614 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:07:09.520028 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.417761 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 168048us
I20260709 13:07:09.394512 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:09.467031 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 118570us
I20260709 13:07:09.540658 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.465924 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.410701 22587 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:09.368100 26672 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:09.676051 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2447) took 1150 ms. Trace:
I20260709 13:07:09.678431 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:09.678289 21860 rpcz_store.cc:276] 0709 13:07:08.525941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.533872 (+  7931us) service_pool.cc:224] Handling call
0709 13:07:08.534257 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.582171 (+ 47914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.582223 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:08.582288 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:08.582342 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.675806 (+1093464us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b54000596d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:09.675857 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.675989 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109651712}
W20260709 13:07:09.425653 26701 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:07:09.699411 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.703433 21926 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:09.408689 26656 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:09.500483 26715 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:09.522763 26714 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:09.730368 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 296386us
I20260709 13:07:09.710378 21934 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:07:09.704090 26658 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:07:09.369315 26646 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.677424 26477 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:09.735358 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.731950 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:09.759644 21934 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.750324 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.523653 26322 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.736902 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.765334 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:09.767030 21899 leader_election.cc:336] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 1.660s (SENT)
I20260709 13:07:09.725312 26451 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:09.728480 26277 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:09.724090 26614 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:09.711942 26606 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:07:09.821841 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:09.825560 21858 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:07:09.826376 21858 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:09.789692 22585 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:09.730599 26715 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:09.843997 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.729171 26656 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.846581 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.871675 21860 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:09.872679 26716 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
W20260709 13:07:09.873703 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 106718us
I20260709 13:07:09.849300 26715 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:09.852017 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 173757us
I20260709 13:07:09.767243 26714 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.880020 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:09.880249 21860 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:09.877733 26601 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.881268 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.785722 26698 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:09.882566 21860 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
W20260709 13:07:09.882506 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 224055us
I20260709 13:07:09.789655 26661 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:09.882990 21926 raft_consensus.cc:2368] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:07:09.885702 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:09.885651 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:09.848743 26614 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:09.840894 26658 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:07:09.889930 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.888548 21855 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
W20260709 13:07:09.881820 25915 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.003s
I20260709 13:07:09.768469 26477 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.825968 26277 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.889144 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:09.889443 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:09.904044 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.905141 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.834656 26451 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.907505 21855 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:09.908718 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.909446 21855 raft_consensus.cc:2410] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:07:09.910256 22556 leader_election.cc:400] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:07:09.910490 22556 leader_election.cc:403] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:07:09.913897 26614 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.915318 26235 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:09.901561 26110 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:09.918394 26277 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.889521 26698 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.919425 26599 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.900470 26666 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:09.920125 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:09.949579 21926 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:09.891497 26661 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.904807 25915 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:09.922981 26451 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.939510 26418 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:09.925536 26322 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.925760 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:09.927497 26656 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.767740 26672 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.937381 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:09.888150 26694 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:07:09.975841 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:09.976931 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.977442 21855 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.986163 21927 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:09.995848 21926 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:09.952421 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:09.953184 21934 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:09.923537 26646 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.957242 26698 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:09.940047 26110 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:09.940384 26733 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:09.962100 26720 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:07:09.962953 26730 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:09.965530 26715 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:09.998342 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.009640 21928 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:09.999152 21860 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:10.001669 21854 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 9: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:10.002189 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:09.948019 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.003023 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:09.973097 26726 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:07:09.979012 26477 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:09.979331 26591 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.995672 26418 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:09.968968 26700 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.000s
I20260709 13:07:09.981803 26732 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:10.014007 26699 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.645s	user 0.007s	sys 0.003s
I20260709 13:07:09.973966 26672 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.018252 21855 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:10.039925 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:10.021095 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 110912us
I20260709 13:07:10.041640 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.042129 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:10.042685 21927 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:10.022822 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:09.973805 26415 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:09.976907 26659 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:09.980958 26661 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.011806 21926 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:10.033447 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.034228 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.012873 21858 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:07:10.002363 26713 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:10.007723 26110 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.054004 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.049551 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:10.055579 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.056136 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.073643 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:10.074837 26666 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:10.075091 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:10.075444 26666 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:10.076093 26585 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:10.077194 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.076367 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:10.057013 21928 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:10.079372 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.050390 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.085883 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.076691 26585 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:10.088205 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.090582 21927 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:10.086606 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.092100 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.093129 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.080703 26698 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:10.020220 26733 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.053813 26477 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:10.096905 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.098564 26585 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.099778 26715 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:10.100387 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.101533 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.101662 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2492) took 1626 ms. Trace:
I20260709 13:07:10.070892 26713 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.036393 26732 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.119854 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.109726 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.120622 21860 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.110728 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.091253 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:10.111660 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:10.123145 26714 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:10.123729 26656 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:10.119540 26646 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:07:10.124255 26656 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.124696 26646 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:10.125267 26646 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:10.125685 26418 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.126312 26646 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.127625 26646 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.035604 26700 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.115343 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.129733 21858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:07:10.117643 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.077841 26661 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:10.080562 26596 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:10.080709 26407 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:10.096412 26698 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:10.130354 21858 raft_consensus.cc:606] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:10.091203 26736 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:10.131295 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:10.113827 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:10.048156 26699 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.135250 21858 raft_consensus.cc:614] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:10.111428 22000 rpcz_store.cc:276] 0709 13:07:08.475774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.475984 (+   210us) service_pool.cc:224] Handling call
0709 13:07:08.476421 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.650029 (+173608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.651729 (+  1700us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.652455 (+   726us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.835294 (+182839us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:08.876793 (+ 41499us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.877066 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.877148 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.956822 (+1079674us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b18004b8680. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:10.101352 (+144530us) raft_consensus.cc:1685] finished
0709 13:07:10.101458 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.101600 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25053,"prepare.queue_time_us":95041,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":69365,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":15200,"spinlock_wait_cycles":130923392,"thread_start_us":275360,"threads_started":2,"wal-append.queue_time_us":274685}
I20260709 13:07:10.124650 26656 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.140664 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:10.133625 21824 connection.cc:256] RPC call timeout handler was delayed by -1.2568768240000736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
I20260709 13:07:10.148852 26635 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.149428 26732 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.143893 26629 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:07:10.149688 26635 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.150920 26629 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:10.151269 26629 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:10.152369 26713 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.152679 26635 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.144114 26596 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.154135 26733 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:10.157651 25915 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.137425 26698 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:10.158692 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.137583 26661 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.150148 26407 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.165975 26700 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.071048 26659 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.145586 26477 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:10.170874 21929 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:10.177909 21928 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:10.180148 21853 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:10.180109 21859 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:07:10.182268 21859 pending_rounds.cc:85] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 9
I20260709 13:07:10.182636 21859 pending_rounds.cc:107] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:07:10.182926 22556 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:10.183108 21859 raft_consensus.cc:2889] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:10.184955 26659 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.185153 26322 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.185473 26715 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.219534 26715 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.222118 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 943) took 1008 ms. Trace:
I20260709 13:07:10.185748 26620 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.222537 26715 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.222851 26620 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:10.223840 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 808) took 1646 ms (client timeout 2029 ms). Trace:
I20260709 13:07:10.076395 26666 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:10.224316 21933 rpcz_store.cc:269] 0709 13:07:08.577759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.577905 (+   146us) service_pool.cc:224] Handling call
0709 13:07:10.223616 (+1645711us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400097ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:10.223759 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237972864}
I20260709 13:07:10.225178 26620 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.226230 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.188129 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.188973 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.227305 22008 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:10.227121 21933 raft_consensus.cc:2376] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13. Re-sending same reply.
I20260709 13:07:10.189181 22002 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:10.189248 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.228183 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.186266 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.189597 26598 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.191555 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.192672 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.196087 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.138630 26736 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.197917 26477 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:10.257543 26736 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.201112 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:10.258756 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57760.3us
W20260709 13:07:10.260460 26742 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.185s	user 0.001s	sys 0.001s
I20260709 13:07:10.261178 26651 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:10.261740 26651 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:10.262147 26651 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:10.152140 26629 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.267093 22009 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:10.213163 21857 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:10.143587 26601 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:10.222591 22004 rpcz_store.cc:276] 0709 13:07:09.213255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.213372 (+   117us) service_pool.cc:224] Handling call
0709 13:07:10.221875 (+1008503us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54003890d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:10.222034 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078774144}
I20260709 13:07:10.197711 26504 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.195050 26130 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.228223 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.254140 21856 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:10.255324 26699 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.256385 26684 raft_consensus.cc:993] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:07:10.257649 26477 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.198626 26694 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:10.236819 26732 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:10.209354 26322 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.210134 26407 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:10.267400 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61414.9us
W20260709 13:07:10.203506 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 112579us
I20260709 13:07:10.291885 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:10.299428 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.228541 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.268612 22000 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:10.269174 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.272957 22006 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.236402 21860 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:10.310806 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.273527 21859 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:10.274401 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.216620 26656 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:10.284327 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.258877 21853 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:10.302575 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.303969 26322 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:10.314668 22002 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:10.280786 26678 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.361s	user 0.000s	sys 0.004s
I20260709 13:07:10.271410 26601 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.285315 26629 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.311734 21929 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.319257 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.290205 26652 raft_consensus.cc:1081] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:07:10.328660 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.322676 26601 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.322049 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.287731 26477 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:10.316759 26415 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:10.327179 21860 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:10.367125 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.301563 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:10.290553 26732 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.354789 22006 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:10.344220 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.376929 22004 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:10.322104 26684 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:10.382562 21853 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:10.384467 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.362010 22002 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:10.274200 25915 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 13:07:10.374619 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 73173.1us
I20260709 13:07:10.328569 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.324191 26599 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:07:10.369254 26415 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.378428 26732 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:10.436178 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.394816 26322 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.353802 26661 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.351070 26656 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.393853 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.395174 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.386140 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.388818 21860 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:07:10.469594 26597 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:10.492771 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.397635 21853 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:10.398197 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.380043 26602 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:10.442387 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1024) took 2000 ms. Trace:
I20260709 13:07:10.443696 26636 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.446419 26613 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:10.458374 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2451) took 1915 ms (client timeout 1729 ms). Trace:
W20260709 13:07:10.499073 21852 rpcz_store.cc:269] 0709 13:07:08.542959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.559273 (+ 16314us) service_pool.cc:224] Handling call
0709 13:07:10.458080 (+1898807us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b540008a7d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:10.458246 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988500224}
I20260709 13:07:10.460192 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1090) took 1994 ms. Trace:
I20260709 13:07:10.399468 26415 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.485630 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2593) took 1231 ms. Trace:
I20260709 13:07:10.506317 21853 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:07:10.500469 22593 rpcz_store.cc:276] 0709 13:07:08.466185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.659067 (+192882us) service_pool.cc:224] Handling call
0709 13:07:08.724892 (+ 65825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.905207 (+180315us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:08.905468 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.071534 (+166066us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.071620 (+    86us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:09.071691 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.459931 (+1388240us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400595454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:10.459982 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.460131 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30844,"prepare.queue_time_us":470,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":23989,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":4281,"spinlock_wait_cycles":414067072,"thread_start_us":141496,"threads_started":1,"wal-append.queue_time_us":238827,"wal-append.run_cpu_time_us":1288,"wal-append.run_wall_time_us":1035526}
I20260709 13:07:10.491773 26597 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.505602 21932 rpcz_store.cc:276] 0709 13:07:09.254653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.254784 (+   131us) service_pool.cc:224] Handling call
0709 13:07:09.271630 (+ 16846us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:09.280696 (+  9066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.280724 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:10.099611 (+818887us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:10.100529 (+   918us) log.cc:844] Serialized 52 byte log entry
0709 13:07:10.378581 (+278052us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:10.378618 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.378694 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:10.485278 (+106584us) raft_consensus.cc:1685] finished
0709 13:07:10.485299 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.485547 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8676,"raft.queue_time_us":10615,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5151,"spinlock_wait_cycles":91716992,"thread_start_us":277145,"threads_started":1,"wal-append.queue_time_us":382359}
I20260709 13:07:10.440687 26110 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.491520 21860 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:10.495004 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.392988 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:10.492177 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.497566 22588 rpcz_store.cc:276] 0709 13:07:08.441673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.444195 (+  2522us) service_pool.cc:224] Handling call
0709 13:07:10.442171 (+1997976us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54003791d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:10.442318 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793200384}
W20260709 13:07:10.514791 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 121864us
I20260709 13:07:10.516422 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.517156 22004 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:10.517647 26110 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.517843 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.497208 26602 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.518190 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.518551 26602 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.519524 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.520401 22004 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:10.517406 26661 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:07:10.523125 26597 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:10.523788 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.524106 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.525422 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:10.525754 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.526326 21932 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:10.526466 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.526829 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.527767 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.530284 26661 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:10.498975 26613 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:10.533996 26613 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.534284 26651 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.535161 26415 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.535832 26613 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.503276 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1083) took 2631 ms. Trace:
I20260709 13:07:10.389307 26684 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.518545 22000 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.504119 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.539346 22006 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:10.507977 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:10.500355 26657 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.102s	user 0.012s	sys 0.002s
I20260709 13:07:10.388810 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.548130 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:10.537580 22591 rpcz_store.cc:276] 0709 13:07:07.871696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.878736 (+  7040us) service_pool.cc:224] Handling call
0709 13:07:07.924976 (+ 46240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.278360 (+353384us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:08.565654 (+287294us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.873405 (+307751us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.892923 (+ 19518us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.893146 (+   223us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:08.893212 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.893312 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.002427 (+109115us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b1800345c40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:10.228108 (+1225681us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:10.502993 (+274885us) raft_consensus.cc:1685] finished
0709 13:07:10.503025 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.503201 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25879,"prepare.queue_time_us":293289,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":321023,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":17310,"spinlock_wait_cycles":14151936,"thread_start_us":892394,"threads_started":4,"wal-append.queue_time_us":260384}
I20260709 13:07:10.541893 21929 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:10.553709 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.555028 22004 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:10.487735 26322 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.543785 26636 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:10.549871 21853 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:10.558626 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.560930 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.690131 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.694036 22002 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:10.694787 21758 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:10.672482 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:10.704461 26651 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.547047 26657 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.711640 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:10.712493 21853 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:10.716944 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 391470us
W20260709 13:07:10.708293 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2487) took 2742 ms (client timeout 1660 ms). Trace:
I20260709 13:07:10.710227 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.718003 21927 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:10.718571 22006 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:07:10.689370 26698 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.002s
I20260709 13:07:10.732596 22000 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:10.733829 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:10.734449 22590 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.735245 22000 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:07:10.736581 22000 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:10.702441 26736 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.739010 26322 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.740453 26110 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:10.707943 26235 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:10.742549 26235 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:10.741098 26110 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.743132 26235 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.741554 26657 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.741662 21858 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:10.719573 21934 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:07:10.741817 26620 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.745519 26620 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.749195 26585 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:10.749958 26585 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.753459 26415 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:10.717767 22001 rpcz_store.cc:269] 0709 13:07:07.955697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.955961 (+   264us) service_pool.cc:224] Handling call
0709 13:07:10.697222 (+2741261us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54002152d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:10.697356 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285004672}
I20260709 13:07:10.721719 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:10.669050 26636 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:10.785995 22001 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:10.786901 26636 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.744669 22002 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:10.740993 26736 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.785418 26322 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:10.794974 26699 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:10.795673 26590 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.812247 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bcf30627084357a4745445324a2430"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:10.812573 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:10.819896 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.822131 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.059s	sys 0.115s
W20260709 13:07:10.822667 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.059s	sys 0.115s
I20260709 13:07:10.823856 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:10.813205 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:10.831535 22004 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:10.841986 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 120162us
I20260709 13:07:10.843016 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:10.811043 26510 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:10.818537 26657 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:10.849845 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:10.853525 22593 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:10.854370 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:10.860530 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:10.843384 26616 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:10.864715 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.873967 21934 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 9: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:10.870661 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:10.823905 26590 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.820761 26699 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:10.846670 26656 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:10.871940 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.886219 21929 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:10.792955 26698 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.877851 22004 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:10.880550 22002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:41342
W20260709 13:07:10.889180 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.889482 22002 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:10.889999 22002 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.895768 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:10.896657 21934 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.898204 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:10.899175 22593 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:10.895787 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:10.903947 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:10.842871 26322 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.904078 21853 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:10.904577 22002 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.917300 22591 raft_consensus.cc:606] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:10.918155 22591 raft_consensus.cc:614] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:10.887332 21758 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 6, term changed from 6 to 9, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to <none>, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130), VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 9 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:10.854574 26510 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.886047 26656 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:10.944015 21853 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:07:10.907685 26477 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.011s
W20260709 13:07:10.946766 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.929271 26585 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:10.949579 26271 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:10.957074 26477 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:10.959450 26477 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:10.960242 26741 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.790s	user 0.005s	sys 0.002s
W20260709 13:07:10.962157 26601 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.003s
I20260709 13:07:10.927083 26597 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.963140 21927 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:10.965512 21859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.051s	sys 0.066s
I20260709 13:07:10.906286 26620 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.964987 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:10.978549 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:10.979177 22584 raft_consensus.cc:2368] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
W20260709 13:07:10.979722 21859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.051s	sys 0.066s
I20260709 13:07:10.941250 26684 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.973186 22587 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:10.975770 21929 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:10.948114 26110 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:10.982291 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2139) took 1047 ms. Trace:
I20260709 13:07:10.947777 26656 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:10.983176 21854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.011s	sys 0.021s
W20260709 13:07:11.040181 21854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.011s	sys 0.021s
I20260709 13:07:10.972162 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:11.037571 21856 rpcz_store.cc:276] 0709 13:07:09.934939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.282203 (+347264us) service_pool.cc:224] Handling call
0709 13:07:10.309305 (+ 27102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.309591 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:10.547166 (+237575us) log.cc:844] Serialized 14 byte log entry
0709 13:07:10.981861 (+434695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:10.981933 (+    72us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:10.981989 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.982074 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.982213 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":572375,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":656933,"threads_started":2}
W20260709 13:07:11.037633 26672 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.983s	user 0.018s	sys 0.007s
W20260709 13:07:11.041556 26741 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.077s	user 0.003s	sys 0.000s
I20260709 13:07:11.061125 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2590) took 2742 ms. Trace:
I20260709 13:07:11.070856 22005 rpcz_store.cc:276] 0709 13:07:08.304335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.304661 (+   326us) service_pool.cc:224] Handling call
0709 13:07:08.313317 (+  8656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.375025 (+1061708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.046206 (+1671181us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540038aed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:11.046545 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2423024896}
W20260709 13:07:11.045679 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 136474us
W20260709 13:07:11.073254 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 299437us
W20260709 13:07:11.079507 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.980792 26601 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.087635 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:11.098160 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2594) took 1843 ms. Trace:
I20260709 13:07:11.098047 21900 leader_election.cc:400] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:11.099210 21900 leader_election.cc:403] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:11.098806 21930 rpcz_store.cc:276] 0709 13:07:09.254876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.266343 (+ 11467us) service_pool.cc:224] Handling call
0709 13:07:09.266676 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.973500 (+706824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.973535 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:09.974018 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:07:10.079447 (+105429us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:10.097629 (+ 18182us) log.cc:844] Serialized 14 byte log entry
0709 13:07:10.098055 (+   426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.525409 (+427354us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400229054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:10.525461 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:10.959673 (+434212us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:11.097849 (+138176us) raft_consensus.cc:1685] finished
0709 13:07:11.097872 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.098079 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21289,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":14099,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":10846,"spinlock_wait_cycles":1648887040,"thread_start_us":105050,"threads_started":1,"wal-append.queue_time_us":157894}
W20260709 13:07:11.122063 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 279208us
I20260709 13:07:11.129642 26427 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:11.132856 26732 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.699s	user 0.005s	sys 0.017s
I20260709 13:07:11.133345 26732 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.134574 22007 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17.
I20260709 13:07:11.117141 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:11.136119 26732 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:11.140398 26602 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.621s	user 0.017s	sys 0.004s
I20260709 13:07:11.136641 26451 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:11.140982 26602 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.143606 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2471) took 1162 ms. Trace:
I20260709 13:07:11.153453 21859 rpcz_store.cc:276] 0709 13:07:09.981286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.359715 (+378429us) service_pool.cc:224] Handling call
0709 13:07:10.360002 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.360377 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.360404 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.371855 (+ 11451us) log.cc:844] Serialized 26 byte log entry
0709 13:07:10.982568 (+610713us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:10.982605 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.982668 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.142224 (+159556us) raft_consensus.cc:1685] finished
0709 13:07:11.142252 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.143523 (+  1271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":885,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":11913,"thread_start_us":539866,"threads_started":1,"wal-append.queue_time_us":679190}
I20260709 13:07:11.155246 26322 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:11.143931 26451 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:11.145888 26235 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.157596 22588 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:11.157276 26451 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.158757 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2603) took 1444 ms. Trace:
I20260709 13:07:11.160246 26510 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.159219 21932 rpcz_store.cc:276] 0709 13:07:09.713932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.531007 (+817075us) service_pool.cc:224] Handling call
0709 13:07:10.531343 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:10.545824 (+ 14481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.545853 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:10.549056 (+  3203us) log.cc:844] Serialized 52 byte log entry
0709 13:07:10.985427 (+436371us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:10.985489 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.994600 (+  9111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.158505 (+163905us) raft_consensus.cc:1685] finished
0709 13:07:11.158528 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.158694 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11326,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":177,"thread_start_us":290747,"threads_started":1,"wal-append.queue_time_us":426734}
I20260709 13:07:11.161484 26698 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.164461 26235 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:11.165285 26646 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.037s	user 0.011s	sys 0.002s
I20260709 13:07:11.166507 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.121322 26697 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:11.169844 26719 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:11.170843 26719 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:11.171404 26719 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:11.116560 26609 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
W20260709 13:07:11.085945 26130 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.804s	user 0.008s	sys 0.004s
I20260709 13:07:11.182113 21855 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:11.177735 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 90193.9us
I20260709 13:07:11.180171 22590 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:07:11.180773 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.186710 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.197965 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:11.201282 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 352819us
W20260709 13:07:11.205166 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.204649 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:11.217339 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.220854 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.223397 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:11.225404 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:11.228897 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:11.229691 21859 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:11.182679 26418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.054s	user 0.004s	sys 0.029s
I20260709 13:07:11.187484 26322 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:11.194949 26629 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.005s
I20260709 13:07:11.187892 26690 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.185173 26583 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.225201 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:11.238623 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.236786 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.246037 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.247093 21853 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:11.254307 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.254632 21933 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:11.221969 26672 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:11.224685 26620 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.256377 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:11.226895 26110 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:11.257252 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2573) took 3227 ms (client timeout 1698 ms). Trace:
W20260709 13:07:11.357589 21935 rpcz_store.cc:269] 0709 13:07:08.023035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.027131 (+  4096us) service_pool.cc:224] Handling call
0709 13:07:11.249933 (+3222802us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b5400354dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:11.250049 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2002443136}
W20260709 13:07:11.358588 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.359120 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:11.233351 26418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.004s	sys 0.029s
W20260709 13:07:11.354720 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.233140 26130 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:11.225943 26590 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.356861 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.369426 21859 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:11.371028 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.371809 21929 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:07:11.375031 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.232462 26235 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 13:07:11.380151 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:11.380406 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.381312 22001 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:11.357656 26698 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:11.368765 26510 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:11.404191 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2464) took 1727 ms. Trace:
I20260709 13:07:11.406548 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:11.409873 21853 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:11.407558 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:11.411772 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:11.415496 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.409672 21854 rpcz_store.cc:276] 0709 13:07:09.676947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.210528 (+533581us) service_pool.cc:224] Handling call
0709 13:07:10.211028 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.280180 (+ 69152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.280230 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.320197 (+ 39967us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.040634 (+720437us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:11.040682 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.040763 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.403902 (+363139us) raft_consensus.cc:1685] finished
0709 13:07:11.403932 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.404114 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56601,"raft.queue_time_us":56748,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5175,"thread_start_us":663499,"threads_started":1,"wal-append.queue_time_us":798373}
I20260709 13:07:11.416538 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:11.417340 21935 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:11.417737 21935 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:11.249284 26690 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.380033 26130 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:11.405056 26616 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.000s
I20260709 13:07:11.455587 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1020) took 4033 ms. Trace:
I20260709 13:07:11.388984 26110 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:11.400141 26646 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:11.454820 21935 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.402292 26613 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.463725 22585 rpcz_store.cc:276] 0709 13:07:07.422636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.435183 (+ 12547us) service_pool.cc:224] Handling call
0709 13:07:07.435521 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.830214 (+2394693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.451169 (+1620955us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b540076fcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:11.455521 (+  4352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5413487488}
I20260709 13:07:11.370667 26764 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:07:11.469039 21931 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:11.470706 26659 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.470609 26613 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.456205 26601 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.474884 22593 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:11.471454 26659 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.448678 26510 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.485335 26764 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:11.485787 26764 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:11.470602 26110 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.490674 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2513) took 1274 ms. Trace:
I20260709 13:07:11.490964 26110 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:11.491294 22008 rpcz_store.cc:276] 0709 13:07:10.215956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.785773 (+569817us) service_pool.cc:224] Handling call
0709 13:07:10.786086 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.952056 (+165970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.952100 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:10.952154 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:10.952198 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.490317 (+538119us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002db4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:11.490400 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.490590 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381142400}
I20260709 13:07:11.492982 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.465818 26646 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:11.449038 26698 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.507121 26764 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.508615 26764 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:11.512343 21899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 1.504s (SENT) [suppressed 18 similar messages]
I20260709 13:07:11.497601 26646 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:11.519712 22585 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:11.488860 26656 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.520529 21853 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:11.524410 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.504536 26602 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:11.526773 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.527601 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.540544 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.539000 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:11.542294 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:11.554199 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:11.567739 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:11.514145 26130 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.545352 26110 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:11.574138 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.589295 26613 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:11.590770 26698 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.599676 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.567337 26616 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:11.577311 26377 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:11.579753 26652 raft_consensus.cc:2764] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:07:11.611881 26737 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.474s	user 0.006s	sys 0.010s
I20260709 13:07:11.617794 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:11.630684 22586 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:11.634624 22587 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:11.612591 26766 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:07:11.615484 26735 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.524s	user 0.007s	sys 0.009s
I20260709 13:07:11.613804 26756 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:11.579934 26672 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:07:11.614344 26662 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:11.616353 26377 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:11.579625 26418 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:07:11.651827 22589 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEARNER]: Advancing to term 12
I20260709 13:07:11.675616 22586 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:07:11.633585 26772 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.001s	sys 0.001s
I20260709 13:07:11.665714 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:11.677250 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 101288us
I20260709 13:07:11.681066 22590 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.679139 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.616247 26613 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:07:11.679637 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 456374us
I20260709 13:07:11.681963 22586 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:11.691846 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:11.612211 26784 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.097s
W20260709 13:07:11.678159 26757 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 13:07:11.707377 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.707643 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:11.665446 26756 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.696031 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:11.710386 26597 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.007s
I20260709 13:07:11.721226 26781 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:11.679325 26769 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:07:11.711282 22590 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:11.716596 22005 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:11.617417 26616 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.721884 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.722728 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.722671 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:11.706317 26110 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:11.731295 26377 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:11.733194 26773 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:07:11.735064 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:11.782969 22589 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:11.736184 26591 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:07:11.693588 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 116313us
I20260709 13:07:11.740343 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:35104 (request call id 10) took 1024 ms. Trace:
I20260709 13:07:11.787047 22591 rpcz_store.cc:276] 0709 13:07:10.715827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.716020 (+   193us) service_pool.cc:224] Handling call
0709 13:07:11.739898 (+1023878us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54006676d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707c9853 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691 00007f3b6ed1d60c 00007f3b6ed1c8c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:11.740242 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234241280}
I20260709 13:07:11.788576 22591 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:11.742163 26699 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.790899 26616 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.746093 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2102) took 1721 ms. Trace:
I20260709 13:07:11.793671 21933 rpcz_store.cc:276] 0709 13:07:10.024323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.545685 (+521362us) service_pool.cc:224] Handling call
0709 13:07:11.250226 (+704541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.264488 (+ 14262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.745839 (+481351us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b5400354dd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:11.746033 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614737664}
I20260709 13:07:11.770509 26597 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:11.806123 26616 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.806844 26616 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.773746 26656 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.772186 26781 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.811285 26781 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:11.811913 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:11.813148 22591 raft_consensus.cc:606] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:11.749151 26752 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.705s	user 0.005s	sys 0.003s
I20260709 13:07:11.755913 26596 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:11.760828 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.822888 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.729353 26756 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:11.824258 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.825767 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.826691 22585 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:11.827382 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:11.827689 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:11.828133 22008 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:11.828926 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:11.829041 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:11.830129 22591 raft_consensus.cc:614] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:11.831915 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:11.834376 26767 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.318s	user 0.000s	sys 0.002s
I20260709 13:07:11.835877 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.836649 22591 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:11.837164 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1091) took 1121 ms. Trace:
I20260709 13:07:11.837435 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.837525 22588 rpcz_store.cc:276] 0709 13:07:10.716248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.719399 (+  3151us) service_pool.cc:224] Handling call
0709 13:07:10.748852 (+ 29453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.158079 (+409227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.836922 (+678843us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54006009d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:11.837109 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939506048}
I20260709 13:07:11.838346 22591 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:11.838960 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:11.839401 22591 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:11.842159 26510 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.847113 22591 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:11.846308 26659 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:11.849207 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.850263 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.850611 21935 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.721498 26784 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:07:11.767074 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 141943us
I20260709 13:07:11.726517 26672 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:11.779357 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.781404 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:11.782987 22587 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.709976 26662 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 13:07:11.735282 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.784888 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.892691 22589 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:11.780829 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.767827 26609 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.812461 26616 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:11.814122 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2142) took 1878 ms. Trace:
W20260709 13:07:11.817215 26697 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.648s	user 0.005s	sys 0.006s
W20260709 13:07:11.749171 26778 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
W20260709 13:07:11.766040 26772 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:07:11.852820 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 43808.4us
I20260709 13:07:11.854172 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:11.902805 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48694us
I20260709 13:07:11.855895 22584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:11.893560 22590 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:11.903720 22584 raft_consensus.cc:606] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:11.857359 22005 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.839470 22588 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:11.893704 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:11.896345 21857 rpcz_store.cc:276] 0709 13:07:09.935792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.321087 (+385295us) service_pool.cc:224] Handling call
0709 13:07:10.321533 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.321995 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:10.879952 (+557957us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b18003d4ec0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf 00007f3b707e9038
0709 13:07:11.043304 (+163352us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.475810 (+432506us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.487644 (+ 11834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.488092 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.488241 (+   149us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:11.488272 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.488334 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.813890 (+325556us) raft_consensus.cc:1685] finished
0709 13:07:11.813913 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.814063 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40012,"prepare.queue_time_us":560853,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":216009,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":11830,"spinlock_wait_cycles":95046272,"thread_start_us":573910,"threads_started":3,"wal-append.queue_time_us":423301}
I20260709 13:07:11.832943 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:11.903227 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:11.914326 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.921447 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.923648 21935 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:11.786851 26591 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:11.856622 26672 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:11.930905 21857 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:11.890933 26788 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:11.891049 26271 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:11.940267 26271 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.940680 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:11.901930 26652 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:07:11.941830 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:11.942104 26613 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:11.932298 21932 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:11.912592 26697 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:11.931934 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.933748 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.930709 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:07:11.934829 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.936800 22592 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:11.891880 26659 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:07:11.895005 26597 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:07:11.951790 26597 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:11.950239 21935 raft_consensus.cc:2376] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9. Re-sending same reply.
I20260709 13:07:11.939536 26784 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:11.924901 26322 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:11.952939 26322 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.953373 26322 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:11.953668 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60283.1us
I20260709 13:07:11.955427 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.956303 26784 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.933421 21933 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.942886 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:11.956769 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.957511 22003 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:11.957674 26784 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:11.958346 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:11.958732 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.952147 26597 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:11.939432 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:11.959803 21855 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:11.960834 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 726) took 1433 ms. Trace:
I20260709 13:07:11.958907 22003 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:11.946689 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:11.930636 26740 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:11.966931 21933 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:11.961306 21852 rpcz_store.cc:276] 0709 13:07:10.528089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.528236 (+   147us) service_pool.cc:224] Handling call
0709 13:07:10.528595 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.528847 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:11.059574 (+530727us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.763415 (+703841us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b1800065f80. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287
0709 13:07:11.763517 (+   102us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.763966 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.764097 (+   131us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:11.764334 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.764510 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.764576 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.960613 (+196037us) raft_consensus.cc:1685] finished
0709 13:07:11.960638 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.960776 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":359095,"prepare.queue_time_us":356556,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":2804,"raft.queue_time_us":149842,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":40436,"spinlock_wait_cycles":5568384,"thread_start_us":514487,"threads_started":3,"wal-append.queue_time_us":451081}
W20260709 13:07:11.964960 26688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.921s	user 0.011s	sys 0.023s
I20260709 13:07:11.912209 26699 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:07:11.942338 26271 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:11.973835 22587 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:11.976564 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:11.974391 26377 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:11.912395 26451 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:07:11.987426 26594 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:11.993561 26594 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:11.994427 26594 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.995707 26594 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:11.947944 26788 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:11.980502 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.942603 26613 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:11.983731 22588 raft_consensus.cc:2410] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:07:11.998040 21933 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.001519 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.012063 22003 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:07:12.015643 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69084.8us
I20260709 13:07:12.013762 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:12.015965 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.038019 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.053107 21933 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:07:12.016513 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.000501 26770 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.524s	user 0.001s	sys 0.000s
I20260709 13:07:12.038165 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.061930 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2502) took 1848 ms. Trace:
I20260709 13:07:12.062681 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.033996 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 725) took 2608 ms. Trace:
I20260709 13:07:12.069483 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.071318 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.065162 21932 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
W20260709 13:07:11.984041 26688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.946s	user 0.012s	sys 0.023s
I20260709 13:07:12.069744 22006 rpcz_store.cc:276] 0709 13:07:10.213049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.721972 (+508923us) service_pool.cc:224] Handling call
0709 13:07:10.745345 (+ 23373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.745707 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:10.982929 (+237222us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.403281 (+420352us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.460112 (+ 56831us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.460290 (+   178us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:11.460552 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.460727 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.460818 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.061720 (+600902us) raft_consensus.cc:1685] finished
0709 13:07:12.061741 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.061863 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":305353,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":13428,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":48261,"spinlock_wait_cycles":2304,"thread_start_us":563892,"threads_started":2,"wal-append.queue_time_us":459539}
I20260709 13:07:12.075736 21927 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:12.076027 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:11.997238 26377 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:12.051576 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 130247us
I20260709 13:07:12.077802 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.072239 22003 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.072562 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:11.948537 26110 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.034593 26451 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:12.066766 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 215011us
I20260709 13:07:12.076126 21935 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:11.999579 26740 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:12.085947 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.034945 26619 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:12.034024 26799 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:12.053623 26271 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:12.071197 21858 rpcz_store.cc:276] 0709 13:07:09.431932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.147737 (+715805us) service_pool.cc:224] Handling call
0709 13:07:10.148084 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.744886 (+596802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.033700 (+1288814us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54002b9b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:12.033926 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364640640}
I20260709 13:07:12.001653 26697 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 13:07:12.080557 26377 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:12.014153 26646 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.096273 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.098757 21933 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:12.099812 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.054984 26322 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.087756 26784 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:12.013599 26597 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.012495 26591 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:11.996976 26756 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.099954 26673 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:12.087226 26613 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.124847 26613 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:12.126665 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2589) took 2873 ms (client timeout 2125 ms). Trace:
W20260709 13:07:12.127065 21931 rpcz_store.cc:269] 0709 13:07:09.253119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.253237 (+   118us) service_pool.cc:224] Handling call
0709 13:07:12.126476 (+2873239us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54002e86d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:12.126599 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5037802752}
I20260709 13:07:12.128082 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:12.100150 26271 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:12.108697 26719 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.937s	user 0.014s	sys 0.000s
I20260709 13:07:12.131090 26719 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.015436 26672 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:12.121886 26740 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:12.114056 26802 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:07:12.115190 26646 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.134831 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 728) took 1606 ms. Trace:
I20260709 13:07:12.102162 26781 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:12.115717 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2500) took 1903 ms. Trace:
I20260709 13:07:12.117861 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:12.118749 26322 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:12.119294 21932 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:12.120155 26451 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:12.120772 26659 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.117754 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:12.140153 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.141412 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:12.145717 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47319.1us
I20260709 13:07:12.133386 26719 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:12.172834 22009 rpcz_store.cc:276] 0709 13:07:10.212545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.695214 (+482669us) service_pool.cc:224] Handling call
0709 13:07:10.695626 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.707857 (+ 12231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.707884 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.743685 (+1035801us) spinlock_profiling.cc:243] Waited 952 us on lock 0x7b18002c9fe0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a
0709 13:07:11.744110 (+   425us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:11.744599 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.949578 (+204979us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:11.949620 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.949684 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.115453 (+165769us) raft_consensus.cc:1685] finished
0709 13:07:12.115490 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.115652 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14958,"raft.queue_time_us":14068,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":9883,"spinlock_wait_cycles":8023552,"thread_start_us":204625,"threads_started":1,"wal-append.queue_time_us":309081}
I20260709 13:07:12.164798 21860 rpcz_store.cc:276] 0709 13:07:10.528786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.529025 (+   239us) service_pool.cc:224] Handling call
0709 13:07:10.529469 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.530115 (+   646us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:11.043466 (+513351us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.223088 (+179622us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.674467 (+451379us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.677871 (+  3404us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.678010 (+   139us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:11.678053 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.678121 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.999842 (+321721us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:12.134592 (+134750us) raft_consensus.cc:1685] finished
0709 13:07:12.134616 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.134767 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356682,"prepare.queue_time_us":355096,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":301857,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":27283,"spinlock_wait_cycles":12764032,"thread_start_us":786260,"threads_started":3,"wal-append.queue_time_us":354984}
I20260709 13:07:12.142482 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.207324 22005 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:12.142180 26756 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:12.209223 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.132994 26271 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:12.203294 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.145411 26591 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:12.144320 26788 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:12.205137 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.211511 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:12.193578 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 192061us
I20260709 13:07:12.211591 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.216522 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:12.212869 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2106) took 2117 ms (client timeout 1788 ms). Trace:
I20260709 13:07:12.181262 26597 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.172531 26781 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.216610 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:12.217327 21928 rpcz_store.cc:269] 0709 13:07:10.095671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.096639 (+   968us) service_pool.cc:224] Handling call
0709 13:07:12.212723 (+2116084us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b54003075d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:12.212816 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153675520}
I20260709 13:07:12.149930 26799 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:07:12.219659 26788 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.220269 26788 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.222332 26788 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:12.224978 26758 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:12.149823 26805 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 13:07:12.226547 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.225404 26758 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:12.228174 22009 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.228854 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.229173 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:12.230302 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.230676 22005 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.231591 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.233364 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.234963 26781 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.235416 26765 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:12.235780 26779 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.710s	user 0.002s	sys 0.000s
I20260709 13:07:12.234555 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:12.237560 26805 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:12.238039 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:12.238299 26805 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.238759 26322 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:12.239324 26322 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:12.239952 26771 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.729s	user 0.002s	sys 0.000s
I20260709 13:07:12.239890 26781 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:12.241369 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 947) took 1777 ms. Trace:
I20260709 13:07:12.241766 22000 rpcz_store.cc:276] 0709 13:07:10.463799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.787147 (+323348us) service_pool.cc:224] Handling call
0709 13:07:10.787436 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.787757 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.787782 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.788379 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.279167 (+490788us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:11.279225 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.279315 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.239398 (+960083us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:12.241048 (+  1650us) raft_consensus.cc:1685] finished
0709 13:07:12.241070 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.241303 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1875,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":12354432,"thread_start_us":468437,"threads_started":1,"wal-append.queue_time_us":679844}
I20260709 13:07:12.243427 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.244923 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.245429 26613 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:12.253898 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.254084 26613 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:12.254844 22000 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:12.255796 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.258284 22584 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:12.255311 26613 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.227741 26758 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:12.261804 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:12.262528 22000 raft_consensus.cc:2376] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11. Re-sending same reply.
I20260709 13:07:12.244838 26646 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:12.264397 26646 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.264843 26646 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:12.265035 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.265899 22000 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:12.266825 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.269309 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.270677 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.272541 22000 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.268018 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.274076 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.275339 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:12.276077 22009 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:12.240271 26322 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.268817 22005 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:12.278117 26322 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:12.279917 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.280783 22005 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:07:12.281615 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.276873 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.288672 22005 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.290510 22000 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.292091 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:12.292809 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49467.8us
W20260709 13:07:12.292923 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 211299us
I20260709 13:07:12.293797 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.296855 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.298880 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.282994 26764 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 13:07:12.312887 22007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2517) took 2095 ms (client timeout 1543 ms). Trace:
W20260709 13:07:12.317898 22007 rpcz_store.cc:269] 0709 13:07:10.217047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.274094 (+ 57047us) service_pool.cc:224] Handling call
0709 13:07:12.312685 (+2038591us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b5400426154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:12.312823 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891281152}
I20260709 13:07:12.339082 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.308470 26130 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:12.291045 26377 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.344026 22000 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:12.308717 26651 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.339323 22007 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:12.298663 26594 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:07:12.350625 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.313838 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.310403 26813 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:12.357105 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:12.357167 22007 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:12.319049 26744 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:12.361991 22007 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:12.346063 26719 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:12.385664 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:12.387616 22000 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:12.369436 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 57117.2us
I20260709 13:07:12.363835 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:12.392431 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.361642 26130 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:12.362382 26110 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.316620 26788 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:12.364533 26805 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:12.377084 22008 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:12.358901 26454 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:12.318465 26756 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:12.347375 26688 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:12.397682 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:12.407218 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 331679us
W20260709 13:07:12.406177 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 119215us
I20260709 13:07:12.409411 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:12.410065 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.372316 26620 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:12.364684 26744 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:12.405125 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:12.384958 26451 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:12.312650 26810 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.414512 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.399722 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:12.407733 26688 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:12.421636 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.386775 26510 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:07:12.424283 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.402565 26811 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:12.404582 26805 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.417640 26764 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.436771 22002 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:12.420764 26765 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:12.416958 26652 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:12.445104 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.435339 26672 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:12.456830 26271 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.457885 26271 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.448562 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:12.462028 26744 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.463637 26744 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:12.466627 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.467443 26781 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:12.468103 26781 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:12.451277 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:12.468856 26594 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:12.443413 26620 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.472672 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:12.473213 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.473349 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.474066 21930 raft_consensus.cc:2376] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14. Re-sending same reply.
I20260709 13:07:12.472625 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:12.474710 21931 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:12.474268 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.475106 21928 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:12.475474 21926 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:12.424722 26130 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.476614 22006 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:12.453351 21860 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:12.478039 26817 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:12.478560 26817 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:12.479125 26807 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:12.479313 26616 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:12.479460 26817 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.480192 26807 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:12.480320 26616 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:12.480612 26807 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:12.480728 26616 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:12.407785 26454 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.481683 26616 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.453338 21935 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:12.483906 21928 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:12.474567 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:12.474233 26764 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.501628 22589 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:12.481629 26807 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.447903 26756 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:12.486534 26594 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:12.455667 26377 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:12.452037 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:12.448648 26805 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:12.468079 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:12.513259 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45238.5us
I20260709 13:07:12.513684 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.475699 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:12.515141 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.483139 26454 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.503912 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:12.493818 26688 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.430011 26813 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:12.508136 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41342 (request call id 2521) took 1596 ms. Trace:
I20260709 13:07:12.483057 26616 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:12.450675 21853 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [7.7-10.9]
I20260709 13:07:12.521476 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.500442 22009 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
W20260709 13:07:12.520396 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71313.9us
I20260709 13:07:12.520215 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.460558 26271 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:12.531574 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.540761 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.509836 26756 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.527398 26377 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:12.532442 26788 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:12.521544 22002 rpcz_store.cc:276] 0709 13:07:10.856390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.857101 (+   711us) service_pool.cc:224] Handling call
0709 13:07:12.436625 (+1579524us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b4c0020b240. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6ea9ba20 00007f3b6eaaca29 00007f3b6eaad13f 00007f3b6ea712ab 00007f3b6ea9ef3c 00007f3b6ea9f3fe 00007f3b6ed2c88b 00007f3b7072b495 00007f3b7077c30f 00007f3b707c6a55 00007f3b75796cd0
0709 13:07:12.447979 (+ 11354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3109120}
I20260709 13:07:12.564556 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.565243 22590 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:12.521514 26765 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:12.520939 26451 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 13:07:12.576344 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 54942.8us
I20260709 13:07:12.541834 21932 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:12.544495 26591 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:12.581990 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.544750 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:12.536370 26646 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.545482 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:12.549314 22591 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:12.491842 26758 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:12.551839 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.558493 26756 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:12.562739 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.557773 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:12.563392 21758 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 11, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 11 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:12.522256 26781 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.533849 26807 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:12.626294 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47404.2us
W20260709 13:07:12.627233 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.628489 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51266.4us
I20260709 13:07:12.629235 26788 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:12.629931 26788 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:12.631433 26777 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.017s	user 0.002s	sys 0.001s
I20260709 13:07:12.634724 26758 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.540059 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:12.635810 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 91772us
I20260709 13:07:12.617422 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:12.636333 22005 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:12.431954 26810 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.579824 26813 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:12.615248 22003 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:12.575300 26765 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.576749 26377 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:12.552031 26664 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:12.558619 26594 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:12.625604 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:12.648108 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:12.636992 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.637117 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.634750 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 729) took 2097 ms. Trace:
I20260709 13:07:12.575023 26510 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 13:07:12.615732 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2523) took 1424 ms (client timeout 1820 ms). Trace:
W20260709 13:07:12.651974 22001 rpcz_store.cc:269] 0709 13:07:11.191038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.191175 (+   137us) service_pool.cc:224] Handling call
0709 13:07:12.608061 (+1416886us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540042c2d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:12.615676 (+  7615us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326963968}
I20260709 13:07:12.641645 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:12.554674 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.020s	sys 0.014s
I20260709 13:07:12.577221 26764 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.643179 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:12.655710 22009 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:07:12.656142 22009 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
I20260709 13:07:12.656388 22009 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:07:12.656256 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.646922 21855 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:12.649825 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.649219 21851 rpcz_store.cc:276] 0709 13:07:10.537408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.875951 (+338543us) service_pool.cc:224] Handling call
0709 13:07:10.881469 (+  5518us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:10.881877 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.881910 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:10.882550 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 13:07:11.370495 (+487945us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:11.370533 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.370592 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.630715 (+1260123us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:12.634440 (+  3725us) raft_consensus.cc:1685] finished
0709 13:07:12.634462 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.634686 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":15863040,"thread_start_us":487569,"threads_started":1,"wal-append.queue_time_us":669852}
I20260709 13:07:12.568056 26271 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:07:12.624636 26811 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:12.655180 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.426s	user 0.021s	sys 0.015s
I20260709 13:07:12.640456 26591 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:12.656867 22009 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:12.660957 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:12.658790 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.647951 26662 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:12.642123 26765 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:12.720311 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59412.1us
W20260709 13:07:12.714912 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.48578e+06us
W20260709 13:07:12.721391 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 156924us
I20260709 13:07:12.721689 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.723587 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:12.720571 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 70859.3us
I20260709 13:07:12.724112 26811 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.725492 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:12.657146 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:12.659855 26781 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.709849 26764 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:12.710390 26130 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:12.661986 26758 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:12.737579 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80505.6us
I20260709 13:07:12.702526 26821 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:12.659679 26594 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:12.651793 26676 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:07:12.701891 26815 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:07:12.727098 26591 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:12.733407 26699 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.735070 26454 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:12.746816 26418 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:12.744017 26829 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:12.727473 26662 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:12.768857 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45362.8us
I20260709 13:07:12.750952 26764 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:12.773113 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:12.776013 26418 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:12.788147 26662 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.773669 26758 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:12.753588 26377 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.789059 26594 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.763165 26699 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:12.799067 26764 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.781203 26831 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:12.773940 26271 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:12.811291 26810 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:12.755303 26821 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:12.814673 26821 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:12.815976 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:12.816756 21932 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:12.782126 26834 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.823154 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:12.838399 22593 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:12.820818 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:12.923595 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 102878us
W20260709 13:07:12.927995 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 202659us
I20260709 13:07:12.929970 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.930732 21853 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:12.932291 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:12.950435 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:12.952168 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 293406us
I20260709 13:07:12.952958 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.954424 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:12.954799 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.955475 21859 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:07:12.957029 26766 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.263s	user 0.007s	sys 0.002s
I20260709 13:07:12.961252 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.962263 21851 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17.
I20260709 13:07:12.963424 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.822180 26646 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:12.966389 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:12.971772 21858 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:12.966580 22003 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:07:12.967914 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.968524 21852 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:12.969530 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:12.975513 22003 pending_rounds.cc:85] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
I20260709 13:07:12.976269 22003 pending_rounds.cc:107] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:07:12.987397 22003 raft_consensus.cc:2889] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:12.988608 21859 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:07:12.976236 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 254511us
I20260709 13:07:12.920656 26821 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:12.949419 26830 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.980453 21854 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:07:12.959046 26827 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:07:12.960330 26427 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.955785 26676 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:12.995463 26744 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.000s
I20260709 13:07:12.985914 26646 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:13.037428 26646 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.038760 26646 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.982908 26418 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.045658 26690 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:13.000080 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.056246 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.060051 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69736.5us
I20260709 13:07:13.061093 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:12.990819 22003 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:13.004501 21851 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:13.062036 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.008693 22593 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:13.013939 26831 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.978386 26837 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 13:07:13.032104 26776 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.442s	user 0.003s	sys 0.000s
I20260709 13:07:13.034322 26830 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:12.833379 26591 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.036487 26676 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.063797 26784 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 13:07:13.046707 26672 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
W20260709 13:07:13.047703 26805 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
W20260709 13:07:13.050309 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.997865 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.051047 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50845.6us
I20260709 13:07:13.101832 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:13.106169 22003 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:13.066964 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:13.067507 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:13.070204 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:12.970744 26271 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:07:13.072337 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:13.070873 26766 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:07:12.996096 26664 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:13.095849 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 314069us
I20260709 13:07:13.108899 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.114326 21935 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:13.109916 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:13.128957 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.130956 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:13.071175 26758 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:13.121889 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.073319 26676 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.073449 26744 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:13.136039 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.109902 26784 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:13.140654 21851 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:13.020694 26765 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:13.146085 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 155289us
I20260709 13:07:13.165639 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.123087 26407 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.123204 26415 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:13.139956 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 73116.8us
I20260709 13:07:13.142864 26630 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:13.151759 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.147198 21932 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:13.123167 26805 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.059489 26821 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.142560 26831 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:13.164194 26510 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:13.150504 26744 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.148948 26836 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.184165 22003 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:13.160626 26758 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.134526 26756 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.145932 26690 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:07:13.193212 22007 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 13. Ops: []
I20260709 13:07:13.194177 22593 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:13.160921 26815 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:07:13.165725 26454 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:07:13.169999 26845 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:13.203949 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:13.182127 26841 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:13.206300 22003 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.181169 26630 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:13.169896 26835 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:07:13.181857 26784 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:07:13.197979 26832 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:07:13.160988 26829 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:07:13.160211 26817 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:07:13.217615 22003 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:13.227972 21932 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:13.170236 26796 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:07:13.232415 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:13.233850 26843 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:07:13.236136 22003 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:07:13.244803 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 97762.8us
I20260709 13:07:13.246235 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.247228 22003 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:13.225817 26630 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.251632 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:13.255440 22003 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:13.214164 26841 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:13.263152 26756 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.257380 26841 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:13.267954 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 959) took 1284 ms. Trace:
I20260709 13:07:13.268378 22006 rpcz_store.cc:276] 0709 13:07:11.983836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.100750 (+116914us) service_pool.cc:224] Handling call
0709 13:07:13.267741 (+1166991us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400425ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:13.267892 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625383680}
I20260709 13:07:13.270206 26815 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.270447 26834 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:13.271044 26837 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:13.270828 26815 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.271449 26837 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.273521 26815 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.273082 26758 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.275365 26805 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.276204 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2616) took 1561 ms. Trace:
I20260709 13:07:13.277047 26656 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:13.277441 26656 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.279915 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:13.280552 22586 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.280700 22584 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:13.281522 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.282301 21935 raft_consensus.cc:2376] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15. Re-sending same reply.
I20260709 13:07:13.276595 22004 rpcz_store.cc:276] 0709 13:07:11.714726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.714861 (+   135us) service_pool.cc:224] Handling call
0709 13:07:12.616140 (+901279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.616356 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:12.619387 (+  3031us) log.cc:844] Serialized 14 byte log entry
0709 13:07:12.749525 (+130138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:12.749579 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:12.749623 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.749686 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.276014 (+526328us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b540042c2d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:13.276141 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2810,"prepare.queue_time_us":3759,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1063,"raft.queue_time_us":1660,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3607,"spinlock_wait_cycles":1983481728,"thread_start_us":129792,"threads_started":1,"wal-append.queue_time_us":165843}
I20260709 13:07:13.285779 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:13.286453 21859 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.287442 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.297760 21851 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:13.279111 26764 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.302210 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.309943 21852 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:13.272406 26837 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.303251 21859 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:13.254768 26744 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.334803 22586 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:13.329190 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.336755 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:13.341763 22004 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.342665 22006 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:13.345551 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.312356 26830 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.311805 26813 raft_consensus.cc:2764] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:13.346716 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.347688 26758 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.348793 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:13.348654 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:13.351668 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.324126 26845 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:13.361279 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.375509 22584 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.375712 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.376689 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.406576 21935 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.382630 26407 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:13.383332 26852 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:07:13.384287 26849 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:07:13.385306 26850 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:07:13.385766 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.433871 21926 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:13.387413 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.388621 26130 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:13.396745 22004 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:13.397815 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:13.381660 26853 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:13.411348 21858 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17.
I20260709 13:07:13.436715 21853 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:13.320600 26805 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:07:13.421494 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:13.411911 26690 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:13.352151 26646 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.336424 26597 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:07:13.350356 26765 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.435773 26130 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:13.430116 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 81392.5us
I20260709 13:07:13.351348 26837 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.313256 26764 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.436236 26596 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:13.459292 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.397384 26758 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.442360 22006 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:13.442766 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.440062 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 260681us
I20260709 13:07:13.460624 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:13.426855 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.522753 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:13.480078 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.481256 26764 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:13.482463 21856 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:07:13.487259 26851 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
I20260709 13:07:13.427716 26817 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:07:13.479064 26646 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.495715 26841 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.496870 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.551434 21854 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:13.498862 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.552464 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2141) took 1079 ms. Trace:
I20260709 13:07:13.552802 21929 rpcz_store.cc:276] 0709 13:07:12.472850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.474048 (+  1198us) service_pool.cc:224] Handling call
0709 13:07:13.498695 (+1024647us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:13.552408 (+ 53713us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49371520}
I20260709 13:07:13.499953 26662 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:13.564867 26662 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:13.565797 26662 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.566565 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.567057 26662 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.567380 21855 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:13.568312 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.568943 21855 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:07:13.436255 26844 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:07:13.485383 26676 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.571417 26836 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:13.572371 26676 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.501741 26130 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.507572 26630 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.467785 26845 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:13.514701 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:13.574134 21852 raft_consensus.cc:2376] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13. Re-sending same reply.
I20260709 13:07:13.574550 26630 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:13.575184 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 949) took 1672 ms. Trace:
I20260709 13:07:13.575590 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2618) took 1850 ms. Trace:
I20260709 13:07:13.575752 21933 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:13.575995 22005 rpcz_store.cc:276] 0709 13:07:11.725537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.299445 (+573908us) service_pool.cc:224] Handling call
0709 13:07:12.311952 (+ 12507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.637059 (+325107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.575369 (+938310us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b540045abd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:13.575536 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745461632}
I20260709 13:07:13.511720 26656 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.577193 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.577720 26656 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.579806 26841 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:13.581483 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.581964 26821 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.582136 22006 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:13.517655 22002 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:13.582563 26235 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:13.518589 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.583269 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.583724 22003 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:13.519264 26377 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.462409 26427 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:13.584641 26377 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.531348 21935 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:13.520354 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:13.586511 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66289.7us
I20260709 13:07:13.539474 22584 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:13.539366 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.588613 21860 raft_consensus.cc:2368] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14.
I20260709 13:07:13.559051 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.560400 21934 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:13.562925 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.590603 22004 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:13.590838 21853 raft_consensus.cc:2368] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14.
I20260709 13:07:13.563516 26758 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:13.564328 26860 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:13.572140 26651 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:13.538753 26271 raft_consensus.cc:2764] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:07:13.574209 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2629) took 1321 ms. Trace:
I20260709 13:07:13.597177 22009 rpcz_store.cc:276] 0709 13:07:12.252803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.537452 (+284649us) service_pool.cc:224] Handling call
0709 13:07:12.537756 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.719870 (+182114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.573974 (+854104us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400166554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:13.574146 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264555776}
I20260709 13:07:13.516479 26688 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.607115 21758 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.584992 26427 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:13.615129 26427 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.479043 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 967) took 1035 ms. Trace:
I20260709 13:07:13.617189 22008 rpcz_store.cc:276] 0709 13:07:12.444131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.702267 (+258136us) service_pool.cc:224] Handling call
0709 13:07:12.727966 (+ 25699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:12.735569 (+  7603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.735619 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.038357 (+302738us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:13.038857 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.164494 (+125637us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:13.164544 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.164645 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:13.478762 (+314117us) raft_consensus.cc:1685] finished
0709 13:07:13.478783 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.478984 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3499,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10927,"spinlock_wait_cycles":16630144,"thread_start_us":124556,"threads_started":1,"wal-append.queue_time_us":213323}
I20260709 13:07:13.554248 26798 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:13.575582 22007 rpcz_store.cc:276] 0709 13:07:11.902462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.363084 (+460622us) service_pool.cc:224] Handling call
0709 13:07:12.363356 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.197796 (+834440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.574946 (+377150us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b540042b3d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:13.575127 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907967232}
I20260709 13:07:13.558948 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.619232 26798 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:13.619673 21926 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:13.620357 26798 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.621670 26798 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.572698 26690 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:13.587250 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.608884 21758 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 11, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 11 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:13.624063 21758 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.594019 21853 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:13.548676 26870 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:13.625801 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.617190 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.596148 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.629467 21934 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:13.598613 26860 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.605698 21928 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:13.606145 26684 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:13.611078 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:13.611284 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.612751 26377 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:13.614038 26859 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.202s	user 0.002s	sys 0.000s
I20260709 13:07:13.632009 21926 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:13.628281 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.645367 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.650490 21929 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:13.595973 26676 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:13.658924 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.662343 21930 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:13.664213 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.664598 26784 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.650557 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.665323 21926 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:13.665208 26784 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.666616 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.667600 26784 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.667799 21934 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:13.668166 26784 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.669019 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:13.670017 26377 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:13.670665 21898 leader_election.cc:400] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17.
I20260709 13:07:13.671064 21898 leader_election.cc:403] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:13.671693 26377 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17.
I20260709 13:07:13.669709 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.672609 26732 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.673171 26732 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.673951 21927 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:13.676025 26732 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.676579 26732 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.677428 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.678148 21927 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:13.679615 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.680186 21927 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
W20260709 13:07:13.680622 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.681562 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:13.683670 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.683823 22009 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:13.683725 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.684419 22591 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:13.684664 22584 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:13.686617 22009 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:13.687788 26732 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:13.688853 22009 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:13.672132 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:13.690553 26377 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:13.690972 26377 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:13.691424 26377 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.692142 26732 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.668663 26784 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.697587 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.698812 26784 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.701159 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:13.707396 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.708305 22009 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:13.708205 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:13.641559 26684 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:13.639930 26799 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:13.730564 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.655242 26834 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:13.639708 26758 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:13.724893 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:13.655884 26110 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:13.736495 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.699783 26676 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.737176 21931 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:13.741489 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:13.738322 26110 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:13.742836 26684 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.747720 26110 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.754016 21928 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:13.754698 26377 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:13.757270 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 331787us
I20260709 13:07:13.721789 26451 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:13.760914 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.762845 22591 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:13.767336 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.748814 26110 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:13.738453 26781 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:13.745672 26676 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.721328 26765 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:07:13.651280 26870 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:13.779618 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.758805 26799 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:13.745999 26864 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:13.765292 26788 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.811395 26758 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.779161 26619 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:13.781255 26781 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:13.823431 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 283354us
W20260709 13:07:13.828043 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.820122 26862 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:13.814764 26377 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.815892 26864 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:13.829907 22556 leader_election.cc:400] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14.
I20260709 13:07:13.776700 26451 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.830618 22556 leader_election.cc:403] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:13.823768 26811 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.812320 26841 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:07:13.829571 26271 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:13.840013 21929 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:07:13.821785 26788 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.831698 26864 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.846206 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:13.847167 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 195156us
I20260709 13:07:13.842428 26860 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.866173 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.869621 21851 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:13.867249 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.845646 26271 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:13.836023 26811 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.879292 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.897027 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:13.898998 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.844288 26799 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.883823 26837 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:13.884241 26784 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.906174 22009 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:13.909324 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.899986 26845 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:13.904032 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:13.847738 26781 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.913712 22009 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:13.898672 26839 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14.
I20260709 13:07:13.904917 26837 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:13.915063 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.915058 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.868275 26758 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.898402 26834 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:13.902882 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 161464us
W20260709 13:07:13.912590 26867 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:07:13.916541 26845 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.905851 26860 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:07:13.930949 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:13.915577 26870 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:13.939215 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.941222 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.941517 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.922269 26837 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.940402 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:13.939198 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:13.955844 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:13.910580 26784 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:13.965556 21854 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:07:13.930680 26830 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.950839 26845 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:13.933801 26883 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:13.956540 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
W20260709 13:07:13.968768 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 133631us
I20260709 13:07:13.953586 26860 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:13.952860 26110 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:13.975394 22009 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:13.979219 22009 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:13.916057 26271 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.976866 21974 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.977274 22001 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:13.936349 26841 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:07:13.960325 26872 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:13.968506 26697 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:13.975986 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:13.969333 26451 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:13.989769 22007 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:13.985296 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.993057 22008 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:13.993512 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:13.980019 22009 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.988255 26872 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:13.988092 26799 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:13.997175 26872 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.997032 26688 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 13:07:13.998575 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.969869 26377 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:07:13.989207 26882 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.994829 26788 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.995736 26697 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:14.004590 26844 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:13.980749 26830 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:13.980429 26883 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.006587 26799 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.037683 26883 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.039901 26676 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.042382 26676 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.998293 26839 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:07:14.044662 26845 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.045462 26839 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:14.045850 26839 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:14.046695 26839 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.047955 26839 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.000074 26427 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:07:14.011489 26619 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:07:14.010010 26887 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:14.051229 26887 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.051672 26887 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.053165 26845 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:14.027369 26889 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:07:13.989733 26834 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:14.030434 26888 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:07:14.017427 26882 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.088512 21933 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:14.089783 26882 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.035272 26784 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:13.988793 26110 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.013902 26811 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:14.070245 26893 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:07:14.092263 26896 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.092870 26896 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.078871 26699 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.018800 26697 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:14.094653 26656 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:07:14.114761 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.114662 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:14.115729 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:14.117066 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.121079 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.123786 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:14.117609 21934 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:14.092304 26646 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.137490 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.124279 21932 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:14.137799 22589 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:14.140058 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.139597 21934 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:14.140707 21930 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:14.126967 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:14.142820 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:14.143445 21926 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.123003 26110 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:14.146556 26895 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:14.142764 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.152930 22008 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:07:14.153095 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.153260 21926 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:14.153481 22008 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 7
I20260709 13:07:14.153827 22008 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:07:14.153790 21927 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:14.154225 22007 raft_consensus.cc:1217] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[9.6-13.8]   Dedup: 9.6->[11.7-13.8]
I20260709 13:07:14.154881 22007 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [9.6-13.8]
I20260709 13:07:14.156213 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:14.156786 22004 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:14.156790 26896 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.154234 22008 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:14.158951 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:14.159667 21930 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.135347 26271 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.160502 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.162216 21934 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:14.166482 22004 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:14.165222 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.168819 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:14.169469 22004 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:14.169776 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.099309 26864 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.171576 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.162431 26900 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.162822 26811 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.181027 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.161929 26646 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:14.163647 26699 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.176640 26900 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.191735 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.175735 26110 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:14.192442 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:14.191818 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 198376us
I20260709 13:07:14.193786 21934 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:14.179965 26784 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:14.178112 26842 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:14.203862 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:14.204648 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.208053 21930 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:07:14.210157 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 204036us
I20260709 13:07:14.207540 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.184356 26130 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:14.213521 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:14.185056 26830 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.218075 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:14.215410 21974 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.175706 26656 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:07:14.190120 26862 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:14.191802 26629 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:14.177803 26882 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:07:14.338774 26629 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.339231 26629 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.216059 26235 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:14.341094 26902 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.341593 26902 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.343889 26902 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.211324 26813 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.204886 26784 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.347888 22591 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:14.302826 22584 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEARNER]: Advancing to term 13
W20260709 13:07:14.327075 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.269357 26847 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.075s	user 0.002s	sys 0.003s
I20260709 13:07:14.216755 26619 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:14.278020 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 56943.7us
I20260709 13:07:14.195241 26699 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.314138 26837 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.214066 26688 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:14.269907 26781 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:14.218871 26676 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:14.279124 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.270797 26807 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:14.266177 26744 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.202013 26860 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:14.349735 26784 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:14.334916 26830 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:07:14.270862 26872 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:14.335363 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 375580us
I20260709 13:07:14.212301 26110 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.202355 26788 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:07:14.314980 26900 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:14.277496 26836 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 13:07:14.336508 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.436801 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.437700 22589 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:14.312942 26909 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.305576 26887 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:14.368129 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.368656 22584 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:14.336886 26842 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:14.347703 26813 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.299609 22593 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:14.357625 26619 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.409758 26688 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:14.387255 26860 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.361307 26656 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:14.460575 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.461726 21931 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:14.338378 26862 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.363891 26271 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:14.407083 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 191269us
I20260709 13:07:14.466401 21974 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:14.377734 26807 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:14.465922 26836 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.469966 26842 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.470523 26807 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.453397 26918 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:14.391136 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:14.471560 26807 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.471815 26918 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.472241 26918 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:14.472385 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81374us
I20260709 13:07:14.439321 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.416800 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2484) took 1458 ms. Trace:
I20260709 13:07:14.460727 26619 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:14.444478 26907 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:07:14.446062 26510 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:14.475687 26510 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:14.450371 26872 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:13.653520 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:14.476562 26510 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.478230 26510 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.479202 26916 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:14.411429 26110 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:14.479809 26916 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.481206 26916 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:14.482699 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.483906 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.484898 26917 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.485610 26917 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.479097 26882 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:14.487879 26882 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.488317 26882 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.462936 26813 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.496214 26684 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.479900 26690 raft_consensus.cc:993] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:07:14.497031 26690 raft_consensus.cc:1081] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:07:14.474862 21857 rpcz_store.cc:276] 0709 13:07:12.958302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.958431 (+   129us) service_pool.cc:224] Handling call
0709 13:07:12.958708 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:12.959063 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.959105 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:12.968742 (+  9637us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.072897 (+104155us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:13.072943 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.073427 (+   484us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.176855 (+1103428us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:14.407510 (+230655us) raft_consensus.cc:1685] finished
0709 13:07:14.407531 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.416726 (+  9195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1962,"raft.queue_time_us":85,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":16228,"spinlock_wait_cycles":7590272,"thread_start_us":81333,"threads_started":1,"wal-append.queue_time_us":170171}
I20260709 13:07:14.501529 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:14.502318 22593 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:14.503005 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.504467 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.462687 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.467762 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.450405 26699 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.471102 26836 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.474299 26898 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:14.474875 26271 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:14.479735 26646 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:14.480370 26915 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:14.506292 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48750.2us
I20260709 13:07:14.531563 22002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:32904
I20260709 13:07:14.532352 22002 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.532783 22002 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.461674 21758 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 5 to 8, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 8 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.508811 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:14.503839 21857 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:14.414952 26913 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:07:14.467394 26862 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.530395 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.545783 26864 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.564442 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.519971 26830 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:14.520645 26898 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:14.577342 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.576903 26619 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.578267 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2501) took 1233 ms. Trace:
I20260709 13:07:14.578655 21858 rpcz_store.cc:276] 0709 13:07:13.344757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.514378 (+169621us) service_pool.cc:224] Handling call
0709 13:07:14.577173 (+1062795us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:14.578203 (+  1030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12106624}
I20260709 13:07:14.579854 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:14.580544 21858 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:14.580744 26619 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:14.583568 26620 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:14.583940 26620 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.584261 26620 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.587646 26900 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:14.591683 26885 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.585s	user 0.003s	sys 0.000s
W20260709 13:07:14.593848 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.593477 26826 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:14.529421 26788 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.597381 26788 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.594614 22002 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.599386 26619 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:14.599689 26788 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:14.599948 26619 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:14.600208 26788 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.593991 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2492) took 1251 ms. Trace:
W20260709 13:07:14.602371 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 72070.7us
I20260709 13:07:14.602562 21856 rpcz_store.cc:276] 0709 13:07:13.342528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.343075 (+   547us) service_pool.cc:224] Handling call
0709 13:07:13.385429 (+ 42354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.540157 (+154728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.593731 (+1053574us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b540008acd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:14.593922 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222331648}
I20260709 13:07:14.608587 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2175) took 1102 ms. Trace:
I20260709 13:07:14.600874 26619 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.610941 26870 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.454526 26909 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.608948 21853 rpcz_store.cc:276] 0709 13:07:13.506204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.625697 (+119493us) service_pool.cc:224] Handling call
0709 13:07:13.625985 (+   288us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:13.641174 (+ 15189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.641205 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:13.655426 (+ 14221us) log.cc:844] Serialized 52 byte log entry
0709 13:07:13.863214 (+207788us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:13.863270 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.863360 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.591018 (+727658us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:14.608320 (+ 17302us) raft_consensus.cc:1685] finished
0709 13:07:14.608341 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.608530 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":15496,"spinlock_wait_cycles":7113088,"thread_start_us":124275,"threads_started":1,"wal-append.queue_time_us":244787}
I20260709 13:07:14.612257 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:14.614136 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 78338.8us
I20260709 13:07:14.616907 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.617165 22002 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.617602 21858 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:14.619274 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.620108 22586 raft_consensus.cc:1217] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Deduplicated request from leader. Original: 9.8->[10.9-10.9]   Dedup: 10.9->[]
I20260709 13:07:14.600654 26788 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.626253 21856 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:14.620801 22586 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:07:14.612848 26676 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:14.626494 26831 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.002s	sys 0.026s
I20260709 13:07:14.632273 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:14.615401 26860 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.549194 26915 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.644318 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:14.616214 26917 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:14.646100 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:14.607565 26916 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:14.642305 26831 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.002s	sys 0.027s
I20260709 13:07:14.635422 26656 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:14.607862 26830 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.651444 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:14.638266 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:14.641535 26138 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:07:14.645052 26873 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:14.652832 26377 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:14.607321 26688 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.626147 26870 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:14.656142 26912 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
W20260709 13:07:14.655792 26919 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:14.660291 26407 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:14.656152 26845 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.658015 26646 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.662097 26860 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.656040 26834 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:14.716450 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:14.713249 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.720570 21857 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:14.726626 21859 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:14.727769 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.728490 21859 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:14.710584 26864 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.710338 26887 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.661226 26656 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.664585 26664 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:07:14.738205 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.719434 26928 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.722553 26877 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:14.750602 22592 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:14.656693 26898 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.751124 22592 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:14.760699 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 122618us
I20260709 13:07:14.737567 26864 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.729631 26676 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.753260 26928 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.778288 21851 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:14.732817 26834 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.723968 26845 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.727991 26646 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:14.757467 26924 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:07:14.765834 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2177) took 1259 ms. Trace:
I20260709 13:07:14.785207 21854 rpcz_store.cc:276] 0709 13:07:13.506801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.967267 (+460466us) service_pool.cc:224] Handling call
0709 13:07:13.967589 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:14.001578 (+ 33989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.001613 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:14.002120 (+   507us) log.cc:844] Serialized 52 byte log entry
0709 13:07:14.210606 (+208486us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:14.210663 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.210749 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.765652 (+554903us) raft_consensus.cc:1685] finished
0709 13:07:14.765672 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.765790 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60488,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":140898,"threads_started":2,"wal-append.queue_time_us":369792}
I20260709 13:07:14.786446 26927 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.787286 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:14.769977 26920 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:14.787931 21854 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:14.788156 26920 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.725625 26882 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.767000 26922 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:14.773022 26916 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:14.764858 26697 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.001s
I20260709 13:07:14.782433 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:14.787002 26927 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.737880 26836 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.727384 26860 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:14.805187 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 166952us
I20260709 13:07:14.792958 21859 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:14.808116 22593 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:14.806813 26898 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.812757 26656 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.814095 26646 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.817135 26697 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.817546 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.818123 21856 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:14.819011 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:14.819182 26697 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.819711 21856 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:14.819468 26813 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.820989 26928 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.823029 26882 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.823166 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2507) took 1300 ms. Trace:
I20260709 13:07:14.823776 21853 rpcz_store.cc:276] 0709 13:07:13.522378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.822264 (+1299886us) service_pool.cc:224] Handling call
0709 13:07:14.822659 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.822838 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.822866 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:14.822909 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:14.822944 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.823012 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.823114 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:14.824553 26920 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.825760 26870 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.759680 26877 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.826251 26629 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:14.828405 26877 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.828501 26629 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.831840 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.832525 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:14.826201 21858 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:14.833941 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.835212 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:14.783200 26664 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:14.840315 22593 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:14.842111 21974 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:14.820992 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2506) took 1299 ms. Trace:
I20260709 13:07:14.834038 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 740) took 1296 ms. Trace:
I20260709 13:07:14.842964 21858 rpcz_store.cc:276] 0709 13:07:13.538084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.825703 (+1287619us) service_pool.cc:224] Handling call
0709 13:07:14.826022 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.833784 (+  7762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.833976 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:14.843456 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 182340us
I20260709 13:07:14.837931 26834 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.839359 26864 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.844512 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:14.842890 21856 rpcz_store.cc:276] 0709 13:07:13.522038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.820277 (+1298239us) service_pool.cc:224] Handling call
0709 13:07:14.820553 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.820697 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.820721 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:14.820762 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:14.820792 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.820843 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.820940 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:14.844509 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.806067 26922 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.854373 22589 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:14.856030 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.856603 21858 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:14.857410 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:14.857998 21858 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:14.867437 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:14.870313 21856 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:14.870983 21854 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:14.835384 26877 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.880983 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:14.873780 21858 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:14.885663 21854 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:14.868957 26920 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:14.889846 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:14.888947 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.886341 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2178) took 1260 ms. Trace:
I20260709 13:07:14.905148 21854 rpcz_store.cc:276] 0709 13:07:13.626100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.871692 (+1245592us) service_pool.cc:224] Handling call
0709 13:07:14.871997 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.886119 (+ 14122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.886288 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:14.872985 26684 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:14.862452 26922 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:14.886276 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 741) took 1348 ms. Trace:
I20260709 13:07:14.912079 21858 rpcz_store.cc:276] 0709 13:07:13.538359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.872281 (+1333922us) service_pool.cc:224] Handling call
0709 13:07:14.873410 (+  1129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.886014 (+ 12604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.886216 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:14.914002 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.917577 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:14.918882 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:14.921661 22007 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:07:14.823050 26918 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.853165 26664 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.891427 26808 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:14.872692 26138 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:07:14.909664 26407 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:14.933297 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:14.891157 26934 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:14.873191 26836 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:07:14.873310 26697 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:07:14.890249 26920 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:14.929622 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.939713 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.940642 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.888705 26877 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:14.961349 21933 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:07:14.967557 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78669.2us
I20260709 13:07:14.969102 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.969563 21930 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:07:14.969784 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.970579 21934 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:14.971678 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.972514 21933 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:14.975010 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:14.931013 26744 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.002s
I20260709 13:07:14.924623 26916 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:14.979265 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:14.969897 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:14.977788 21933 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:14.926533 26933 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:14.978991 21926 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:14.983340 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:14.937089 26784 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.004s
I20260709 13:07:14.985433 21854 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:14.991194 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.985323 26916 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.989161 26933 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.989624 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:14.931905 26845 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.938004 26932 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.961200 26920 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.996347 26619 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.968483 26934 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:14.998096 26928 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:14.998225 26664 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:14.981794 26860 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:14.994046 26870 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.012590 21931 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:14.995364 26916 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.017179 21932 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:15.018743 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:14.998523 26933 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:15.020390 21856 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:15.024838 21856 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: [7.7-10.9]
I20260709 13:07:15.025678 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2183) took 1338 ms. Trace:
I20260709 13:07:15.026078 21856 rpcz_store.cc:276] 0709 13:07:13.687593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.023969 (+1336376us) service_pool.cc:224] Handling call
0709 13:07:15.024464 (+   495us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:15.025335 (+   871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.025610 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.027633 21856 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:15.035516 26927 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.036667 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:15.037290 26877 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.036623 26620 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.001426 26619 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.043494 26619 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:15.044241 26934 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.040858 26837 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.002910 26932 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.042614 22007 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:15.020676 26870 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.049374 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:15.049913 26932 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:15.023231 26858 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.589s	user 0.004s	sys 0.000s
I20260709 13:07:15.042531 26918 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.054126 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 737) took 1517 ms. Trace:
I20260709 13:07:15.054613 26651 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:15.055106 26651 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.055694 26651 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:15.054760 21853 rpcz_store.cc:276] 0709 13:07:13.537160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.824883 (+1287723us) service_pool.cc:224] Handling call
0709 13:07:14.835647 (+ 10764us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:14.851647 (+ 16000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.851699 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:14.852894 (+  1195us) log.cc:844] Serialized 52 byte log entry
0709 13:07:14.921251 (+ 68357us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:14.921293 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.921735 (+   442us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.053877 (+132142us) raft_consensus.cc:1685] finished
0709 13:07:15.053897 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.054066 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2394,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":31942,"threads_started":1,"wal-append.queue_time_us":128439}
I20260709 13:07:15.056265 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2541) took 1956 ms. Trace:
I20260709 13:07:15.056835 22000 rpcz_store.cc:276] 0709 13:07:13.100089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.102745 (+  2656us) service_pool.cc:224] Handling call
0709 13:07:13.103194 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.103632 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.103672 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.119377 (+ 15705us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.246412 (+127035us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:13.246453 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.247632 (+  1179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.904002 (+1656370us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:15.055861 (+151859us) raft_consensus.cc:1685] finished
0709 13:07:15.055893 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.056184 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":726,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":21480,"spinlock_wait_cycles":18223488,"thread_start_us":81730,"threads_started":1,"wal-append.queue_time_us":187214}
I20260709 13:07:15.053438 26941 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.061352 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.062003 22584 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:15.061106 26941 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.075434 22584 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:14.999143 26784 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.082683 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.083511 21935 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.091656 21935 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:15.077432 26870 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.094724 21856 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:15.094853 26941 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.096125 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.096416 26909 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.106429 26620 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.135077 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.128683 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.137859 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:15.137979 22584 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:15.138731 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2510) took 1036 ms. Trace:
I20260709 13:07:15.139102 21855 rpcz_store.cc:276] 0709 13:07:14.102201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.104675 (+1002474us) service_pool.cc:224] Handling call
0709 13:07:15.138665 (+ 33990us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:15.108904 26862 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.001s
I20260709 13:07:15.140801 26862 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.143759 26862 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:15.145081 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 175235us
I20260709 13:07:15.149403 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.149636 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.149856 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.157400 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.158003 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:15.158082 21935 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:15.161980 22007 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:07:15.171859 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.108565 26916 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.188651 26664 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:07:15.236095 26684 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:15.189639 26941 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:07:15.189256 26620 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:07:15.190240 26833 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:07:15.175988 26928 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:15.253232 26941 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:15.250799 26922 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.256857 26619 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.290371 21852 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:15.291618 26920 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.293049 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:15.306748 21855 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:15.299886 26933 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.280014 26927 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.321131 22007 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:07:15.321233 22008 raft_consensus.cc:2368] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15.
I20260709 13:07:15.322238 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2187) took 1021 ms. Trace:
I20260709 13:07:15.323137 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:15.323725 26651 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.324314 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:15.325598 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:15.322575 21853 rpcz_store.cc:276] 0709 13:07:14.300675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.056470 (+755795us) service_pool.cc:224] Handling call
0709 13:07:15.056757 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.056977 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:15.260526 (+203549us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.309669 (+ 49143us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.310280 (+   611us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.321926 (+ 11646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:15.321973 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:15.322013 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.322071 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.322187 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4665,"prepare.queue_time_us":234881,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":16714,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10126,"spinlock_wait_cycles":8988544,"thread_start_us":223056,"threads_started":2}
I20260709 13:07:15.325754 26920 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.328904 21852 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:15.290439 26784 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.331693 21933 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:07:15.332401 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:15.359257 22001 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:15.301232 26928 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.363241 21855 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:15.365063 21855 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:15.320709 26877 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.325798 26651 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.328325 26927 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.381979 26927 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.383409 26922 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.374465 26877 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.387072 26877 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.388613 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:15.389343 22003 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:15.389734 22009 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:15.391120 26619 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.391259 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:15.391533 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.392234 22593 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.392517 26651 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:15.392899 26914 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:15.393361 26914 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:15.394234 26914 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.395251 26914 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:15.397142 26933 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:15.398680 26886 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.471s	user 0.000s	sys 0.003s
I20260709 13:07:15.400453 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2498) took 2056 ms. Trace:
I20260709 13:07:15.400789 21860 rpcz_store.cc:276] 0709 13:07:13.344232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.589308 (+245076us) service_pool.cc:224] Handling call
0709 13:07:13.589702 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.595579 (+  5877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.595608 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.622729 (+ 27121us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.863564 (+240835us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:13.870667 (+  7103us) log.cc:844] Serialized 14 byte log entry
0709 13:07:13.875550 (+  4883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.875671 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.398101 (+1522430us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:15.400152 (+  2051us) raft_consensus.cc:1685] finished
0709 13:07:15.400254 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.400390 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4458,"prepare.queue_time_us":6917,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":8378,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":1218,"spinlock_wait_cycles":3766656,"thread_start_us":240482,"threads_started":1,"wal-append.queue_time_us":280111}
I20260709 13:07:15.404687 26916 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.406863 26811 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:15.407487 26811 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:15.428386 21758 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:15.429530 26930 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.641s	user 0.000s	sys 0.007s
I20260709 13:07:15.434969 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2182) took 1753 ms. Trace:
I20260709 13:07:15.435534 21851 rpcz_store.cc:276] 0709 13:07:13.681706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.903706 (+222000us) service_pool.cc:224] Handling call
0709 13:07:15.434755 (+1531049us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:15.434894 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984451968}
I20260709 13:07:15.438580 22592 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.439781 22592 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:15.445173 26877 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.445775 26877 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.446357 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:15.448128 26662 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:15.448841 26662 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:15.450467 26813 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.450767 21852 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:15.456102 26808 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:15.456550 26808 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:15.456666 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:15.457456 22000 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:15.459442 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:15.460186 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.463068 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.466415 22584 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:07:15.466569 26891 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.466861 22584 pending_rounds.cc:85] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 9
I20260709 13:07:15.467206 22584 pending_rounds.cc:107] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:07:15.467166 26891 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.467569 26808 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.468595 22590 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:15.468923 26808 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.467695 22584 raft_consensus.cc:2889] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:15.469749 26891 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.470172 22001 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
W20260709 13:07:15.471800 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.474162 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.475582 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:15.476212 21858 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:15.477700 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.478914 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.479027 22584 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:15.479331 21853 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:15.426597 22593 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:15.480173 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.481482 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.481557 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.482195 22584 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.482717 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:15.483372 22009 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:15.483690 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:15.485235 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 969) took 1510 ms. Trace:
I20260709 13:07:15.485618 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.488109 22588 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:15.392928 26651 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.495318 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.495716 22584 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:15.497474 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.497805 21853 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:15.496516 22588 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:15.450646 26877 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.501891 21932 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:15.503364 21860 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 16. Ops: []
I20260709 13:07:15.487390 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:15.506570 22590 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:15.508179 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:15.510109 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:15.508948 22584 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:15.510818 22009 rpcz_store.cc:276] 0709 13:07:13.974900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.975015 (+   115us) service_pool.cc:224] Handling call
0709 13:07:13.975282 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.389165 (+1413883us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b580065e1d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7072b0aa 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4
0709 13:07:15.484879 (+ 95714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.485181 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1203339008}
I20260709 13:07:15.506904 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:15.518906 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:15.519708 22588 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.520393 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:15.491029 26740 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:07:15.525908 21853 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:15.538905 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.539599 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.540228 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.540846 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.544736 21927 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:15.486912 26732 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:15.551956 21860 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:15.578830 21860 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:07:15.524464 26931 raft_consensus.cc:476] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 13:07:15.490877 26813 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:15.557219 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.559854 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:15.567785 26788 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:15.570000 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.569885 26914 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.571808 26798 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:15.573467 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:15.573810 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1055) took 1353 ms. Trace:
I20260709 13:07:15.592954 22591 rpcz_store.cc:276] 0709 13:07:14.220525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.221188 (+   663us) service_pool.cc:224] Handling call
0709 13:07:14.260492 (+ 39304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.348448 (+ 87956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.573467 (+1225019us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b540060d6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:15.573754 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154016000}
I20260709 13:07:15.574432 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:15.526093 26916 raft_consensus.cc:2764] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:15.579973 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:15.497447 26651 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.575181 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:15.596900 22588 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:15.541689 26909 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:15.605325 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:15.615648 26664 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:15.484618 26811 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:15.577284 26873 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.589232 26877 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.594362 26798 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:15.608897 26758 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:15.589918 26891 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:07:15.627158 22592 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:15.622357 26664 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:15.591377 26764 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 13:07:15.656826 26807 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.011s	sys 0.014s
I20260709 13:07:15.598920 26914 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.672369 26914 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.620581 26697 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:07:15.674976 26914 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:15.675729 26914 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.680418 26914 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:15.689095 26807 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.012s	sys 0.014s
I20260709 13:07:15.636600 26909 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:15.628319 26877 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.721843 26909 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.723003 26877 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:15.705683 26697 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:15.723630 26697 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:15.724439 26697 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.710278 26966 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:15.738098 26966 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.624145 26873 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.660347 26758 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:15.699239 26840 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.718297 26811 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:15.718997 26798 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.657327 26861 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:07:15.702224 26664 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:15.732053 26697 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:15.740710 26966 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:15.747923 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 137855us
I20260709 13:07:15.750905 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.751662 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.755218 21932 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.756582 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:15.759449 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.780073 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.781713 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:15.762725 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 235770us
I20260709 13:07:15.742241 26877 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.783507 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:15.744289 26130 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:15.796813 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.797696 21926 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:15.801725 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.751403 26811 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.751587 26861 consensus_queue.cc:1243] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:15.803045 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:15.785066 26966 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:15.789896 26798 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.744136 26840 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.843487 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.850492 21931 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:15.804270 26808 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:15.846604 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.844931 21932 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:15.869552 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.854278 21926 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:15.867357 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.871344 21931 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:15.872915 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.873571 21932 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.787032 26927 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:07:15.894870 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.896144 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.896659 21926 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:15.812994 26130 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.802009 26877 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.835606 26811 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.920312 21931 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:15.921409 21926 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:07:15.940899 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 137979us
I20260709 13:07:15.941685 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.942710 21926 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:15.890143 26971 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:15.947247 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:15.945917 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:15.953985 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.849305 26966 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.878269 26808 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:15.973481 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:15.947733 26971 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:15.983340 26976 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:07:15.983495 26971 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:15.986040 26982 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:07:15.873642 26664 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:15.954227 26967 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:15.991684 21935 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:15.992993 21935 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:15.993700 26808 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.993779 26811 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:15.994215 21935 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:07:15.994597 21935 pending_rounds.cc:85] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 10
I20260709 13:07:15.994828 21935 pending_rounds.cc:107] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:15.995105 21935 raft_consensus.cc:2889] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:15.997900 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:16.001022 21935 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:16.021517 21900 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:16.021325 26967 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.996240 26898 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.029680 26967 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:16.034983 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93092.3us
I20260709 13:07:16.040359 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:15.877516 26758 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.042213 21932 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
W20260709 13:07:16.045264 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.045217 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:16.047930 22009 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:16.048676 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.083005 22001 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:16.061719 26826 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:16.033727 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:16.107169 21929 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:16.113322 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 73055us
I20260709 13:07:16.041916 26861 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:16.125731 26973 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:07:16.125418 26991 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:16.126777 26690 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:16.127406 26991 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.128118 26991 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.132071 26991 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.137843 26978 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:16.138474 26978 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:16.139503 26978 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.140910 26978 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.141654 26840 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.120328 26808 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:07:16.143178 26808 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:16.143515 26808 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:16.145084 27000 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:07:16.143527 26915 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:16.149590 26130 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:16.151073 26651 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260709 13:07:16.151517 26651 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.119437 26986 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:16.086395 26979 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:16.103471 26620 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:16.112526 26987 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:16.155102 26987 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:16.156026 26987 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.159504 26967 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.160115 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2650) took 1182 ms. Trace:
I20260709 13:07:16.160557 26877 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:16.161376 26989 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:16.161768 26989 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:16.162632 26989 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.163988 26989 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.110212 26873 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:16.168090 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.144341 26808 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:16.167404 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 381801us
I20260709 13:07:16.160485 21933 rpcz_store.cc:276] 0709 13:07:14.977706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.984712 (+  7006us) service_pool.cc:224] Handling call
0709 13:07:14.988070 (+  3358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.360267 (+372197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.159852 (+799585us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54002e7cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:16.160055 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789419904}
I20260709 13:07:16.172721 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:16.042668 26898 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.156806 26830 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:16.211025 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.168655 26861 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.182677 26620 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:16.185075 26235 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:16.216110 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.181923 22556 leader_election.cc:336] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.663s (SENT)
I20260709 13:07:16.223354 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:16.226322 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:16.227207 22007 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:16.229485 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.232201 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:16.233520 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.238274 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:16.239451 22004 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:16.241607 22000 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:16.245024 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.249521 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:16.251575 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.262926 22001 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:16.175125 26807 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:16.127441 26690 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.177320 26740 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:16.262051 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.258178 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.253217 26931 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.254572 22000 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:16.261147 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.268023 22002 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:16.268468 22005 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:16.221400 26235 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.272367 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:16.261544 26697 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.003s
I20260709 13:07:16.255005 26877 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.255120 26130 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:16.261302 26651 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.252858 26110 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:16.277853 26235 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:16.278020 26740 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:16.282483 26905 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.924s	user 0.004s	sys 0.007s
W20260709 13:07:16.284664 22592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 935) took 1665 ms (client timeout 2151 ms). Trace:
W20260709 13:07:16.306007 22592 rpcz_store.cc:269] 0709 13:07:14.619264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.630391 (+ 11127us) service_pool.cc:224] Handling call
0709 13:07:15.438150 (+807759us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b54006a0ca0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7074d06f 00007f3b707c5f60 00007f3b707c77de 00007f3b707d327c 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2
0709 13:07:16.284408 (+846258us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54006a04d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:16.284579 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252324480}
I20260709 13:07:16.285292 26684 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:16.277513 26931 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.308046 26684 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:16.299417 26110 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.297489 26877 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.297801 26620 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:16.401626 21901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.014s	sys 0.002s
W20260709 13:07:16.402662 21928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.010s	sys 0.018s
W20260709 13:07:16.402424 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 283437us
W20260709 13:07:16.401626 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.030s	sys 0.066s
I20260709 13:07:16.429384 22007 raft_consensus.cc:2410] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:07:16.434418 22590 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:16.434178 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.436398 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:16.437436 22593 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:16.439366 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.440471 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.440726 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.446609 22006 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:16.446245 26684 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.450868 26690 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.451874 26130 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:16.313673 26898 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.334640 26931 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.490486 26740 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.313663 26966 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.493134 22005 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:07:16.494014 21928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.011s	sys 0.018s
W20260709 13:07:16.494459 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.031s	sys 0.066s
W20260709 13:07:16.495302 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 713066us
I20260709 13:07:16.496239 22002 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:16.485487 21901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.015s	sys 0.003s
I20260709 13:07:16.499584 26877 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:16.500273 26684 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.340538 26971 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.501359 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:16.502486 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45645.2us
I20260709 13:07:16.503973 26971 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.505605 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:16.506178 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.506742 21852 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.511346 26662 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:16.512023 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.513135 21855 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:16.514050 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:16.328488 26861 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.514634 26826 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.515121 21858 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.515901 26826 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.517922 26110 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:16.518029 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:16.518558 26902 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.482824 26983 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:16.521703 26130 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.522650 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.514057 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:16.523257 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 856) took 1018 ms. Trace:
I20260709 13:07:16.523368 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:16.524060 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:16.524901 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.526439 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:16.510354 26808 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:16.541602 26662 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:16.542508 26662 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.542886 21857 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.514688 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:16.482797 22000 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:16.546132 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49469.5us
I20260709 13:07:16.548372 26995 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:16.522719 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:16.534022 26690 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:16.547405 27001 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:16.573231 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.577929 27001 consensus_queue.cc:1243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:16.578948 27001 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:16.579491 27001 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:16.583841 21857 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
W20260709 13:07:16.586390 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52257.6us
I20260709 13:07:16.557539 26690 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:16.606699 22005 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.608613 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:16.611136 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 88942.2us
I20260709 13:07:16.573318 26978 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:16.604279 26690 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.614614 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.618592 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1061) took 1026 ms. Trace:
W20260709 13:07:16.621110 21823 connection.cc:256] RPC call timeout handler was delayed by -1.5099448890000531s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
I20260709 13:07:16.621389 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.621914 22009 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:16.621202 26826 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:16.621980 21823 connection.cc:256] RPC call timeout handler was delayed by -1.2282448890000524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:07:16.626060 21823 connection.cc:256] RPC call timeout handler was delayed by -1.2021448890000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1947s for a response
W20260709 13:07:16.633060 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.619202 22589 rpcz_store.cc:276] 0709 13:07:15.460759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.460870 (+   111us) service_pool.cc:224] Handling call
0709 13:07:15.461190 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.465179 (+  3989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.465207 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.465660 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.287723 (+822063us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b180048dd20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:16.469351 (+181628us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:16.469500 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.469665 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.472830 (+  3165us) raft_consensus.cc:1685] finished
0709 13:07:16.472848 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.487430 (+ 14582us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4670,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":5202,"spinlock_wait_cycles":9653120,"thread_start_us":180938,"threads_started":1,"wal-append.queue_time_us":1003506}
W20260709 13:07:16.635836 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 77078.2us
I20260709 13:07:16.628506 26110 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:16.646510 21852 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:16.648314 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.649746 27011 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:16.657234 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.651113 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 133491us
I20260709 13:07:16.609895 26902 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.657270 22004 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:16.534591 21928 rpcz_store.cc:276] 0709 13:07:15.493282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.493395 (+   113us) service_pool.cc:224] Handling call
0709 13:07:15.537894 (+ 44499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.538132 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:15.795742 (+257610us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.509747 (+714005us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.510351 (+   604us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.510515 (+   164us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.510569 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:16.510676 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.510799 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.511173 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27088,"prepare.queue_time_us":159043,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":132,"raft.queue_time_us":28820,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":255,"thread_start_us":932249,"threads_started":2,"wal-append.queue_time_us":520060}
I20260709 13:07:16.665274 26902 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.668182 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.601707 26995 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.674801 21852 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:16.678225 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.679396 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.680431 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.681522 26902 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.682207 26902 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:16.674583 22556 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.789s (SENT)
I20260709 13:07:16.683681 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:16.685415 22556 leader_election.cc:336] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.792s (SENT)
I20260709 13:07:16.676141 27011 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.685423 26840 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:16.685916 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:16.688421 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.811s (SENT) [suppressed 26 similar messages]
I20260709 13:07:16.688944 26840 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:16.688630 27011 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:16.692416 22556 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.930s (SENT)
I20260709 13:07:16.693030 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:16.690595 21852 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:16.693859 26840 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:16.695343 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 2.117s (SENT)
I20260709 13:07:16.674175 21853 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:16.700961 22000 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:16.670289 21858 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:16.665870 26110 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:16.628629 26983 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.706899 21852 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:16.610471 27001 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.711644 26110 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.717236 26826 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:16.674021 26997 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.351s	user 0.001s	sys 0.002s
I20260709 13:07:16.721959 26130 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.674302 26909 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.738973 26235 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.741510 22588 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:07:16.744365 26662 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.673985 26978 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.747189 26662 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.747579 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.753325 26898 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:16.754233 26898 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.757910 27000 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:07:16.758873 26798 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.759932 26995 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.765547 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.745362 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2180) took 3081 ms. Trace:
I20260709 13:07:16.769218 26931 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:16.769696 21855 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:07:16.774920 21855 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 7
I20260709 13:07:16.756734 26898 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.780834 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.781570 26909 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.780831 26798 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.785157 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.785540 26798 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.744230 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.788203 21857 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:16.788410 21931 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:16.762809 27000 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:16.770035 26998 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.438s	user 0.002s	sys 0.000s
I20260709 13:07:16.775826 22005 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
W20260709 13:07:16.792868 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.775241 26999 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.444s	user 0.001s	sys 0.001s
I20260709 13:07:16.775251 21855 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:07:16.793586 21855 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:16.794039 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:16.795774 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.796300 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.796329 26620 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:16.778410 26235 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:16.778591 27003 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.448s	user 0.000s	sys 0.000s
I20260709 13:07:16.774194 21854 rpcz_store.cc:276] 0709 13:07:13.663384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.907130 (+1243746us) service_pool.cc:224] Handling call
0709 13:07:14.907575 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.991375 (+ 83800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.744797 (+1753422us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b540025ab54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:16.745166 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178165504}
I20260709 13:07:16.789134 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.747644 26978 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:16.812655 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.815675 26898 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.816220 22004 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:16.816557 26898 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:16.819001 26985 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.689s	user 0.003s	sys 0.000s
I20260709 13:07:16.819510 26898 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.821408 21860 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:07:16.831136 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.800760 26620 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:16.846589 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2167) took 1134 ms. Trace:
I20260709 13:07:16.847168 21930 rpcz_store.cc:276] 0709 13:07:15.711967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.715735 (+  3768us) service_pool.cc:224] Handling call
0709 13:07:15.716253 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.801104 (+ 84851us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:15.801133 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.804510 (+  3377us) log.cc:844] Serialized 26 byte log entry
0709 13:07:15.952626 (+148116us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:15.952676 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.952755 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.818039 (+865284us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:16.822269 (+  4230us) raft_consensus.cc:1685] finished
0709 13:07:16.822287 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.846386 (+ 24099us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8784,"raft.queue_time_us":160342,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":24425,"spinlock_wait_cycles":65732352,"thread_start_us":231726,"threads_started":2,"wal-append.queue_time_us":324589}
I20260709 13:07:16.849134 26983 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.851577 26697 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.852603 26909 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:16.853302 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.853863 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.855484 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.856947 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.857457 21927 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:16.857921 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:16.858784 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.858963 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:16.859807 21934 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:16.862293 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.863547 26971 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:16.864187 26971 consensus_queue.cc:1243] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:16.813544 27011 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.865603 21931 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:16.866080 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.866513 27011 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.860632 26840 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:16.861466 26130 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:16.859954 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:16.874480 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.876284 21929 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:16.868947 27011 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.887815 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.891436 21927 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.891549 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.891549 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.852640 26697 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.869061 26684 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:16.882575 26130 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:16.919255 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.923770 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:16.917315 26620 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.888764 26840 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:16.924947 21932 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.882628 26902 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.931492 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.933965 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:16.935027 21933 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
W20260709 13:07:16.935969 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56664.4us
I20260709 13:07:16.937590 26620 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:16.937388 21758 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 14, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 14 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:16.928639 26130 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:16.938726 26591 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:07:16.939930 26591 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:16.940356 26591 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:16.942695 26697 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:16.943992 27008 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
I20260709 13:07:16.943533 26684 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:16.946265 26684 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:16.947924 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.948803 22004 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:16.938223 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:16.895239 26688 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:16.963375 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.962425 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.968703 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:16.969118 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:16.971987 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.968750 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2204) took 2132 ms. Trace:
I20260709 13:07:16.977065 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:16.982175 21899 leader_election.cc:400] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15.
I20260709 13:07:16.983996 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2173) took 1193 ms. Trace:
I20260709 13:07:16.986389 21935 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:16.968765 26837 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.960490 26898 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:17.003543 22004 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:17.005862 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:16.974061 21856 rpcz_store.cc:276] 0709 13:07:14.836359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.095621 (+259262us) service_pool.cc:224] Handling call
0709 13:07:15.095917 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.096123 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:15.257309 (+161186us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.257676 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.674354 (+1416678us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b18003d4320. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b70385d70 00007f3b70379293 00007f3b7036ecea 00007f3b7036f23a 00007f3b707d9a2a 00007f3b707e90bf 00007f3b707e9038
0709 13:07:16.918789 (+244435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.947643 (+ 28854us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:16.956520 (+  8877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.959928 (+  3408us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.968502 (+  8574us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45292,"prepare.queue_time_us":241880,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":20382,"raft.run_cpu_time_us":485,"raft.run_wall_time_us":7605,"spinlock_wait_cycles":9249024,"thread_start_us":355193,"threads_started":2}
I20260709 13:07:17.011777 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.012190 21851 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:07:17.013330 21860 raft_consensus.cc:2368] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:07:17.009392 21933 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:16.973521 26688 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.048477 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:16.984539 21926 rpcz_store.cc:276] 0709 13:07:15.761750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.957148 (+195398us) service_pool.cc:224] Handling call
0709 13:07:15.957491 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.016246 (+ 58755us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:16.024041 (+  7795us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.402524 (+378483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.402714 (+   190us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:16.402903 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.771072 (+368169us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:16.951962 (+180890us) spinlock_profiling.cc:243] Waited 58.4 ms on lock 0x7b54003070d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:16.952052 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.955261 (+  3209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7355,"prepare.queue_time_us":13255,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":65522,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":221,"spinlock_wait_cycles":147818112,"thread_start_us":377550,"threads_started":1,"wal-append.queue_time_us":300749}
I20260709 13:07:16.990811 26931 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:07:17.062537 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:17.026750 26909 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:17.002233 26690 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:17.072206 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 117238us
I20260709 13:07:17.087555 22005 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:17.022786 26898 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.101991 22590 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:17.106665 21927 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:17.103183 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.110006 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:17.119515 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.120815 26987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.005s	sys 0.029s
W20260709 13:07:17.121341 26987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.006s	sys 0.029s
I20260709 13:07:17.123232 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.125200 26989 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:17.125780 26989 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.014091 26591 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.124828 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.054420 26837 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.089445 26840 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.098577 26909 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:17.124639 21930 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.136422 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.138849 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:17.137081 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86764.5us
I20260709 13:07:17.139896 22591 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:17.143103 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:17.143430 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.145387 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.202890 21857 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
W20260709 13:07:17.139892 21756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 288) took 38702 ms (client timeout 15000 ms). Trace:
I20260709 13:07:17.146225 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:17.199262 22002 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:17.201551 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.147068 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:17.157048 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.206128 21859 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:17.146054 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.159852 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.146432 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:17.208472 22000 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:17.204846 21926 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:17.164880 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.167558 21932 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:17.176745 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.206117 21935 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:17.155391 22589 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:17.155802 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:17.160931 22008 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:07:17.215937 21929 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 8.
W20260709 13:07:17.216502 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.163956 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.145710 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.161681 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.157802 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.180249 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:17.161427 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:17.224987 21931 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:07:17.168897 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:17.225171 21933 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:17.225708 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.152587 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.150334 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:17.234922 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:17.164639 26917 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.229s	user 0.027s	sys 0.023s
I20260709 13:07:17.238636 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.132232 27001 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.139066 21758 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 11, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 11 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:17.167533 26690 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:17.240875 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 248997us
I20260709 13:07:17.242383 21860 raft_consensus.cc:2368] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18.
I20260709 13:07:17.242233 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.244478 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.242018 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.247365 22008 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:17.132025 26983 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:07:17.244961 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.128973 26989 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.248979 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:17.245417 22001 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:17.165688 26697 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.140643 26837 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:17.238858 26684 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:17.173208 26688 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:17.239147 26917 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.450s	user 0.028s	sys 0.023s
W20260709 13:07:17.251050 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70957.5us
I20260709 13:07:17.262908 22000 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:17.252362 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:17.250762 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.264400 21854 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:17.251384 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:17.201213 26902 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:07:17.256656 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.251269 26697 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.257974 21856 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:17.251638 21758 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 10, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 10 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:07:17.259357 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:17.260496 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 108028us
I20260709 13:07:17.261277 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.267390 22005 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.255666 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.267606 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:17.270639 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:17.273240 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 123093us
I20260709 13:07:17.274226 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:17.275066 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.276261 21857 pending_rounds.cc:85] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 6
I20260709 13:07:17.276561 21857 pending_rounds.cc:107] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:07:17.276854 21857 raft_consensus.cc:2889] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:17.274264 22002 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:17.275836 26764 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:07:17.280462 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.281347 22003 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:17.281685 22586 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:17.282156 26978 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.282563 26978 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.282943 26978 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.285216 26831 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.286279 22006 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:17.286798 22005 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:17.288399 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.289538 26983 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.290063 26983 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.289431 21758 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.290457 26983 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.291950 21758 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.293028 26988 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:17.294077 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:17.295145 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:17.298771 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 122254us
I20260709 13:07:17.299552 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.242071 27011 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:07:17.294667 21860 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:17.244421 26899 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:07:17.160543 26909 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.312960 21853 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
W20260709 13:07:17.319774 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 77841us
I20260709 13:07:17.322697 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:17.305893 26831 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:17.332247 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.333199 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:17.333189 21758 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 2 to 13. New cstate: current_term: 13 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.325112 26840 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:17.343160 27018 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.328810 26684 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.339941 26917 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:17.354846 26902 raft_consensus.cc:2764] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:07:17.357544 26854 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.358050 26854 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:17.358413 26854 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.351184 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.370486 26917 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:17.371213 26697 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.371620 26917 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.372985 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.373176 27034 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:17.374339 21930 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:17.373492 26591 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.376068 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:17.377753 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58781.2us
I20260709 13:07:17.374152 21758 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.381805 21900 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.352712 26831 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:17.384388 26995 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:17.384881 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:17.344388 26840 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:17.385227 27034 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:17.388489 26684 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.390132 27034 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:17.393963 27034 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.392416 27026 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:17.393018 26902 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:17.393271 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:17.415446 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 62738.6us
W20260709 13:07:17.392318 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62106us
W20260709 13:07:17.399158 27015 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
W20260709 13:07:17.401906 27021 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.182s	user 0.003s	sys 0.000s
I20260709 13:07:17.413142 26831 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.421201 27020 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.422472 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.426172 26891 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:17.426704 27011 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:17.427206 22007 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:17.428886 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:07:17.431761 27030 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:07:17.433043 27000 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.433471 27000 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:17.433869 27000 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.435082 26891 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:07:17.444449 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 67707.3us
I20260709 13:07:17.450680 26798 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:17.451261 26798 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:17.452436 26798 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.453933 26837 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:17.454370 26837 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:17.204541 21756 rpcz_store.cc:269] 0709 13:06:38.436931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.437220 (+   289us) service_pool.cc:224] Handling call
0709 13:07:17.139616 (+38702396us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:03.470502 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:03.515913 (+ 45411us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:03.519545 (+  3632us) write_op.cc:435] Acquired schema lock
0709 13:07:03.525903 (+  6358us) tablet.cc:662] Decoding operations
0709 13:07:03.614103 (+ 88200us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:03.614159 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:03.614185 (+    26us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:07:03.615359 (+  1174us) tablet.cc:701] Row locks acquired
0709 13:07:03.615378 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:07:03.615522 (+   144us) write_op.cc:270] Start()
0709 13:07:03.615638 (+   116us) write_op.cc:276] Timestamp: P: 1783602423615479 usec, L: 0
0709 13:07:03.615670 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:07:03.616354 (+   684us) log.cc:844] Serialized 6720 byte log entry
0709 13:07:04.097545 (+481191us) op_driver.cc:464] REPLICATION: finished
0709 13:07:04.533195 (+435650us) write_op.cc:301] APPLY: starting
0709 13:07:04.548043 (+ 14848us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:04.902271 (+354228us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:04.911352 (+  9081us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:05.721193 (+809841us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:05.721650 (+   457us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:07:05.721878 (+   228us) write_op.cc:312] APPLY: finished
0709 13:07:05.722684 (+   806us) log.cc:844] Serialized 156 byte log entry
0709 13:07:05.805813 (+ 83129us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:06.065499 (+259686us) write_op.cc:454] Released schema lock
0709 13:07:06.065586 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:06.072491 (+  6905us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:06.072527 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2369120,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":409854,"apply.run_cpu_time_us":49097,"apply.run_wall_time_us":1552424,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75289,"num_ops":17,"prepare.queue_time_us":432078,"prepare.run_cpu_time_us":23603,"prepare.run_wall_time_us":371143,"raft.queue_time_us":263238,"raft.run_cpu_time_us":21759,"raft.run_wall_time_us":285231,"replication_time_us":481693,"thread_start_us":971408,"threads_started":5,"wal-append.queue_time_us":363812,"wal-append.run_cpu_time_us":53828,"wal-append.run_wall_time_us":2331817}]]}
I20260709 13:07:17.424659 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.485719 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:17.454169 26798 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.432909 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:17.488770 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:17.394784 26898 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.494130 27000 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:17.497802 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73230.1us
I20260709 13:07:17.499723 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.490264 26690 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:17.509565 26690 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:17.510069 26690 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.509856 26840 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.511402 26840 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:17.452389 26983 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.487946 21758 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 11, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 11 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.488337 27020 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:17.456866 26978 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:17.530123 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 184054us
I20260709 13:07:17.537860 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.539335 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:17.543431 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 85997.3us
I20260709 13:07:17.550225 26684 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:17.553068 26811 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.553170 26684 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:17.553694 26811 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.554023 26684 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:17.554100 26811 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.562230 26988 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:17.563200 26988 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:17.574443 26684 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.577724 26983 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.578473 26983 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:17.578832 26983 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:17.579536 26684 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.530073 26989 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.529707 26697 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.542562 26995 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.541118 26898 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.592710 21758 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 11, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 11 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.605109 22009 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:07:17.606212 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 173452us
I20260709 13:07:17.605998 26854 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.608330 26854 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:17.609777 26978 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.612334 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.617937 26130 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:07:17.619721 27041 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
W20260709 13:07:17.624276 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 81135us
I20260709 13:07:17.628026 21855 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:17.630689 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:17.631660 26891 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:17.633301 26891 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:17.633167 27020 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:17.633930 26891 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:17.634387 26989 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.635041 26891 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.635438 27020 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:17.636986 26902 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:17.637429 26902 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:17.638237 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.640031 26697 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.640595 26697 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.640841 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:17.642839 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 96719us
I20260709 13:07:17.643481 26697 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:17.643767 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:17.648866 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.652987 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:17.652884 26798 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.655907 26758 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:17.656677 26931 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.656795 26758 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:17.654582 26978 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.657541 26758 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.656018 26798 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.667062 26798 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.579743 26983 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.638301 26902 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.674443 26764 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:17.685894 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73636us
W20260709 13:07:17.688230 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50073.9us
W20260709 13:07:17.701901 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.710721 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:17.711295 26902 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:17.712914 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.714016 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:17.719241 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:17.721367 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 175318us
I20260709 13:07:17.726311 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:17.720504 26591 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:17.728452 26591 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:17.728686 26854 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:17.729378 26854 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:17.730376 26854 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.730561 26591 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.720687 26995 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:17.735250 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.735175 26995 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.735672 26995 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.738507 26837 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.739717 26837 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.739934 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:17.743090 27000 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:17.732549 27026 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:07:17.764511 22009 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:17.748729 26764 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:17.786166 22589 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:17.788821 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:17.791913 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:17.791973 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81754.9us
I20260709 13:07:17.794162 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:17.794756 22000 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.763301 27045 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.724705 26690 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:17.792850 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:17.798226 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 72055.8us
I20260709 13:07:17.806493 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:17.785377 26917 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:17.785782 26764 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.925210 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:17.786573 26978 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:17.966077 22586 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:17.805100 22003 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
W20260709 13:07:17.923864 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117443us
W20260709 13:07:17.885062 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.924275 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.929165 26688 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:17.939823 27044 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.148s	user 0.004s	sys 0.006s
I20260709 13:07:17.939308 26130 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.965595 26764 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:18.032665 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 119841us
I20260709 13:07:17.973090 26688 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:18.045322 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:18.007938 26899 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:18.049626 22000 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
W20260709 13:07:18.037241 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 326602us
I20260709 13:07:17.943666 27001 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
W20260709 13:07:18.053413 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 297195us
I20260709 13:07:18.017506 26831 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:18.043697 26840 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.532s	user 0.014s	sys 0.002s
I20260709 13:07:17.940683 26917 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:17.979341 26697 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.922436 26995 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:17.921219 27045 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.044415 26688 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:18.089543 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44293.8us
I20260709 13:07:17.925792 26798 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:18.052429 27034 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.000s
I20260709 13:07:18.099376 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:18.088562 26697 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.103111 26697 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.103652 26697 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.105566 22589 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:18.053746 26898 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.115298 26826 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.115782 27026 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.117888 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:18.118615 26995 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:18.119782 27044 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:07:18.120338 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.121906 27053 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:07:18.123811 27045 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:18.044498 26978 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.125876 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:18.104125 26697 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.097271 26917 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:18.044214 26130 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.060014 26811 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:18.144869 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.015s	sys 0.037s
I20260709 13:07:18.138298 27026 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:18.184177 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.016s	sys 0.037s
I20260709 13:07:18.150213 26620 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:18.153396 26978 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:18.155575 27051 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.235s	user 0.003s	sys 0.000s
I20260709 13:07:18.157681 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:18.158483 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.191602 22000 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:18.158629 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.159482 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:18.160535 22009 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:07:18.161355 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.162021 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.162582 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:18.162684 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.193845 22004 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.194274 27001 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.194875 27001 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.132702 27018 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:07:18.197759 27001 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.163466 26989 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.142911 26909 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:07:18.199559 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:18.164673 27049 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:07:18.058641 26899 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.204087 26899 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.168433 26915 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:07:18.114537 27034 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.205376 26915 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:18.206758 22588 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:18.207291 27034 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:18.181733 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66473.4us
I20260709 13:07:18.092931 26831 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.215842 22006 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:18.190454 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 72691.4us
W20260709 13:07:18.184860 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 638095us
I20260709 13:07:18.149358 27060 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.207886 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:18.208773 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 765) took 1058 ms. Trace:
I20260709 13:07:18.170608 26798 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.167609 26697 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:18.210713 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53125us
I20260709 13:07:18.149694 26826 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.188510 26837 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:18.225237 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:18.220336 21857 rpcz_store.cc:276] 0709 13:07:17.149851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.249800 (+ 99949us) service_pool.cc:224] Handling call
0709 13:07:17.270604 (+ 20804us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:07:17.339185 (+ 68581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.339216 (+    31us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:07:17.354285 (+ 15069us) log.cc:844] Serialized 104 byte log entry
0709 13:07:17.542970 (+188685us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:17.566671 (+ 23701us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.625385 (+ 58714us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.625721 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.625945 (+   224us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.626068 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.626134 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.208513 (+582379us) raft_consensus.cc:1685] finished
0709 13:07:18.208541 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.208702 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61549,"prepare.queue_time_us":28308,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"raft.queue_time_us":65053,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":24229,"thread_start_us":141676,"threads_started":1,"wal-append.queue_time_us":277945}
I20260709 13:07:18.219765 22590 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:18.254604 22586 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:18.224663 26909 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:07:18.228952 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:18.229988 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:18.223305 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:18.246763 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:18.251200 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:18.225576 26758 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.003s
I20260709 13:07:18.208775 27018 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:18.284628 27018 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:18.289923 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.290735 21930 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:18.290846 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.292310 26995 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:07:18.292477 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 63604.4us
I20260709 13:07:18.294059 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:18.294466 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 71237.5us
I20260709 13:07:18.212985 26915 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:07:18.261992 22008 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:18.262444 26758 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.263417 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.264122 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.244957 27058 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:18.265190 21858 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:18.312384 22004 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:18.265321 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.313103 22000 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:18.265923 26831 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:18.266426 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:18.270354 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.273784 22002 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:07:18.275101 26130 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.277304 26826 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:18.328111 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.220655 27020 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:07:18.281065 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.283461 26764 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.293249 26995 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:18.297801 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:18.335069 21854 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:18.225795 27065 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:18.226648 26978 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:18.339452 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.293347 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:18.305032 21972 connection.cc:256] RPC call timeout handler was delayed by -1.5621902039998758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
I20260709 13:07:18.265563 21855 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:18.300338 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:18.317257 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.315289 21859 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:18.318560 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.323714 21927 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:18.330171 21926 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:18.331748 21931 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:07:18.331903 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 105553us
I20260709 13:07:18.338925 21928 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:18.344748 21860 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:18.339066 21852 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:18.308786 27063 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:18.342917 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:18.339089 22003 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
W20260709 13:07:18.346907 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 119783us
I20260709 13:07:18.306638 27026 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:18.357316 21932 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:18.367209 22006 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:18.251679 27060 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.387244 21935 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:18.389642 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:18.392007 21859 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:18.347275 26130 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:18.339689 26978 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.339493 26831 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.396329 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.357105 27063 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.358347 26798 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:18.286058 27018 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.401978 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:18.373612 26915 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:18.410060 21855 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:18.407398 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:18.376926 26684 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:18.393826 26697 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.396673 27020 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:18.400211 26130 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.433597 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.403270 26978 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:18.439106 21854 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 6
I20260709 13:07:18.439710 21854 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:07:18.392148 27026 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:18.436735 21932 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:18.412941 27063 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.410490 26798 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.440055 21854 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:18.395421 26620 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:07:18.429127 26591 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:18.450392 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60858.6us
I20260709 13:07:18.427057 26684 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.439059 26130 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:18.437342 26808 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:18.409974 26831 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:18.463469 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.759s (SENT)
W20260709 13:07:18.478368 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.448321 21852 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:18.473439 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.391913 27065 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.440637 27069 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.444067 27018 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:18.470983 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 242340us
I20260709 13:07:18.484673 21928 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:18.489528 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:18.448295 26690 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:18.469969 26684 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.493696 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:18.515583 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:18.515641 21932 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.449680 26798 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:18.473661 26591 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:18.516253 22007 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:18.474215 26811 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:18.432727 26915 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:18.522706 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 128239us
I20260709 13:07:18.527275 21926 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
W20260709 13:07:18.529165 21972 connection.cc:256] RPC call timeout handler was delayed by -1.483196616999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
I20260709 13:07:18.479859 26902 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:18.529830 21972 connection.cc:256] RPC call timeout handler was delayed by -1.4696966169999541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:07:18.532222 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:18.536362 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46922.6us
I20260709 13:07:18.456830 27026 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.457079 26697 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:18.517491 27066 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:07:18.541314 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 77896.8us
I20260709 13:07:18.493155 26891 raft_consensus.cc:2764] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:18.544186 26915 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:18.548349 27065 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:18.550066 26983 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:18.550117 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:18.588541 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50618.6us
I20260709 13:07:18.551573 27060 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.518687 27069 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.533154 21928 raft_consensus.cc:2376] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17. Re-sending same reply.
I20260709 13:07:18.556623 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:18.578732 22556 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.811s (SENT)
I20260709 13:07:18.583925 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:18.496052 26808 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:18.596266 26808 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:18.598662 27068 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:07:18.601670 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.601245 26591 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.602325 21931 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:18.605827 21932 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:18.595093 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:18.602900 26591 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:18.557631 26837 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:18.665464 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:18.671258 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:18.615666 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.557320 26898 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:18.672076 21855 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:18.557772 27026 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:18.675354 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:18.676133 21927 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:18.683508 27055 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.637s	user 0.003s	sys 0.000s
I20260709 13:07:18.688161 27074 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:07:18.689316 27074 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:18.689679 27074 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:18.695971 27064 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:18.696971 27064 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.698621 27069 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:18.701264 27001 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:18.702036 27070 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:18.701714 27064 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:18.702713 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2401966170000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
I20260709 13:07:18.702814 27070 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:18.703480 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2131966169999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:07:18.703814 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2032966169999781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 13:07:18.704090 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1348915569999463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:07:18.704384 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1258966169999667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 13:07:18.704699 21972 connection.cc:256] RPC call timeout handler was delayed by -1.0628966169999785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 13:07:18.704905 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 142157us
I20260709 13:07:18.706290 21855 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:18.707799 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:18.710618 27073 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:18.711215 27073 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.713927 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.714716 21858 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:18.723397 26899 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:18.725271 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:18.726675 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.730130 21852 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:18.556550 26688 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:18.735249 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 151419us
W20260709 13:07:18.738787 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 160069us
I20260709 13:07:18.745968 22007 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:18.749094 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.587097 26758 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.613622 26690 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.755775 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.764698 22000 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:18.759315 26684 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.762239 27074 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.593614 27071 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:18.773486 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.765285 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:18.777354 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 187951us
I20260709 13:07:18.786026 22005 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:18.734846 26764 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:18.616115 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:18.798151 21851 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:18.799286 21928 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:18.795053 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:18.781692 27069 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:18.823179 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.824056 21860 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:18.790599 27065 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.828250 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:18.839395 21926 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:18.796518 27026 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.844297 21927 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:18.850070 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.851049 21859 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:18.776417 26995 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:18.781901 27078 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:18.826251 26898 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:18.865458 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.875994 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.862788 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.891224 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:18.892176 21859 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
I20260709 13:07:18.793848 26690 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:18.895438 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 170280us
I20260709 13:07:18.898092 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:18.898716 22000 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:18.847643 27074 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:18.900803 22004 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:18.843405 27073 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:18.902876 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:18.904248 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:18.906036 22009 raft_consensus.cc:2368] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13.
W20260709 13:07:18.909384 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.001s	sys 0.010s
W20260709 13:07:18.909824 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.002s	sys 0.010s
W20260709 13:07:18.911247 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 173815us
I20260709 13:07:18.911964 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:18.876991 27064 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:18.916201 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:18.860972 27026 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:18.853835 27065 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:18.913872 27058 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:18.931866 27062 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.934464 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:18.937486 21927 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:18.921367 26988 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:18.937484 21859 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:18.933614 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 189811us
I20260709 13:07:18.941507 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:18.952319 22556 leader_election.cc:336] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.927s (SENT)
I20260709 13:07:18.930454 26758 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.002230 21926 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.003614 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:18.932417 27062 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.022380 22000 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:18.922158 27064 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:19.024801 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81801.8us
I20260709 13:07:18.949048 26831 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.028306 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.035712 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.028846 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:18.917348 27084 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:07:19.025431 21860 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:19.030300 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.040210 21851 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:19.027715 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.009497 26758 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:18.936208 27054 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.235s
W20260709 13:07:19.043820 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 121454us
I20260709 13:07:19.046300 22588 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:19.028251 26130 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.541s	user 0.017s	sys 0.004s
I20260709 13:07:19.027915 26764 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.031200 26688 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.049068 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.055862 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:19.055723 26130 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.060798 26130 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:19.065299 26798 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.531s	user 0.003s	sys 0.006s
I20260709 13:07:19.066047 26798 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.068120 26798 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.071332 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.072273 26688 raft_consensus.cc:2764] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:19.073534 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.074425 26688 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:19.000586 26995 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.045825 27086 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.047262 26758 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.076984 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.009622 26988 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.052055 27034 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:19.097055 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.085561 21926 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:19.082688 27011 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:19.105201 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:19.100298 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94786.1us
I20260709 13:07:19.104557 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.081915 27099 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.105890 21935 raft_consensus.cc:2368] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:07:19.107419 21928 raft_consensus.cc:2368] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13.
I20260709 13:07:19.108269 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.108471 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.108768 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:19.109699 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 157420us
W20260709 13:07:19.110399 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.111491 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:19.112735 27087 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:07:19.116518 27018 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.602s	user 0.007s	sys 0.002s
I20260709 13:07:19.117229 27089 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.117708 27089 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.119630 27089 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.120138 21934 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
W20260709 13:07:19.120574 26983 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.000s
I20260709 13:07:19.122267 26988 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.122422 26690 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.122503 26931 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:19.123082 26931 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:19.123517 27099 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.123909 26931 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.132128 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.133230 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.133877 21933 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:19.136329 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.137583 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.057667 26764 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.140331 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.135592 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:19.141173 21857 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:07:19.141575 22590 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:19.144325 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67458.3us
I20260709 13:07:19.103217 26688 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:19.149796 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:19.143985 21856 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:19.149811 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.152758 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:19.153339 22589 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.105572 27011 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:19.154748 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:19.155503 27079 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.263s	user 0.001s	sys 0.001s
I20260709 13:07:19.156139 27096 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:19.156836 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.158304 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.158895 21851 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:19.149917 27060 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.179199 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1018) took 1013 ms. Trace:
I20260709 13:07:19.130198 27099 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.159132 26992 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:07:19.163187 27062 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.163506 26764 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.164206 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.143075 26989 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:19.164588 27026 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.164774 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:19.165911 26917 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.956s	user 0.012s	sys 0.000s
W20260709 13:07:19.166010 27088 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:07:19.166625 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.150133 26798 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.201995 26917 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.202534 26798 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.203616 22008 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:19.204209 26917 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.204523 26798 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.143290 26130 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:07:19.146368 26988 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:19.150260 26690 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
W20260709 13:07:19.176138 26978 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.719s	user 0.007s	sys 0.002s
I20260709 13:07:19.205685 27011 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.206760 27011 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.180469 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:19.183390 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.168689 26688 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:19.208663 26688 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.177961 26697 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.192535 21860 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
W20260709 13:07:19.193979 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61919.5us
I20260709 13:07:19.186522 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.214970 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.215818 26854 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:19.216526 26854 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:19.218691 26854 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:19.195124 21859 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:19.179689 27060 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:19.187646 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50489.6us
W20260709 13:07:19.159153 27063 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.002s
I20260709 13:07:19.221377 27096 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:19.221524 27063 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.222400 27096 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.222807 27060 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.179605 22002 rpcz_store.cc:276] 0709 13:07:18.166218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.318460 (+152242us) service_pool.cc:224] Handling call
0709 13:07:18.339923 (+ 21463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.387303 (+ 47380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.387334 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.391541 (+  4207us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.520447 (+128906us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:18.520493 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.520559 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.178971 (+658412us) raft_consensus.cc:1685] finished
0709 13:07:19.178994 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.179129 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27457,"raft.queue_time_us":18031,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8334,"thread_start_us":70786,"threads_started":1,"wal-append.queue_time_us":342658}
I20260709 13:07:19.223672 27063 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.206444 26978 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.225054 21933 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:19.225276 27065 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.225965 26978 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.161365 26931 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.226584 21857 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:19.226944 26620 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:19.228868 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:19.229254 27064 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:19.229704 26620 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:19.230232 26620 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:19.231254 26620 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.233690 27102 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.234236 27102 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.236507 27102 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.238137 26758 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.240551 26758 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.240993 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.242614 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.243398 21928 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:19.245249 21935 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:19.254617 22590 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
W20260709 13:07:19.256333 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75934.1us
I20260709 13:07:19.257282 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.259200 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:19.250322 21900 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.268968 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.283502 21928 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:19.263039 26917 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:19.297398 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47148us
W20260709 13:07:19.299700 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.192479 26235 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.263187 27102 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:07:19.310415 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69515.3us
I20260709 13:07:19.292950 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.281903 26130 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:19.289214 26989 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:19.297780 27103 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:19.297233 26988 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:19.312162 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.319991 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:19.310602 26831 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:19.313731 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.255602 26808 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:19.311949 26235 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:19.330258 21928 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:19.322314 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:19.333559 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.333386 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.336112 21926 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:19.338331 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.338945 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.341144 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.323400 26992 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:19.319940 26130 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.322613 26917 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:19.344228 21926 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:19.329295 26988 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:19.335633 26235 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.342772 26992 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.353979 26992 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:19.354991 27064 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:19.355473 27064 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:19.344018 26130 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:19.359019 26688 raft_consensus.cc:2764] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:19.358587 26917 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:19.354003 26989 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.362890 26808 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:19.365513 21926 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:19.368973 26684 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:19.372395 27072 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.530s	user 0.002s	sys 0.002s
I20260709 13:07:19.356314 27064 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.342963 26798 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:19.341220 26983 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:19.380899 21926 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:19.383594 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.383594 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.351737 27065 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:19.385252 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.386737 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:19.387483 21859 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.384270 21857 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:19.393486 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.353463 26988 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:19.402295 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:19.401973 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.403594 21854 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:07:19.382452 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:19.404753 22591 raft_consensus.cc:606] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.405128 22591 raft_consensus.cc:614] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:19.397878 26983 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:19.406780 26983 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.400638 26831 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:19.408542 26992 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.411211 21858 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:19.412693 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.413919 27065 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:19.414078 21854 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:19.382067 26798 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.408807 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.422648 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.424216 21853 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:19.400537 27069 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.427150 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.428303 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.435418 22589 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
W20260709 13:07:19.430225 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 110320us
I20260709 13:07:19.436873 21935 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:19.437597 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.438570 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.422508 26798 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:19.441615 21859 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:19.437369 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:19.443071 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:19.426111 26909 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:19.446619 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.446766 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4021839a030d44ab886833307930ad33"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:19.447253 22591 raft_consensus.cc:606] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.447252 21933 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:19.426896 27067 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:19.454391 26110 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:19.458556 26110 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:19.459095 26110 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:19.433928 27069 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:19.457656 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.462725 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.463302 21933 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.443492 26690 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:19.465564 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.469301 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.493808 21928 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:19.495076 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.469969 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.496719 21926 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:19.464773 27069 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.499409 27070 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.448872 26909 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:19.474996 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.501477 26909 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:19.476058 26831 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.501696 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.502398 21857 raft_consensus.cc:2376] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16. Re-sending same reply.
I20260709 13:07:19.505241 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:19.501801 21859 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:19.445828 27095 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:07:19.481406 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.488643 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.484895 27065 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.473016 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.507603 21929 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:19.505708 21855 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:19.491182 27116 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:19.449703 27103 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:19.449265 26792 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:19.518116 26130 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.365183 26808 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.525530 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.527395 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.532707 21824 leader_election.cc:400] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13.
I20260709 13:07:19.533885 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.538120 21824 leader_election.cc:403] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.538343 21931 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:19.539480 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:19.548846 21930 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:19.546913 21926 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:19.553496 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:19.553895 26130 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.553856 21928 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:19.533711 27067 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:07:19.555840 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.556116 22005 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:19.449882 26688 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:07:19.560694 22006 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:19.556443 22002 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
W20260709 13:07:19.570469 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45105.1us
I20260709 13:07:19.571931 21823 leader_election.cc:400] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13.
I20260709 13:07:19.573828 27054 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:19.574494 27054 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:19.574903 27054 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.579919 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.580508 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f588420255042dda396ba6419abf6bc"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:19.581161 22591 raft_consensus.cc:606] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.581466 22591 raft_consensus.cc:614] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:19.581723 21823 leader_election.cc:400] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:07:19.582011 21823 leader_election.cc:403] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.553035 26758 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13.
I20260709 13:07:19.593089 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.600056 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.607048 21931 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:19.558660 27070 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:19.565074 27103 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:19.588183 26902 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.590266 27063 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.621704 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 164100us
I20260709 13:07:19.583719 26992 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:19.624027 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 145970us
I20260709 13:07:19.624286 21929 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:19.615568 26110 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.597599 27074 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11.
I20260709 13:07:19.646211 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.626827 21933 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
W20260709 13:07:19.640389 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 152533us
I20260709 13:07:19.652357 21859 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:19.609210 27116 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:19.607550 27011 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:19.617595 27086 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.620847 27070 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:19.622509 26902 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:19.657903 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.667810 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.672380 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:19.672739 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.674739 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.675364 21927 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:19.677560 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.636185 27020 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:19.681512 21853 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:19.682291 21931 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:19.683935 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.684628 21927 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16.
I20260709 13:07:19.661319 27116 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:19.688711 27116 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.690207 27116 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.669992 26902 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.693871 21929 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:07:19.694232 21929 pending_rounds.cc:85] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 8
I20260709 13:07:19.694440 21929 pending_rounds.cc:107] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:07:19.694661 21929 raft_consensus.cc:2889] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:19.697923 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.699050 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:19.700277 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.702989 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:19.703830 21933 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:19.708709 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:19.628494 27063 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.710378 27074 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:19.715180 27084 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:19.715607 27084 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.715940 27084 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:19.716252 27063 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.704886 27020 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.718691 27020 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:19.642781 26931 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:07:19.711125 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.730743 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:19.709921 21933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:07:19.711038 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:19.733052 21933 raft_consensus.cc:606] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:19.733207 21854 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.733424 21933 raft_consensus.cc:614] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:19.648705 26758 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:07:19.747306 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.720924 26902 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.761106 26902 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.742990 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:19.763088 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:19.764145 21853 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:19.765025 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.766745 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 115826us
I20260709 13:07:19.769815 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.772502 22001 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:19.774922 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.743309 26983 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.778556 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.703470 27026 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:07:19.765862 26758 consensus_queue.cc:1243] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:19.755391 27074 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:19.794502 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.773717 26902 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:19.804205 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:19.802120 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45773.7us
I20260709 13:07:19.805753 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.805300 22004 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:19.806950 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.780929 26690 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:19.742391 26798 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.814921 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:19.765698 27020 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:07:19.845626 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:19.861544 22001 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:19.768532 26591 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:07:19.792569 26831 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:19.732012 27103 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.763025 26917 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:07:19.844014 22006 raft_consensus.cc:2410] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:07:19.790721 26983 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.882079 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:19.858659 26902 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:19.792703 27070 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.857064 26931 raft_consensus.cc:2764] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:19.853271 26758 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:19.913287 26758 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:19.915506 26758 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:19.882325 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76734.7us
I20260709 13:07:19.895062 21854 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:07:19.884245 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 240663us
I20260709 13:07:19.917918 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.918246 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:19.919229 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.919373 26909 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.903574 26902 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:19.906493 26983 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:19.917903 26854 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:19.855996 27121 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:07:19.901162 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 132174us
I20260709 13:07:19.934798 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:19.938086 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:19.940732 22000 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:19.856990 27124 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 13:07:19.985459 27125 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:07:19.933488 27064 raft_consensus.cc:2764] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:07:19.989825 27128 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:07:19.919997 26992 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:19.992205 27077 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 1.162s	user 0.006s	sys 0.000s
W20260709 13:07:19.944051 27118 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:07:20.000550 22005 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:07:19.999413 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52869us
I20260709 13:07:20.033191 27105 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:20.033829 27105 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:19.943063 27060 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:20.049883 27120 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.365s	user 0.000s	sys 0.002s
W20260709 13:07:20.050511 26235 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.693s	user 0.007s	sys 0.006s
I20260709 13:07:20.050926 26235 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:20.053803 27126 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:07:20.055291 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117295us
I20260709 13:07:20.057559 27064 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:20.058749 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.066691 26992 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:20.067307 26992 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:20.067704 26992 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.070611 27136 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:20.072844 27105 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.073686 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:20.074029 27105 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:20.074337 21927 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:20.075012 27130 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:07:20.077172 26909 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:20.085204 27134 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:20.085691 27134 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:20.086727 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:20.086625 27134 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.087291 21928 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.087905 27134 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:20.090247 27001 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:20.090790 27001 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:20.091676 27001 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.008941 26931 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:07:20.095741 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.099587 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.100452 21856 raft_consensus.cc:2410] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:07:20.100841 22588 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:20.101334 22593 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:19.944473 27095 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:07:20.105487 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.106163 21856 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:20.106272 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:20.106575 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:20.107018 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:20.108053 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:20.108239 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.105497 22589 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:20.110298 21856 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:20.111107 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.111958 21927 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:20.098258 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:20.113920 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:20.092782 27001 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:20.157624 21853 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:20.164372 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:20.047514 27071 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:20.117445 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:20.120752 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.125924 22005 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:20.126633 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:20.128041 27069 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.001s
W20260709 13:07:20.128600 27132 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:07:20.182034 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:20.136152 27084 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.137645 26798 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.138391 22593 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:20.109393 27060 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.146270 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1088) took 1119 ms. Trace:
I20260709 13:07:20.146759 26931 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:20.128084 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:20.188521 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60522us
I20260709 13:07:20.191310 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:20.193188 22004 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:07:20.150877 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52897.3us
I20260709 13:07:20.197513 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:20.151822 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.176477 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:20.177724 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 257906us
I20260709 13:07:20.178694 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:20.180054 21858 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:07:20.199889 21858 pending_rounds.cc:85] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 7
I20260709 13:07:20.200189 21858 pending_rounds.cc:107] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:07:20.200471 21858 raft_consensus.cc:2889] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:20.173269 21856 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:20.183805 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:20.184535 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:20.185346 22008 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:20.202255 22006 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:20.203291 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:20.204789 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53069us
I20260709 13:07:20.166983 26758 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.207836 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:20.241420 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44013.8us
I20260709 13:07:20.166750 27020 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:20.256336 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.252743 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:20.186034 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.216194 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.182924 26235 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.259658 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:20.167166 26983 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:20.186870 22592 rpcz_store.cc:276] 0709 13:07:19.027200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.027366 (+   166us) service_pool.cc:224] Handling call
0709 13:07:19.027688 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.046920 (+ 19232us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:19.291224 (+244304us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.725568 (+434344us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.725633 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:19.725691 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.145957 (+420266us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b54006009d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:20.146010 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.146166 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9470,"prepare.queue_time_us":11851,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":63349,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":16568,"spinlock_wait_cycles":70853120,"thread_start_us":667445,"threads_started":3,"wal-append.queue_time_us":373265}
I20260709 13:07:20.172024 27071 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:20.308574 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:20.303126 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50458.8us
W20260709 13:07:20.311409 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 103651us
I20260709 13:07:20.313380 27086 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:20.254657 27099 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:20.207087 26591 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:20.317370 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:20.312564 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 96504us
I20260709 13:07:20.167222 27070 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:07:20.314839 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:20.255722 27116 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:20.261910 27054 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.687s	user 0.016s	sys 0.006s
I20260709 13:07:20.259598 27107 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:20.254611 27115 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:20.208155 27105 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:20.184742 27069 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.255172 27026 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:07:20.262964 27129 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:07:20.311865 26989 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:07:20.375669 21856 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:20.309983 26983 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:20.386873 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 69540.1us
I20260709 13:07:20.261413 27065 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:20.342092 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.382755 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:20.504475 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 186520us
I20260709 13:07:20.263415 26758 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.320591 27064 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:20.354552 21928 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:20.524859 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:20.527094 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 185119us
I20260709 13:07:20.326184 27099 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.527503 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 977) took 2171 ms. Trace:
I20260709 13:07:20.374469 27144 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:20.350214 27096 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:20.535810 22007 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:20.534138 22586 rpcz_store.cc:276] 0709 13:07:18.348604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.348749 (+   145us) service_pool.cc:224] Handling call
0709 13:07:18.387181 (+ 38432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.564287 (+177106us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:18.881283 (+316996us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003eadc0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf 00007f3b707e9038
0709 13:07:19.010238 (+128955us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.181659 (+171421us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.181864 (+   205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.195807 (+ 13943us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.195954 (+   147us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:19.195988 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.519493 (+1323505us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54004d12d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:20.519571 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.519749 (+   178us) raft_consensus.cc:1685] finished
0709 13:07:20.519782 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.519993 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99000,"prepare.queue_time_us":471073,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":55140,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":13498,"spinlock_wait_cycles":352368128,"thread_start_us":197542,"threads_started":2,"wal-append.queue_time_us":170098,"wal-append.run_cpu_time_us":2998,"wal-append.run_wall_time_us":1114879}
W20260709 13:07:20.537942 22584 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1092) took 1372 ms (client timeout 1638 ms). Trace:
W20260709 13:07:20.538511 22584 rpcz_store.cc:269] 0709 13:07:19.165998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.166306 (+   308us) service_pool.cc:224] Handling call
0709 13:07:20.537665 (+1371359us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540060d6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:20.537843 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2418128128}
I20260709 13:07:20.318735 26798 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.204s
I20260709 13:07:20.365763 27063 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:20.501155 27116 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.500900 27115 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:20.549857 21857 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:07:20.590607 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 207940us
I20260709 13:07:20.555696 21856 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:07:20.527626 26983 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.319442 27020 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.559212 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:20.556747 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 315355us
W20260709 13:07:20.592044 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.591164 21856 pending_rounds.cc:85] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 6
I20260709 13:07:20.596693 21856 pending_rounds.cc:107] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:07:20.597082 21856 raft_consensus.cc:2889] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:20.527370 27065 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:20.650381 27151 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
W20260709 13:07:20.655115 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57834.8us
I20260709 13:07:20.551934 26831 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:20.553166 27084 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:20.592967 27107 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:20.597803 27116 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:20.656916 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:20.516613 27054 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.548470 27064 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:20.379781 26931 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.379313 27071 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.592672 27115 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:20.668592 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 109491us
I20260709 13:07:20.686079 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:20.546691 27099 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:20.559131 27063 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.592284 27166 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:20.605324 27065 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:20.680190 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:20.680809 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 139331us
I20260709 13:07:20.680708 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:20.663575 27155 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:07:20.675832 27144 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:20.662240 27096 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:20.668924 27060 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:20.668531 27086 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.661919 26591 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:20.668131 27105 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:07:20.709127 26591 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.711383 26591 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:20.712195 27086 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.714756 27086 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.714571 26983 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.716710 26983 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:20.697165 27160 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:07:20.687388 27103 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:20.698216 27134 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.006s
I20260709 13:07:20.675493 26831 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:20.688730 27154 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:07:20.716943 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:20.737709 26831 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.738456 27020 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:20.738655 22004 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:20.739612 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:20.741178 22590 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
W20260709 13:07:20.741905 27153 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:07:20.707569 27071 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.726097 22001 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:20.700145 27063 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:20.741266 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2251) took 1090 ms. Trace:
I20260709 13:07:20.693734 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:20.759048 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65400.5us
W20260709 13:07:20.759626 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 230530us
I20260709 13:07:20.760556 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:20.763804 21926 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:20.765519 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:20.766193 21933 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:20.766899 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62382.8us
I20260709 13:07:20.761036 21860 rpcz_store.cc:276] 0709 13:07:19.651081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.653232 (+  2151us) service_pool.cc:224] Handling call
0709 13:07:19.653523 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.664508 (+ 10985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.664538 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.675954 (+ 11416us) log.cc:844] Serialized 26 byte log entry
0709 13:07:20.165964 (+490010us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:20.186744 (+ 20780us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.187084 (+   340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.187154 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.741063 (+553909us) raft_consensus.cc:1685] finished
0709 13:07:20.741084 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.741211 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3091,"prepare.queue_time_us":35804,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":30245,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":55202,"thread_start_us":401594,"threads_started":1,"wal-append.queue_time_us":621738}
I20260709 13:07:20.748126 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:20.770874 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.772833 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:20.773541 21860 raft_consensus.cc:2368] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:07:20.774276 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:20.740903 26992 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.004s
I20260709 13:07:20.775960 21860 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:20.778299 22584 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:07:20.776933 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:20.783066 22584 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 8
I20260709 13:07:20.783521 22584 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.9
I20260709 13:07:20.783689 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:20.783902 22584 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Aborting config change with OpId 13.9: Aborted: Op aborted by new leader
I20260709 13:07:20.784346 22584 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:20.784958 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.785593 21860 raft_consensus.cc:2376] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13. Re-sending same reply.
I20260709 13:07:20.786535 21860 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:20.735606 27103 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.790783 21856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:07:20.796204 21856 raft_consensus.cc:606] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.796643 21856 raft_consensus.cc:614] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:20.766403 26983 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:20.724676 27064 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.767624 27020 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:20.807689 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:20.769833 27166 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:20.789920 27161 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:07:20.804365 27069 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:20.770479 27086 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.724758 27144 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.789546 27084 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:20.817338 27145 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:20.817761 27162 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:20.797379 27026 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:07:20.819301 26909 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:20.819873 27166 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.820710 27020 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:20.805717 26988 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.383s	user 0.012s	sys 0.003s
I20260709 13:07:20.794757 26992 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.801741 26798 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.802090 27103 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.825464 21857 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:20.843415 27136 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:20.860605 27109 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.557s	user 0.007s	sys 0.000s
I20260709 13:07:20.863219 26992 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.864921 26798 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.866528 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2701) took 1343 ms. Trace:
I20260709 13:07:20.866983 22007 rpcz_store.cc:276] 0709 13:07:19.523106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.547843 (+ 24737us) service_pool.cc:224] Handling call
0709 13:07:19.548156 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.536923 (+988767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.866231 (+329308us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400169254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:20.866432 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270831488}
I20260709 13:07:20.867153 27136 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.869221 21854 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:20.854544 26988 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:20.871855 26988 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.872714 21929 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:20.855110 26909 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:20.875351 26909 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:20.876091 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 895) took 1408 ms. Trace:
I20260709 13:07:20.854661 26758 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.876502 21932 rpcz_store.cc:276] 0709 13:07:19.467375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.467492 (+   117us) service_pool.cc:224] Handling call
0709 13:07:19.467839 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.468166 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.468193 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.519725 (+ 51532us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.649224 (+129499us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:19.787855 (+138631us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.788249 (+   394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.788388 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.870455 (+1082067us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b24001bef18. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d720fb0 00007f3b70397488 00007f3b70397be8 00007f3b7036f398 00007f3b7036a76a 00007f3b70369bb6
0709 13:07:20.875776 (+  5321us) raft_consensus.cc:1685] finished
0709 13:07:20.875883 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.876012 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284216,"prepare.run_wall_time_us":51,"raft.queue_time_us":2090,"raft.run_cpu_time_us":491,"raft.run_wall_time_us":10623,"spinlock_wait_cycles":45692928,"thread_start_us":239806,"threads_started":2,"wal-append.queue_time_us":293853}
I20260709 13:07:20.878991 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:20.883234 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:20.891405 21926 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:07:20.868753 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:20.893160 22007 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:07:20.896943 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:20.897284 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2707) took 1365 ms. Trace:
I20260709 13:07:20.898262 22000 rpcz_store.cc:276] 0709 13:07:19.531543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.548800 (+ 17257us) service_pool.cc:224] Handling call
0709 13:07:19.565327 (+ 16527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.947857 (+382530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.897046 (+949189us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400197dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:20.897226 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862257920}
I20260709 13:07:20.899529 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:20.900949 27054 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:20.901729 22000 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:20.830240 27084 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:20.907084 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57040.3us
I20260709 13:07:20.909420 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:20.827613 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61545.7us
I20260709 13:07:20.852757 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:20.895304 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:20.937640 22556 leader_election.cc:400] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13.
I20260709 13:07:20.938740 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:20.939867 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:20.927825 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 167389us
I20260709 13:07:20.944759 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:20.945739 21859 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:20.856042 27145 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:20.844602 27086 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:20.928099 27161 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:07:20.950526 21859 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:07:20.953025 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:20.966482 21859 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 8
I20260709 13:07:20.966657 21932 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.967057 21859 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.9
I20260709 13:07:20.967551 21859 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Aborting config change with OpId 13.9: Aborted: Op aborted by new leader
I20260709 13:07:20.968127 21859 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:20.976627 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:20.922062 26758 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:07:20.924752 27162 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:07:20.937453 27054 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:07:20.952680 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:20.922573 27084 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.964711 21852 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:20.973562 27124 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:20.992074 22584 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:20.948495 27121 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:07:20.978806 27020 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:20.961468 27069 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:07:21.000427 22586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:21.006103 22586 raft_consensus.cc:606] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:21.006699 22586 raft_consensus.cc:614] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:20.998651 26831 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:20.995514 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:20.960294 21933 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:21.009295 26831 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:20.980084 26620 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:07:20.993872 27124 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.001789 26978 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.017081 27124 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.017757 22586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bcf30627084357a4745445324a2430"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:21.017532 26978 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.018329 22586 raft_consensus.cc:606] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:21.019727 26978 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:21.020452 27001 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.859s	user 0.014s	sys 0.000s
I20260709 13:07:21.021288 27166 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.021780 27099 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.023286 27086 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.024225 27121 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:21.024430 27099 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.024641 27121 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:21.025353 21927 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:07:21.025611 27121 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.026317 27116 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.026891 27121 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.027346 21927 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:21.027812 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2254) took 1365 ms. Trace:
I20260709 13:07:21.028163 21854 rpcz_store.cc:276] 0709 13:07:19.662386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.919614 (+257228us) service_pool.cc:224] Handling call
0709 13:07:19.920253 (+   639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.869769 (+949516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.027605 (+157836us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540018f6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:21.027757 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181952768}
I20260709 13:07:21.029687 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2213) took 1326 ms. Trace:
I20260709 13:07:21.030056 21929 rpcz_store.cc:276] 0709 13:07:19.702985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.703107 (+   122us) service_pool.cc:224] Handling call
0709 13:07:19.716232 (+ 13125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.873334 (+1157102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.029480 (+156146us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400098dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:21.029633 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659100544}
I20260709 13:07:21.030818 27011 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.031344 27011 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.039111 21932 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:21.039429 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:21.039430 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:21.008208 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.049623 21927 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:21.032879 21929 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:21.041728 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:07:21.040323 22588 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:21.063946 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:21.064306 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.068701 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.072645 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:21.055280 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:21.080704 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:21.006002 26798 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.120671 26620 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:21.121459 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:21.092733 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.095480 22589 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:21.122184 21851 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:21.097882 22586 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:21.104763 27102 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:21.123035 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.105247 27060 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:21.123808 21851 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:21.110642 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2703) took 1586 ms. Trace:
I20260709 13:07:21.125500 22009 rpcz_store.cc:276] 0709 13:07:19.523798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.526995 (+  3197us) service_pool.cc:224] Handling call
0709 13:07:19.566541 (+ 39546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.582777 (+ 16236us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:19.759512 (+176735us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.939484 (+179972us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.940306 (+   822us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.941059 (+   753us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:19.941121 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.941208 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.060188 (+118980us) raft_consensus.cc:1685] finished
0709 13:07:20.060212 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.110450 (+1050238us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b540045b5d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:21.110572 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12921,"prepare.queue_time_us":195579,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":116297,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":25824,"spinlock_wait_cycles":15043968,"thread_start_us":312904,"threads_started":2,"wal-append.queue_time_us":296890,"wal-append.run_cpu_time_us":2051,"wal-append.run_wall_time_us":1002254}
I20260709 13:07:21.111891 26110 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.108533 26992 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:07:21.129252 26992 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:21.129690 26992 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:21.130807 26992 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.132359 26992 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.139654 26854 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:21.141866 27169 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:21.082762 27099 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.119722 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:21.038348 27054 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.151083 26110 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:21.151542 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:21.151686 26110 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:21.042539 27069 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:21.140396 26854 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.105096 27026 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:21.144461 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:21.152395 26110 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:21.164541 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 120724us
I20260709 13:07:21.158094 21972 leader_election.cc:400] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16.
I20260709 13:07:21.037473 26831 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.134s
I20260709 13:07:21.070789 22000 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:21.100224 27162 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:21.120532 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1094) took 1014 ms. Trace:
I20260709 13:07:21.071771 27001 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:07:21.177855 22593 rpcz_store.cc:276] 0709 13:07:20.105830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.105956 (+   126us) service_pool.cc:224] Handling call
0709 13:07:21.120322 (+1014366us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b54005019d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:21.120466 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71876352}
I20260709 13:07:21.155287 27069 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:21.144990 26978 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:07:21.196244 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.153112 21851 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:21.160562 26620 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:21.199102 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44197.6us
I20260709 13:07:21.162165 27026 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:21.202982 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:21.203887 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 214876us
I20260709 13:07:21.193183 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:21.171962 27011 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.191538 27162 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.153678 27054 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.220606 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:21.220861 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.157723 27099 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:21.191895 27177 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:21.230978 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:21.086122 27145 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.240598 21851 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:21.218569 27136 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.217479 27127 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:21.223441 27162 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.194603 27131 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:07:21.245025 26110 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.331321 21851 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:21.284435 26988 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.421984 26110 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:21.252409 27197 raft_consensus.cc:2764] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:07:21.249529 27136 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:21.291165 27020 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.322885 27011 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:21.290155 26620 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.136s
W20260709 13:07:21.427845 27063 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.003s
I20260709 13:07:21.322319 27169 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 13:07:21.244810 27177 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.431595 27166 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:21.457262 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 219891us
I20260709 13:07:21.327064 27165 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:07:21.248013 27069 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.284161 27026 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.430670 26988 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.475018 26988 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.475900 26854 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.459250 27186 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.464098 27166 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:21.439203 27020 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.465391 27011 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.497311 27011 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.499755 27011 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:21.500231 27020 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.513111 26235 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.514006 26235 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:21.517886 27196 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
W20260709 13:07:21.524298 27188 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 13:07:21.524756 27175 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.193s
I20260709 13:07:21.526117 27175 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:21.526523 27175 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:21.527719 27175 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:21.528460 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 296733us
I20260709 13:07:21.533084 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.533985 21930 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:21.520851 27116 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.535786 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2574) took 1431 ms. Trace:
I20260709 13:07:21.537993 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.538261 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.538715 21930 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:21.539803 21929 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:07:21.539927 27195 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:07:21.540526 21854 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:07:21.541321 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:21.541663 27210 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:07:21.542090 21927 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:21.542840 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.543243 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.544126 21932 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:21.544212 27193 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:07:21.546092 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:21.546890 21860 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
W20260709 13:07:21.547307 27192 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.313s	user 0.000s	sys 0.000s
I20260709 13:07:21.547888 27123 raft_consensus.cc:993] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:07:21.548454 27123 raft_consensus.cc:1081] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
W20260709 13:07:21.539821 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 773) took 2403 ms (client timeout 1945 ms). Trace:
W20260709 13:07:21.549146 21855 rpcz_store.cc:269] 0709 13:07:19.136002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.136107 (+   105us) service_pool.cc:224] Handling call
0709 13:07:21.539569 (+2403462us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:21.539711 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848028544}
I20260709 13:07:21.550987 21855 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:21.536264 21853 rpcz_store.cc:276] 0709 13:07:20.104040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.517391 (+413351us) service_pool.cc:224] Handling call
0709 13:07:20.534415 (+ 17024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.664860 (+130445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:20.664897 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:20.664942 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:20.664979 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.535488 (+870509us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:21.535544 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.535721 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299494272}
I20260709 13:07:21.556419 27166 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.544236 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.476922 27152 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.339s
I20260709 13:07:21.458681 27202 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:07:21.563606 27202 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:21.564157 27202 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:21.566258 26931 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:21.566844 26931 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:21.570513 27086 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:21.571084 27086 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:21.572059 27086 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:21.576359 27103 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.004s
I20260709 13:07:21.452322 27063 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:21.579681 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.582412 27177 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.582612 22006 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:21.582680 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:21.583184 27185 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 13:07:21.583571 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.583788 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.584530 22008 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:21.586961 27060 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:21.590693 27054 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.470347 26909 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:21.577476 27099 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.568055 26931 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.633232 21935 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:21.533169 27190 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.310s	user 0.001s	sys 0.001s
I20260709 13:07:21.439652 27136 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.642802 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:21.615204 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.481487 27199 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.616132 22008 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:21.619232 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.621814 21927 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:21.608734 27071 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.846s	user 0.012s	sys 0.005s
I20260709 13:07:21.599572 27069 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.599318 27026 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.627737 21929 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [11.6-14.8]
I20260709 13:07:21.669448 21855 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:21.623556 26909 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:21.632778 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.634582 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:21.636279 21934 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:07:21.614534 27175 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.679680 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:21.616671 27086 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.649365 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:21.634121 26902 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.647647 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:21.670604 21929 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:21.632339 21932 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:21.629112 27054 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:21.662827 27187 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
W20260709 13:07:21.689536 21823 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.533s (SENT) [suppressed 17 similar messages]
I20260709 13:07:21.641189 26235 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.604084 27103 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.711884 21935 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:21.664553 27123 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:21.713999 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 68264.2us
I20260709 13:07:21.715299 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:21.689694 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.687127 26992 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.002s
I20260709 13:07:21.720726 22592 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:07:21.719295 21859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.003s	sys 0.011s
I20260709 13:07:21.628095 27060 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.671900 27099 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:21.688015 26931 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.729346 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:21.732391 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:21.727118 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:21.745510 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:21.728503 27203 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:07:21.725692 22008 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.750499 22008 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.735749 21933 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:21.739041 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:21.739564 22588 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:21.712862 27074 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.713608 27168 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:07:21.727648 27115 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.035s	user 0.015s	sys 0.011s
W20260709 13:07:21.734441 21859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.004s	sys 0.011s
I20260709 13:07:21.751756 21855 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:21.752041 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.000s	sys 0.018s
W20260709 13:07:21.761962 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.000s	sys 0.019s
W20260709 13:07:21.762485 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 610094us
I20260709 13:07:21.728343 26235 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.766774 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:21.719789 27054 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:21.769656 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80151.9us
I20260709 13:07:21.717633 26902 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.774674 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:21.776283 21932 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:21.780902 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2690) took 1019 ms. Trace:
I20260709 13:07:21.759447 27062 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:21.759791 27116 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:21.789517 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.789753 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:21.781633 21928 rpcz_store.cc:276] 0709 13:07:20.761702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.761831 (+   129us) service_pool.cc:224] Handling call
0709 13:07:20.762163 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:20.771880 (+  9717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:20.771909 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:20.795277 (+ 23368us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.034437 (+239160us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:21.337182 (+302745us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.719220 (+382038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.719339 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.780717 (+ 61378us) raft_consensus.cc:1685] finished
0709 13:07:21.780736 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.780848 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4896,"prepare.queue_time_us":160296,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":45,"raft.queue_time_us":37461,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":10108,"thread_start_us":785770,"threads_started":3,"wal-append.queue_time_us":280774}
I20260709 13:07:21.791491 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2622) took 1073 ms. Trace:
I20260709 13:07:21.799445 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.794356 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.764524 26992 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:21.791991 22004 rpcz_store.cc:276] 0709 13:07:20.718258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.740944 (+ 22686us) service_pool.cc:224] Handling call
0709 13:07:20.742054 (+  1110us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:20.742725 (+   671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:20.742757 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:20.743252 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:07:20.967957 (+224705us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:20.968013 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.968104 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.791275 (+823171us) raft_consensus.cc:1685] finished
0709 13:07:21.791299 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.791432 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":224184,"threads_started":1,"wal-append.queue_time_us":347536}
I20260709 13:07:21.806567 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:21.788309 26988 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:07:21.808604 27214 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.760274 26995 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:07:21.810424 26995 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:21.808002 27177 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.812189 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:21.813082 22590 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:21.814752 22003 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:21.778644 26764 raft_consensus.cc:2764] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:07:21.779484 27084 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.779s	user 0.008s	sys 0.006s
I20260709 13:07:21.593161 27186 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.785300 27102 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:21.804975 27204 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:07:21.765659 27074 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.765470 27107 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:07:21.676678 27199 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.824836 27063 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.825482 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:21.826486 27060 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:21.814710 27124 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.012s
I20260709 13:07:21.853443 27124 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.854384 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.855628 27124 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.784471 27200 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:21.856374 27200 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.858630 27200 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:21.812747 27065 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.086s	user 0.014s	sys 0.014s
I20260709 13:07:21.859345 27065 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.832713 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.860301 22005 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:21.764256 27115 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.861434 27065 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.861510 22591 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:21.741498 27123 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.862534 27115 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.836786 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:21.863551 27123 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.836956 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:21.863931 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:21.864238 22592 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:21.864675 22587 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:21.811612 26995 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:07:21.838020 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2624) took 1050 ms. Trace:
I20260709 13:07:21.768074 26909 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.809186 27214 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:21.846397 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.859535 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.775221 26235 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.848843 21856 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:21.871258 22592 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:21.829640 27136 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.834043 22004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2bcf30627084357a4745445324a2430"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:32904
I20260709 13:07:21.879930 22008 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:21.880149 22004 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.880558 22004 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:21.815553 26992 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.842563 27099 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.871838 22001 rpcz_store.cc:276] 0709 13:07:20.787381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.787516 (+   135us) service_pool.cc:224] Handling call
0709 13:07:20.787818 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:20.843461 (+ 55643us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:20.849745 (+  6284us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.137133 (+287388us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b1800132b20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:21.533403 (+396270us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b1800132b20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:21.643499 (+110096us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.686488 (+ 42989us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.686644 (+   156us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:21.686678 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.686740 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.837821 (+151081us) raft_consensus.cc:1685] finished
0709 13:07:21.837845 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.837963 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60891,"prepare.queue_time_us":5603,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":179083,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":11860,"spinlock_wait_cycles":15148288,"thread_start_us":488655,"threads_started":2,"wal-append.queue_time_us":612222}
I20260709 13:07:21.822079 27084 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:21.618376 27202 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.821643 27177 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.928252 27177 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.929380 27202 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.931959 27166 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.933481 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:21.934689 21926 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:21.935717 27186 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:21.937479 26235 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.938639 21928 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:21.941890 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.943634 22006 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:21.945837 26992 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.895162 27099 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.948459 27099 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:21.891013 26909 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.948810 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.941890 27054 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.951838 27074 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.954275 27199 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.954393 26902 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:21.955823 27162 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.002s
I20260709 13:07:21.956947 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:21.957960 21929 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:21.958273 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.958916 27096 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.959149 26992 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.959621 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:21.959722 26992 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.960515 21859 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:21.961984 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.962149 26992 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.936826 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:21.970996 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:21.971560 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.972856 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.975137 22009 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:21.976250 27102 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.976783 27102 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.956261 27162 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.901237 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:21.911103 27103 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.913103 27084 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.913645 21854 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:07:21.917351 27116 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.989701 27116 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:21.990108 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 999) took 1191 ms. Trace:
I20260709 13:07:21.990474 22584 rpcz_store.cc:276] 0709 13:07:20.798301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.798443 (+   142us) service_pool.cc:224] Handling call
0709 13:07:20.798761 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.997178 (+198417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.989869 (+992691us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54006a04d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:21.990048 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443911936}
I20260709 13:07:21.991894 27116 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:21.875226 26697 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 13:07:21.920356 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.962149 27214 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:21.995659 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.996584 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2260) took 1034 ms. Trace:
I20260709 13:07:21.925354 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:22.006831 27086 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:21.926115 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.007468 22003 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:22.007328 27086 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:21.962287 27227 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 13:07:21.959549 27096 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.009866 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:22.010432 21853 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.010825 27096 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:22.010990 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:21.963310 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.011643 26758 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.895144 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:21.964267 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:21.964565 27121 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:21.997117 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:21.922676 21933 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:07:22.003913 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.012806 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:22.006070 27084 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:21.963733 27222 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:07:22.053745 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.018078 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:22.005357 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 68605.4us
I20260709 13:07:22.057674 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:22.020714 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:22.015527 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.058756 26130 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:22.059391 21856 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:07:22.059309 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 158777us
I20260709 13:07:22.019623 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:22.022312 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:22.017526 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:22.060359 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:22.060642 21929 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:22.031198 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.061640 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:22.038802 27224 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:07:22.039742 27166 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:22.054477 21854 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:22.088492 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:22.089103 21854 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.015434 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:22.023584 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.059990 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:22.091531 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 118592us
I20260709 13:07:22.011912 21857 rpcz_store.cc:276] 0709 13:07:20.962541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.962733 (+   192us) service_pool.cc:224] Handling call
0709 13:07:20.982141 (+ 19408us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:20.990768 (+  8627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:20.990806 (+    38us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:21.000439 (+  9633us) log.cc:844] Serialized 78 byte log entry
0709 13:07:21.108539 (+108100us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b18003d4380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:21.323455 (+214916us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:21.773511 (+450056us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.783428 (+  9917us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.995919 (+212491us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b18003d4380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:07:21.996174 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.996270 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.996326 (+    56us) raft_consensus.cc:1685] finished
0709 13:07:21.996343 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.996516 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":371264,"prepare.queue_time_us":362087,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":95,"raft.queue_time_us":244497,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":41304,"spinlock_wait_cycles":41615872,"thread_start_us":525269,"threads_started":2,"wal-append.queue_time_us":472806}
I20260709 13:07:22.087038 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:22.092657 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.037618 27162 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:22.025082 27102 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.013309 21859 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:22.075491 21858 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:22.078370 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.079478 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:22.079945 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.103951 22000 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:07:22.030465 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 996) took 1284 ms. Trace:
I20260709 13:07:22.094372 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.095610 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.096009 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:22.099313 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:22.109210 22001 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.110404 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.110658 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:22.106338 22590 rpcz_store.cc:276] 0709 13:07:20.746157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.748335 (+  2178us) service_pool.cc:224] Handling call
0709 13:07:20.748762 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.815041 (+1066279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.030227 (+215186us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400664254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:22.030393 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447131392}
I20260709 13:07:22.116834 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.119674 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:22.120224 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:22.093842 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.121074 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:22.115824 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.014330 27086 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:22.119237 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:22.126593 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.129904 21934 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:22.015131 27121 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:22.013309 26758 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.143873 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:22.086529 26837 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:22.137125 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:22.138863 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:22.140413 21859 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:22.141279 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.192947 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:22.194726 22593 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:22.142622 22004 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.147541 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:22.197500 22590 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
W20260709 13:07:22.204797 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 88811us
W20260709 13:07:22.194041 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 86202.7us
W20260709 13:07:22.197428 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 100554us
W20260709 13:07:22.196529 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55725.5us
I20260709 13:07:22.195817 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:22.112870 27197 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:22.144935 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.097445 26992 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:22.097054 26235 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.211745 22003 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:22.199481 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60683us
I20260709 13:07:22.220355 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:22.131947 26995 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:22.124327 27214 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:22.086295 26798 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:07:22.199915 22009 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:22.242715 22009 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:22.210296 26988 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:22.135927 27199 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:22.124148 27189 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:22.223302 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:22.131280 27166 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:22.219314 22003 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:22.211705 21854 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:22.205075 27070 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:22.229717 21930 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:22.229877 22004 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:22.235177 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.257773 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.227478 26992 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:22.224766 21825 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.094991 27084 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:07:22.245776 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:22.135867 27162 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
W20260709 13:07:22.261289 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.199033 27074 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:07:22.247844 22009 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.274452 22009 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.144119 26591 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:07:22.257772 22003 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.258703 26758 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:22.271121 22593 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.225589 27064 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:07:22.247148 27103 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:22.279130 22006 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:22.277336 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:22.214105 27219 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:07:22.260298 27166 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.260095 27199 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:22.269366 27070 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.268388 26995 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:22.230568 26235 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.273737 26992 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:22.320152 21855 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:22.269169 27179 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:07:22.287454 27199 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:22.282248 27103 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:22.284073 27166 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:22.295830 27070 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:22.276435 27095 consensus_queue.cc:1175] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 16 status { last_received { term: 16 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:22.366937 27103 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.328161 27152 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:07:22.352802 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.320588 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61647.2us
W20260709 13:07:22.365577 27236 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:07:22.353729 26995 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.329654 22589 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:22.629256 26130 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:22.269796 27121 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.302018 27034 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:22.609670 27233 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.290s	user 0.000s	sys 0.007s
I20260709 13:07:22.616290 22593 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:22.619418 26854 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:22.629169 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 283457us
I20260709 13:07:22.326620 27172 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:22.643909 27145 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:22.696201 26854 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.696894 27145 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:22.697854 27145 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.698827 26854 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:22.699054 27145 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:22.644486 26235 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:22.647213 26909 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.012s
I20260709 13:07:22.701556 27214 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:22.702541 27214 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.611692 27177 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.650601 21858 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:22.681188 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2270) took 1638 ms. Trace:
I20260709 13:07:22.683404 26837 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:22.684368 26764 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:22.684382 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 488585us
W20260709 13:07:22.685148 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 540234us
I20260709 13:07:22.727655 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:22.600313 26989 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.704900 27202 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:22.705792 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 481061us
I20260709 13:07:22.706966 27084 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:22.707041 27172 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:22.707571 26591 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:22.708003 26758 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:22.708405 27152 consensus_queue.cc:1243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:07:22.689112 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61162.4us
W20260709 13:07:22.773059 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.710011 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2271) took 1626 ms. Trace:
I20260709 13:07:22.715873 21851 rpcz_store.cc:276] 0709 13:07:21.043212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.430939 (+387727us) service_pool.cc:224] Handling call
0709 13:07:21.442732 (+ 11793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.442954 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:21.746180 (+303226us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.194126 (+447946us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.680452 (+486326us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b180011d160. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:07:22.680651 (+   199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:22.680718 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:22.680798 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.680917 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.681107 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24538,"prepare.queue_time_us":288705,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":70,"raft.queue_time_us":487475,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"spinlock_wait_cycles":16637696,"thread_start_us":668700,"threads_started":2,"wal-append.queue_time_us":532147}
I20260709 13:07:22.700923 26931 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:22.728319 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:22.775784 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47503.3us
I20260709 13:07:22.774477 21860 rpcz_store.cc:276] 0709 13:07:21.083284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.547716 (+464432us) service_pool.cc:224] Handling call
0709 13:07:21.548045 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.886165 (+338120us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:21.914096 (+ 27931us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.708077 (+793981us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:22.708130 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:22.708189 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.709764 (+  1575us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b540025ab54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:22.709812 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.709952 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":395276,"prepare.queue_time_us":28207,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":401881,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":6037,"spinlock_wait_cycles":777093504,"thread_start_us":398303,"threads_started":1}
W20260709 13:07:22.777032 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49411.6us
I20260709 13:07:22.735059 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2264) took 1745 ms. Trace:
W20260709 13:07:22.773928 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.736115 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:22.744103 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:22.754202 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.085s	sys 0.096s
W20260709 13:07:22.787688 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.085s	sys 0.097s
I20260709 13:07:22.727182 27065 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:22.800134 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 85985.8us
I20260709 13:07:22.779325 21852 rpcz_store.cc:276] 0709 13:07:20.989867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.991869 (+  2002us) service_pool.cc:224] Handling call
0709 13:07:21.003291 (+ 11422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.317633 (+314342us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:21.968552 (+650919us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b18003d44a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf
0709 13:07:21.968702 (+   150us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.684522 (+715820us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.685678 (+  1156us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.685837 (+   159us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:22.694576 (+  8739us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.708912 (+ 14336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.708978 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.734877 (+ 25899us) raft_consensus.cc:1685] finished
0709 13:07:22.734896 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.735006 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":555012,"prepare.queue_time_us":351167,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":3096,"raft.queue_time_us":553706,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":59457,"spinlock_wait_cycles":9609984,"thread_start_us":761181,"threads_started":3,"wal-append.queue_time_us":307101}
I20260709 13:07:22.764534 27074 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.805635 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:22.817826 22001 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.624s	user 0.006s	sys 0.005s
I20260709 13:07:22.755818 27102 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:22.740370 27084 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.760696 27197 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:22.740680 27202 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:22.766680 27172 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.831967 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:22.839352 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:22.824239 22001 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:22.799724 27173 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:07:22.771148 26591 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:22.802389 27086 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:22.786460 26931 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:22.833508 27065 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.957944 26931 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.959246 27065 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:22.959651 26931 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.902267 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.962334 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.963744 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.964654 21852 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:22.771407 26758 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:22.965575 27232 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:07:22.759886 26989 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.965962 26758 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.967900 26758 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.733919 27229 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:07:22.800292 27200 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:22.907415 27197 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:22.974071 27116 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:22.974853 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.981415 21852 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:22.982585 27162 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:22.982720 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:22.983212 27162 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:22.986172 21974 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.20.66.190:46167, user_credentials={real_user=slave}} blocked reactor thread for 84440.7us
W20260709 13:07:22.819636 27233 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:07:22.742995 27177 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.802322 27219 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:22.908816 26837 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:22.913697 26591 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:22.914855 27173 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:07:22.919325 22009 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.002s
I20260709 13:07:22.937397 26902 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:22.941816 27168 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:22.944432 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2276) took 1216 ms. Trace:
I20260709 13:07:22.945421 27020 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:22.905743 27152 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.975701 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 136476us
I20260709 13:07:23.172078 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:22.968727 26989 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:22.976644 27243 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:23.012737 26909 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.977924 27218 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.978355 27189 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.980000 21900 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:22.770627 27121 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:22.823235 27236 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:23.000058 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2631) took 1059 ms. Trace:
I20260709 13:07:23.005512 27197 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:23.010408 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.240521 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.240949 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.013183 22009 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:07:23.242365 22009 pending_rounds.cc:85] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 11
I20260709 13:07:23.242738 22009 pending_rounds.cc:107] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:07:23.243047 22009 raft_consensus.cc:2889] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:23.243896 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1070) took 1124 ms. Trace:
I20260709 13:07:23.244266 22009 rpcz_store.cc:276] 0709 13:07:22.119221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.125347 (+  6126us) service_pool.cc:224] Handling call
0709 13:07:22.199809 (+ 74462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.243569 (+1043760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.243829 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:22.906837 27179 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.888899 27034 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:22.851265 27084 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:22.802314 27011 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.205s
W20260709 13:07:23.024912 26992 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.001s
I20260709 13:07:23.025741 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2284) took 1105 ms. Trace:
I20260709 13:07:23.029325 26902 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:23.044600 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.046s	sys 0.031s
I20260709 13:07:22.997895 27099 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:23.162593 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.277154 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.056s	sys 0.044s
I20260709 13:07:22.999225 27202 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:23.232506 27162 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.169271 21855 rpcz_store.cc:276] 0709 13:07:21.728108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.774024 (+ 45916us) service_pool.cc:224] Handling call
0709 13:07:22.944127 (+1170103us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b540010c054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:22.944309 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070557952}
W20260709 13:07:23.177312 21973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.032s	sys 0.028s
W20260709 13:07:23.296805 21973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.032s	sys 0.029s
I20260709 13:07:23.180855 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:22.842026 27074 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:23.177137 26917 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.015s	sys 0.045s
I20260709 13:07:23.020834 27219 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:23.328425 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.329452 27219 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.329941 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:23.331014 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.333101 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.333334 26978 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.333992 26978 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.336515 26978 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.337291 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.338173 21926 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:23.338469 27247 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.233s	user 0.000s	sys 0.000s
I20260709 13:07:23.338922 26989 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:23.339937 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.340914 21935 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:23.235769 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1007) took 1115 ms. Trace:
I20260709 13:07:23.341416 22589 rpcz_store.cc:276] 0709 13:07:22.120429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.120674 (+   245us) service_pool.cc:224] Handling call
0709 13:07:22.120943 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.632304 (+511361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.235433 (+603129us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54005019d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:23.235687 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479150848}
I20260709 13:07:23.167917 26764 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.342381 27177 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.342756 26902 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:23.342654 26764 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.344049 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 923) took 1221 ms. Trace:
W20260709 13:07:23.219331 27069 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.023s	sys 0.044s
W20260709 13:07:23.248705 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 268762us
I20260709 13:07:23.344466 21932 rpcz_store.cc:276] 0709 13:07:22.122213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.128950 (+  6737us) service_pool.cc:224] Handling call
0709 13:07:22.193344 (+ 64394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:22.213661 (+ 20317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.213697 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:22.221973 (+  8276us) log.cc:844] Serialized 52 byte log entry
0709 13:07:22.730629 (+508656us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:22.786907 (+ 56278us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.787133 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.787222 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.343826 (+556604us) raft_consensus.cc:1685] finished
0709 13:07:23.343850 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.343991 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5291,"prepare.queue_time_us":56153,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":9282,"raft.queue_time_us":17116,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":3230,"spinlock_wait_cycles":13063424,"thread_start_us":472443,"threads_started":1,"wal-append.queue_time_us":1015128}
I20260709 13:07:23.238721 22008 rpcz_store.cc:276] 0709 13:07:21.940333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.969090 (+ 28757us) service_pool.cc:224] Handling call
0709 13:07:21.969430 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.091039 (+121609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.091079 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.101562 (+ 10483us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.692402 (+590840us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:22.692462 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.692557 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.999716 (+307159us) raft_consensus.cc:1685] finished
0709 13:07:22.999746 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.999961 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205332,"raft.queue_time_us":123997,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":14734,"thread_start_us":407460,"threads_started":1,"wal-append.queue_time_us":637534}
I20260709 13:07:23.175762 27173 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:23.169139 27168 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:23.254809 21826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.017s	sys 0.018s
I20260709 13:07:23.209826 27229 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:23.217063 27086 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:23.257560 21853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.019s	sys 0.023s
W20260709 13:07:23.378364 21853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.019s	sys 0.023s
I20260709 13:07:23.259684 21859 rpcz_store.cc:276] 0709 13:07:21.920474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.195128 (+274654us) service_pool.cc:224] Handling call
0709 13:07:22.195460 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.227569 (+ 32109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.227607 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:22.227658 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:22.227701 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.025454 (+797753us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b54000591d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:23.025517 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.025672 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73414912}
W20260709 13:07:23.284327 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 987867us
W20260709 13:07:23.291365 21854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.014s	sys 0.032s
W20260709 13:07:23.379720 21854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.015s	sys 0.032s
W20260709 13:07:23.297523 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.297727 22586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.077s	sys 0.120s
I20260709 13:07:23.179126 27020 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.299314 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:23.380682 22586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.078s	sys 0.120s
W20260709 13:07:23.381727 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.382352 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.302506 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.304461 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.385073 21927 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:23.312343 26130 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:23.386348 26130 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.252012 27067 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:23.318588 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 137835us
I20260709 13:07:23.388517 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:23.389735 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:23.254391 27197 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:23.310137 27107 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:23.211167 27152 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:23.293074 27127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.025s	sys 0.059s
W20260709 13:07:23.315553 26917 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.015s	sys 0.046s
I20260709 13:07:23.340826 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2599) took 1505 ms. Trace:
I20260709 13:07:23.228618 26909 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:23.204833 27103 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.005s
I20260709 13:07:23.255438 27245 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:23.414011 27245 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.414492 27245 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:23.398615 27131 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:23.417142 22002 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:23.344822 26764 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.232259 27218 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.293543 27096 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.290614 27102 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:23.424800 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.000s	sys 0.016s
W20260709 13:07:23.349422 27145 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.001s
I20260709 13:07:23.300818 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:23.390156 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:23.429158 22006 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:23.391778 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:23.256693 27179 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.237733 27189 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.256599 27034 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:23.316172 27162 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.348284 27069 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.024s	sys 0.044s
I20260709 13:07:23.398100 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:23.400574 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.417757 27131 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:23.403159 27127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.025s	sys 0.059s
I20260709 13:07:23.405683 21856 rpcz_store.cc:276] 0709 13:07:21.836040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.110737 (+274697us) service_pool.cc:224] Handling call
0709 13:07:22.111099 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.124006 (+ 12907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.124048 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.212469 (+ 88421us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.630116 (+417647us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:22.630161 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.630230 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.340586 (+710356us) raft_consensus.cc:1685] finished
0709 13:07:23.340612 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.340764 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":479754,"raft.queue_time_us":32442,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":18612,"thread_start_us":370686,"threads_started":1,"wal-append.queue_time_us":587970}
I20260709 13:07:23.413120 27103 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:23.420163 22588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.017s	sys 0.064s
I20260709 13:07:23.394953 27177 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:23.295639 27099 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.423810 26130 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:23.423964 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44420.5us
W20260709 13:07:23.369056 21826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.017s	sys 0.018s
I20260709 13:07:23.454488 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:23.425390 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.000s	sys 0.016s
I20260709 13:07:23.406227 27107 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.440305 27034 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:23.435220 21930 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:23.284327 26992 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.464429 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:23.403344 27067 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.465018 21856 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:23.447132 22588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.018s	sys 0.064s
I20260709 13:07:23.374012 27173 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.402828 27197 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.461057 22593 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:23.493431 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2600) took 1486 ms. Trace:
I20260709 13:07:23.493916 21854 rpcz_store.cc:276] 0709 13:07:22.007395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.259720 (+252325us) service_pool.cc:224] Handling call
0709 13:07:22.260018 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:22.703637 (+443619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.703672 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:22.704241 (+   569us) log.cc:844] Serialized 52 byte log entry
0709 13:07:23.380112 (+675871us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:23.380183 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.380278 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.493156 (+112878us) raft_consensus.cc:1685] finished
0709 13:07:23.493177 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.493362 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":443218,"raft.queue_time_us":446463,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":675449,"threads_started":1,"wal-append.queue_time_us":757808}
I20260709 13:07:23.498538 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:23.502431 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.024s	sys 0.063s
W20260709 13:07:23.502863 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.024s	sys 0.063s
I20260709 13:07:23.502956 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:23.505653 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.431972 27102 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:23.425669 21851 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.006s	sys 0.010s
I20260709 13:07:23.427749 27096 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.467480 26992 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.469091 27107 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.470649 27099 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:23.518915 22007 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:23.472209 27001 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:23.473080 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:23.457895 26764 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:23.522246 22589 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:23.473649 27094 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:23.474373 26909 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:23.475714 27218 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.460071 27011 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.477623 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 794) took 1705 ms. Trace:
I20260709 13:07:23.478255 21856 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:23.478540 27067 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:23.478834 27189 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:23.479560 21860 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:23.479748 27074 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:23.480057 27070 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.849s	user 0.010s	sys 0.000s
I20260709 13:07:23.480165 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2642) took 1401 ms. Trace:
I20260709 13:07:23.480772 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:23.481459 27179 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:23.482900 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:23.484674 27199 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.107s	user 0.007s	sys 0.004s
I20260709 13:07:23.485934 27020 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.489123 22584 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:23.513947 21851 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.006s	sys 0.010s
I20260709 13:07:23.489658 26792 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:07:23.513701 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:23.460479 27086 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:23.446401 27162 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.524330 27094 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:23.524204 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:23.516515 26758 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.548s	user 0.025s	sys 0.003s
W20260709 13:07:23.553988 26792 raft_consensus.cc:1068] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Unable to promote non-voter 34f3b83da84448bf8aac62b37a1486eb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:07:23.575567 27172 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.001s
I20260709 13:07:23.575249 27034 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.429188 27168 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.526437 21853 rpcz_store.cc:276] 0709 13:07:21.772587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.123689 (+351102us) service_pool.cc:224] Handling call
0709 13:07:22.199602 (+ 75913us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.682217 (+482615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.682265 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.682856 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.378799 (+695943us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:23.378842 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.378914 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.477431 (+ 98517us) raft_consensus.cc:1685] finished
0709 13:07:23.477449 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.477571 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":482226,"raft.queue_time_us":484174,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":695598,"threads_started":1,"wal-append.queue_time_us":768663}
I20260709 13:07:23.576067 27172 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.519966 27256 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.576951 27256 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.577607 27034 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.579375 27256 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:23.580966 22003 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:23.586107 27173 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:23.588490 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1201) took 1463 ms. Trace:
I20260709 13:07:23.588858 22586 rpcz_store.cc:276] 0709 13:07:22.125036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.125831 (+   795us) service_pool.cc:224] Handling call
0709 13:07:22.126230 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.196714 (+ 70484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.196760 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.209256 (+ 12496us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.381291 (+1172035us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:23.381352 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.381442 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.588273 (+206831us) raft_consensus.cc:1685] finished
0709 13:07:23.588292 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.588431 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72926,"raft.queue_time_us":72175,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":7918,"thread_start_us":1149612,"threads_started":1,"wal-append.queue_time_us":1301512}
I20260709 13:07:23.526501 26764 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.545552 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2608) took 1478 ms. Trace:
I20260709 13:07:23.546258 27199 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.449350 27131 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.546813 27152 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:23.540907 27070 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.547559 27084 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.546826 27103 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.500550 26831 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.539044 27214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.018s	sys 0.009s
I20260709 13:07:23.555015 22009 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:23.575659 22006 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:23.633649 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:23.634236 22006 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:23.539332 22002 rpcz_store.cc:276] 0709 13:07:22.078900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.079007 (+   107us) service_pool.cc:224] Handling call
0709 13:07:22.098766 (+ 19759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.417630 (+1318864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.479926 (+ 62296us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400426154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:23.480109 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3030956288}
I20260709 13:07:23.596483 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:23.538628 27011 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.617518 21851 rpcz_store.cc:276] 0709 13:07:22.067208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.786374 (+719166us) service_pool.cc:224] Handling call
0709 13:07:22.797358 (+ 10984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.797632 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:22.822777 (+ 25145us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.545154 (+722377us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.545224 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:23.545282 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.545365 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.545487 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":537,"prepare.run_cpu_time_us":146,"prepare.run_wall_time_us":9013,"raft.queue_time_us":147679,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":77,"thread_start_us":722014,"threads_started":1}
W20260709 13:07:23.531948 21934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.014s	sys 0.005s
I20260709 13:07:23.637660 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:23.637938 21852 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 9.
W20260709 13:07:23.575266 27237 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.844s	user 0.001s	sys 0.007s
W20260709 13:07:23.642139 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 245765us
W20260709 13:07:23.597735 26995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.010s	sys 0.013s
I20260709 13:07:23.553886 27202 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.637498 22586 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
W20260709 13:07:23.638041 21934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.014s	sys 0.006s
I20260709 13:07:23.556367 27086 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:23.628219 27253 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:23.669375 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:23.670500 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:23.672135 21858 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:23.643306 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:23.639041 27166 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.263s	user 0.006s	sys 0.007s
I20260709 13:07:23.674984 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:23.646888 22006 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:23.698508 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:23.717994 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:23.719013 26130 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:23.719646 26130 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:23.720402 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45522.8us
I20260709 13:07:23.680161 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:23.719828 22002 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:23.697181 22003 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:07:23.728027 22003 pending_rounds.cc:85] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 11
I20260709 13:07:23.730443 22009 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 9: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:23.730826 22003 pending_rounds.cc:107] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:07:23.730567 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:23.731163 22003 raft_consensus.cc:2889] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:23.730986 26130 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.731631 21856 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:23.731901 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1071) took 1606 ms. Trace:
W20260709 13:07:23.639598 27214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.018s	sys 0.009s
W20260709 13:07:23.673257 26995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.011s	sys 0.014s
I20260709 13:07:23.732348 22003 rpcz_store.cc:276] 0709 13:07:22.125967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.126155 (+   188us) service_pool.cc:224] Handling call
0709 13:07:22.200429 (+ 74274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.670665 (+470236us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b58007b62d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7072b0aa 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4
0709 13:07:23.580495 (+909830us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001a6920. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70744a1d 00007f3b7072b1a4 00007f3b707ad447 00007f3b707c5fe7 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419
0709 13:07:23.731664 (+151169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.731846 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382329984}
I20260709 13:07:23.733915 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:23.699108 27261 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:23.675390 26978 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:23.745102 21857 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:23.742741 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 362878us
I20260709 13:07:23.728410 27186 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:23.749220 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 427115us
W20260709 13:07:23.748814 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 294410us
I20260709 13:07:23.751446 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:23.752556 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:23.752719 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.753419 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.754390 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.758450 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:23.769164 21856 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:23.642823 26831 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.752875 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:23.727643 27166 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.779695 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:23.750337 27186 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.805531 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.792872 22593 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:23.813297 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:23.814692 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.805343 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52924.3us
I20260709 13:07:23.747191 26989 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:07:23.826683 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:23.897960 21932 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:23.854012 27011 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:23.861577 27265 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:23.892060 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:23.870931 26697 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:23.871016 27065 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:07:23.872586 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 59839.2us
I20260709 13:07:23.873334 27070 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.875633 27166 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.876134 26235 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:23.629927 27094 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.885341 22590 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:23.884027 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66454.4us
I20260709 13:07:23.816967 27186 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:23.757614 27219 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:23.811038 27162 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.811354 27096 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.826283 26995 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:23.870481 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 90897us
I20260709 13:07:23.883360 26909 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.914366 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:23.954124 21856 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:23.775942 26792 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:07:23.883021 27172 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:23.886304 27189 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.811865 27245 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.745500 27261 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:23.882813 27272 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:23.901232 26989 consensus_queue.cc:1243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:23.934198 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:23.942672 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2282) took 2022 ms. Trace:
I20260709 13:07:24.004254 21857 rpcz_store.cc:276] 0709 13:07:21.919935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.138487 (+218552us) service_pool.cc:224] Handling call
0709 13:07:22.138720 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.748507 (+1609787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.942437 (+193930us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:23.942610 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3693792640}
I20260709 13:07:24.006208 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.007504 21857 raft_consensus.cc:2410] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:23.948639 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:23.933517 26235 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:23.960430 27279 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:07:23.902760 27121 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:07:24.001876 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45060.1us
I20260709 13:07:24.010046 27067 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.950021 27199 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:23.920177 26697 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:24.027247 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 93137.1us
W20260709 13:07:24.022974 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74473us
I20260709 13:07:24.034760 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.036128 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.036778 21857 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:24.037600 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.038117 21857 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:24.038826 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:24.041826 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.026754 27067 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.050637 27067 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.053102 27276 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:24.054129 27099 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:24.055461 27245 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.027702 27062 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.056653 27062 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.057525 26697 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:23.965461 26909 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.058841 27062 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.060096 26909 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.061218 27095 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:24.061787 27095 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:24.062668 27095 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.045112 27152 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:07:24.066800 27152 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:24.067166 27152 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:24.067976 27152 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.045368 27063 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.021907 27272 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:24.077842 27275 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:07:24.078181 27272 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.073117 27152 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:24.084008 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50371.5us
W20260709 13:07:24.085455 27278 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:07:24.088635 27276 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:24.089552 26235 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.089635 27276 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:23.971869 27136 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:24.098397 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.101549 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.102583 21857 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:24.103303 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61549.4us
I20260709 13:07:24.106225 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:23.997634 27189 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.114634 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.043777 27202 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.094895 27172 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:24.098820 27269 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:07:24.116849 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.121503 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.122069 21928 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:24.117650 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.122967 21927 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:24.115069 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:24.124554 21857 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.126222 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:24.118197 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.127446 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.140336 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.141489 21926 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:24.142930 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.143942 21932 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:24.096818 27173 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:07:24.109933 26792 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.113469 27268 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.157831 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.129128 27272 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.151836 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.159192 21932 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 8.
I20260709 13:07:24.154711 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.160182 21926 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.118911 26235 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:24.155982 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.161926 21927 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.100610 27136 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.146544 27067 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:07:24.194363 21927 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:24.176656 27197 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.179440 27172 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:24.180403 26854 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.182296 27235 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.161s	user 0.004s	sys 0.004s
I20260709 13:07:24.184101 26764 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.185503 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.187650 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.217367 21856 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:24.188455 27131 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.189160 27086 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.189339 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.185110 21857 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:24.127360 27273 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:24.225780 27136 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.171120 27202 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.226687 27202 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.140759 27121 raft_consensus.cc:2764] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:07:24.232435 27121 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.232973 27121 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.235021 27121 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.114339 26909 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:07:24.146530 27285 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.237205 27285 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.237588 27285 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.240240 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.146740 27144 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:07:24.242760 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.244148 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:24.244536 22593 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:24.244882 22589 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.221858 26854 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.245710 27268 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:24.246251 27268 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:24.247211 26854 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.247350 27268 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.225823 27084 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.249478 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.250394 27189 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.250566 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.147028 27179 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:24.192670 27270 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.251446 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.252025 22007 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:24.251696 27270 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.223848 27219 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.252909 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.253075 27219 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.253770 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.251571 27179 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.255261 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.168242 27127 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:24.255899 22003 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:24.255935 27186 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.256302 27127 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.256983 27179 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.258260 27127 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.259109 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:24.260900 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.261030 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.261533 27179 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.261752 21851 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:24.262420 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.208458 27281 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.114s	user 0.001s	sys 0.001s
I20260709 13:07:24.264855 21855 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:24.265141 21931 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:24.265787 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.209610 27064 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:24.268254 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.267047 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.269140 21931 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:24.270539 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.272466 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.273415 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:24.273517 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.275501 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.277218 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.277308 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.278105 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.278461 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.279139 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.279644 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.280148 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.280521 22000 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:07:24.255239 27219 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.266026 27261 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.211135 27172 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:24.281617 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.282692 22006 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:24.282388 27172 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.283720 27172 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:24.283854 22006 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:24.284833 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.285535 22003 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:24.286204 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.287060 22003 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:24.288357 22004 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:24.289374 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.277424 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.290511 21934 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:24.291750 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.290442 21926 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:24.301793 22589 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:24.302399 27186 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.286150 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:24.303200 22006 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.303421 27152 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.303925 27152 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.305991 27152 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.267313 27268 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.317602 27189 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.318450 27189 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.319507 21931 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:24.319664 27065 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:24.322324 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 133050us
W20260709 13:07:24.323503 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.324440 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66985.1us
I20260709 13:07:24.327310 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.323617 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.320358 27065 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.332669 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.387112 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:24.387197 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:24.327418 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50763.1us
I20260709 13:07:24.255209 27270 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.369199 27202 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.332758 27218 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.370999 27065 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.371192 26854 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.372948 26831 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:24.373736 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50207.3us
I20260709 13:07:24.373680 27285 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:24.376084 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48877.4us
I20260709 13:07:24.378149 27062 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:24.262023 27179 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.381616 22001 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:24.429584 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.385823 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1207) took 1596 ms. Trace:
I20260709 13:07:24.331377 22006 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:07:24.430387 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:24.430594 22584 rpcz_store.cc:276] 0709 13:07:22.789507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.789795 (+   288us) service_pool.cc:224] Handling call
0709 13:07:22.790187 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.548783 (+758596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.385593 (+836810us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400595454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:24.385764 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552327936}
W20260709 13:07:24.395236 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.437769 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.439395 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.410418 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.388816 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.444496 21851 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:24.417387 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.420271 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 292863us
I20260709 13:07:24.421784 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.428158 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.446360 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.451510 27270 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.426679 21860 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:24.452049 27270 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.452534 27270 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.318462 27001 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:24.336366 27189 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.403679 27162 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:24.401292 27186 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.468505 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.449064 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.480809 21851 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:24.481158 22008 raft_consensus.cc:2410] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:07:24.469023 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.469583 27001 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:24.332713 26697 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.133s
I20260709 13:07:24.473467 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.475293 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.489941 21860 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
W20260709 13:07:24.490662 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.491889 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.448012 27062 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:24.459570 27127 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.467530 27219 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.471429 26831 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:24.466877 27245 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.480087 27084 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.484889 27189 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:24.469883 27099 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:07:24.538506 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65112.9us
I20260709 13:07:24.447338 26854 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.488966 27162 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:24.495731 27062 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:24.466323 26758 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.546991 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.535578 27144 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.532928 27219 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:24.555932 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.560631 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.561655 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.540532 27297 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.562052 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.493558 27186 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.553893 27162 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:24.547801 26798 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.532473 27127 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.566236 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.563354 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.537024 27245 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.560123 27084 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:24.561867 27219 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.564000 27026 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.593567 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:24.569072 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:24.593976 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 164495us
I20260709 13:07:24.599493 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.566927 27261 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.542634 27202 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:24.603255 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.557524 26758 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.607583 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.610852 21926 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:24.636734 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.611510 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:24.611817 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45654.1us
I20260709 13:07:24.645653 21860 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:24.613296 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.595429 27020 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:24.590229 26798 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.615381 27179 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.618024 26854 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.622391 27001 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:24.596699 27293 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:07:24.559358 27144 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.572239 27296 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:24.595623 27291 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:07:24.647723 26909 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:24.571698 27268 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:07:24.609679 27200 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.654597 27202 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:24.659211 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46025.9us
I20260709 13:07:24.634728 27069 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.643739 27256 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:24.651593 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.643374 27064 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:24.569494 27297 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.644014 26995 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:24.666700 27144 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.667333 27102 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.668272 27245 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.668473 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2606) took 2602 ms. Trace:
I20260709 13:07:24.668684 27285 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.672684 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2616) took 1191 ms. Trace:
I20260709 13:07:24.699508 21852 rpcz_store.cc:276] 0709 13:07:23.481559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.482667 (+  1108us) service_pool.cc:224] Handling call
0709 13:07:24.672487 (+1189820us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b54003d3454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:24.672617 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221894528}
I20260709 13:07:24.681984 27202 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.683297 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.655756 27121 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.659122 27305 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.696095 21858 rpcz_store.cc:276] 0709 13:07:22.066535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.725704 (+659169us) service_pool.cc:224] Handling call
0709 13:07:22.725970 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.720308 (+994338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.668173 (+947865us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540018fbd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:24.668403 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175393024}
I20260709 13:07:24.685067 27256 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.676095 27200 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:24.693270 27268 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.654367 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.664230 27303 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:24.687042 27064 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.704656 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.689131 27084 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:24.709446 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55160.7us
I20260709 13:07:24.710507 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.713609 21852 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:24.717363 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.718133 21858 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:24.719487 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.721545 21852 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:24.723073 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.723688 21858 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:24.710568 27200 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.714682 26909 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:24.731051 26798 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.732173 22590 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:24.710438 27064 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.732225 26758 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.736534 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:24.737658 27304 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:07:24.738075 21858 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:24.710091 27305 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.740386 27305 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:24.741288 27300 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:07:24.741127 27062 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.743227 27062 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.743395 27297 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.744376 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:24.744388 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.745440 22584 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:24.745528 27285 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:24.745851 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.746821 27285 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:24.747169 27285 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:24.747501 27285 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:24.748095 26758 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.748538 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.748582 26758 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.749925 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.750638 26130 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.750995 26130 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.751515 27272 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:24.751801 26130 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.752028 27272 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:24.752694 27272 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.750981 26758 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.755882 27285 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.760157 27069 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:24.720360 27303 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:24.760193 26130 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.761608 27303 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.763445 26758 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:24.764111 26758 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.766492 26758 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.767050 27272 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:24.769656 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 113976us
W20260709 13:07:24.777477 27302 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:24.701951 27270 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.693050 26697 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:07:24.785887 27127 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:24.706708 27202 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.709107 27256 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.713132 27084 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:24.788082 27294 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
I20260709 13:07:24.717679 27268 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.795364 27084 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:24.794543 27186 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.816495 27268 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.822606 27218 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.824777 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.825697 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.829545 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.830684 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.831017 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.831248 22006 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:24.831837 22009 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:24.832266 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.833078 22006 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:07:24.836895 22004 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:24.834627 26758 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:07:24.838251 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.836274 27086 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:24.840114 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.840760 22003 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:24.840160 27086 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:24.841286 27186 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:24.841766 27186 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:24.841984 27086 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.842671 27186 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.846390 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.847290 22000 raft_consensus.cc:2410] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:07:24.847177 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:24.847806 21857 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.849491 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 806) took 1170 ms. Trace:
I20260709 13:07:24.849887 21855 rpcz_store.cc:276] 0709 13:07:23.678946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.679096 (+   150us) service_pool.cc:224] Handling call
0709 13:07:24.849288 (+1170192us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b540025c454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:24.849431 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208611712}
I20260709 13:07:24.852025 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.852380 21851 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:24.818431 27270 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:24.859697 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.859853 27162 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.860874 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.863131 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.867583 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:24.871133 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:24.874415 22006 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:24.883669 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.884523 22584 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:24.889150 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.890551 22584 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:24.891685 21857 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:24.893817 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.895215 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.897485 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.904268 27218 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:24.906679 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.905686 27102 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.894860 27268 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:24.896581 27305 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:24.923333 22584 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:24.916455 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.915753 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:24.925792 22590 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:24.927441 27060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.013s	sys 0.005s
I20260709 13:07:24.891916 27270 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
W20260709 13:07:24.944818 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59077.5us
I20260709 13:07:24.923368 26798 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.922974 27261 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:24.917899 27218 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:24.951025 22000 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:24.952723 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.947796 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.955586 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.956593 21930 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:24.958138 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.959775 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.967136 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.968008 21930 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:24.968521 21858 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:24.969870 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.969628 27084 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.971997 27084 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.972644 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:24.974215 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.974781 21930 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:24.974901 21927 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:24.975551 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.976105 21930 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:07:24.976142 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.977525 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2612) took 1513 ms. Trace:
I20260709 13:07:24.977906 21859 rpcz_store.cc:276] 0709 13:07:23.464084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.465935 (+  1851us) service_pool.cc:224] Handling call
0709 13:07:23.466192 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.520340 (+ 54148us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:23.748243 (+227903us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.887404 (+139161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.887470 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:23.887531 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.977310 (+1089779us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b540018ecd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:24.977357 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.977471 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":352041,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":523,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":10547,"spinlock_wait_cycles":123889536,"thread_start_us":349528,"threads_started":2,"wal-append.queue_time_us":270715}
W20260709 13:07:24.979892 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.981215 26758 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.981741 26758 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.983507 21930 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:24.984010 26758 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:24.984696 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.985459 21930 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:24.986382 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.987475 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.987932 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.988850 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:24.989032 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.989614 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.935992 27062 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:24.990752 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:24.991346 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.978981 27064 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.974337 27218 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:24.992411 21927 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:24.993347 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:24.990203 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:24.995180 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.995641 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.005575 21930 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:25.007470 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:24.972345 27202 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:24.994489 27309 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.994740 27062 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.002300 27270 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:25.003341 21859 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:25.020944 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.021298 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:24.949164 27026 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:25.004842 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:25.023456 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.018311 21930 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:25.024758 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.930455 27268 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.030196 21927 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:25.073035 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:25.068244 27272 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.017907 27261 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:25.051934 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.080953 21852 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.054207 27256 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.027801 26758 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:07:25.062772 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.065258 26591 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.051108 27270 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.018357 27127 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:25.092752 22593 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:25.015939 27162 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.062813 22004 pending_rounds.cc:85] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 10
I20260709 13:07:25.095822 22004 pending_rounds.cc:107] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:07:25.096207 22004 raft_consensus.cc:2889] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:24.989815 26697 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:07:25.019114 27102 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.071868 21853 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:25.072396 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66213.4us
I20260709 13:07:25.031337 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.025343 22584 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:25.041790 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.024081 27062 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:25.075637 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:25.078421 22590 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:25.145551 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.147177 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.148494 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.048916 27084 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:25.071074 26989 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.097697 21855 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:07:25.083268 27261 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.074872 27219 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.022938 27200 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:25.136951 21852 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:25.080678 27065 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.141513 21860 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:25.079844 21927 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:07:25.153142 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 77576.8us
I20260709 13:07:25.154062 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.154688 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:25.161731 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.162102 21927 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:25.164510 21852 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:25.163295 21859 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:25.171574 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:25.194669 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.156845 27065 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:25.152007 26792 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:25.151377 27069 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:25.262406 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:25.243784 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50405.6us
I20260709 13:07:25.175308 21860 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:25.158533 27261 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.293203 22590 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:25.196416 21852 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:25.163632 27152 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:25.159360 27084 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:25.166008 27297 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:25.309409 26931 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.165351 27060 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:25.167762 26995 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:25.167681 26697 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:25.297231 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 102628us
I20260709 13:07:25.295660 27065 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.159758 26989 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.325695 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.302130 27103 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:25.345579 22585 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:25.346825 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.347131 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.347893 22592 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:25.348026 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.348611 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.309495 27297 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:25.349218 27297 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:25.349349 22592 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:25.349903 27322 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:07:25.350203 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:25.254415 27245 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:07:25.340950 27256 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:25.341334 27321 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:07:25.341941 27219 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.315884 27202 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.324689 27064 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:25.336967 26697 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.320572 27060 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:25.297464 27309 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:25.352941 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:25.354244 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61381.2us
I20260709 13:07:25.338287 27121 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:25.355615 22584 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:25.392573 21853 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:25.372637 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.374039 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.394232 22592 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:25.394630 22591 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 11: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:25.339727 27152 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.376334 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.376713 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2300) took 1201 ms. Trace:
W20260709 13:07:25.351485 27323 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:07:25.248466 27270 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.364292 27103 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:25.383641 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:25.322198 26931 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.384538 27179 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:25.387828 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.321965 26995 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:25.415374 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.393617 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:25.416566 21824 connection.cc:256] RPC call timeout handler was delayed by -1.0303991099999621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 13:07:25.374325 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1204) took 3160 ms. Trace:
I20260709 13:07:25.351747 26798 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:25.396102 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.418071 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.419116 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.419675 22006 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:25.396741 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.421427 22003 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:25.422217 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.352728 27268 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.254196 27127 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:07:25.395826 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.423900 22002 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.396103 22585 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:25.401829 21855 rpcz_store.cc:276] 0709 13:07:24.174928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.850760 (+675832us) service_pool.cc:224] Handling call
0709 13:07:24.851107 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.152281 (+301174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.376410 (+224129us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400016b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:25.376644 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566480256}
I20260709 13:07:25.323904 26909 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:07:25.406692 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.427680 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.401124 27309 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.412145 27202 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.412707 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:25.391251 27060 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.459517 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.460117 22005 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:25.460976 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52275.8us
I20260709 13:07:25.414652 27179 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.425498 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.430579 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.468766 22003 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:25.469969 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.470472 22003 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:25.431736 22009 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:25.431823 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:25.395802 27121 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.471652 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.391464 27102 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:25.417224 22588 rpcz_store.cc:276] 0709 13:07:22.214341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.214475 (+   134us) service_pool.cc:224] Handling call
0709 13:07:22.214820 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.224877 (+ 10057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.224920 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.674167 (+449247us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b18003ffe80. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a
0709 13:07:22.674928 (+   761us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b2c001465b8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:22.675539 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.465828 (+790289us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:23.465877 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.465961 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.372283 (+1906322us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b240031bf98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d720fb0 00007f3b70397488 00007f3b70397be8 00007f3b7036f398 00007f3b7036a76a 00007f3b70369bb6
0709 13:07:25.374113 (+  1830us) raft_consensus.cc:1685] finished
0709 13:07:25.374135 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.374266 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9601,"raft.queue_time_us":11848,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":12578,"spinlock_wait_cycles":32062464,"thread_start_us":789950,"threads_started":1,"wal-append.queue_time_us":862634}
I20260709 13:07:25.439970 21856 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:25.480123 22005 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
W20260709 13:07:25.441176 27314 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.500s	user 0.003s	sys 0.002s
I20260709 13:07:25.443032 27186 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.445523 27062 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.449777 26130 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.450314 26697 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:25.453233 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.454282 22002 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:25.455524 27127 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:25.455308 27152 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.457636 26798 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:25.458354 26931 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.457808 26995 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.395378 27086 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:07:25.463398 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.435626 27261 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.465127 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.405560 26792 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 13:07:25.433508 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45876.3us
I20260709 13:07:25.473135 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.466887 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.474404 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.463951 27202 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.516455 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.472430 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.517745 26909 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:25.518162 26909 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.519043 26909 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.520278 26909 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:25.511821 27086 consensus_queue.cc:1243] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:07:25.522518 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.523088 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.523515 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.523658 27096 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:25.524222 27096 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.524650 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:25.525843 27026 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:25.526233 27026 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.526324 27096 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.526588 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2243) took 1058 ms. Trace:
I20260709 13:07:25.526618 27026 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.526962 21926 rpcz_store.cc:276] 0709 13:07:24.468618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.468785 (+   167us) service_pool.cc:224] Handling call
0709 13:07:25.526384 (+1057599us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:25.526502 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296588672}
I20260709 13:07:25.527868 27309 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.528285 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.528726 26989 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:25.482870 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51148.3us
I20260709 13:07:25.483400 26591 raft_consensus.cc:2764] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:25.488883 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.485702 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.405196 27084 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.466382 27256 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:25.498582 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.425401 27329 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:07:25.472268 27296 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:25.483942 27020 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.507103 21853 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:25.420526 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.637442 22006 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:25.513517 22003 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:25.494364 27200 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:25.530550 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.501427 27127 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:25.579345 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 106333us
I20260709 13:07:25.541066 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2298) took 1366 ms. Trace:
I20260709 13:07:25.550304 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.474504 27095 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:07:25.528971 21926 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:25.531576 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59222.1us
I20260709 13:07:25.550657 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.514817 26995 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:25.523535 26758 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:25.583426 27333 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:07:25.615034 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 63775.9us
I20260709 13:07:25.539208 27094 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:25.616945 21852 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:25.593959 27326 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:07:25.534015 27001 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:25.578447 27069 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:25.617656 26130 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:25.632058 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.536487 27026 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.578668 27086 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:07:25.640558 22003 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:07:25.641697 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.656877 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:25.642431 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.544075 27121 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.643271 22000 raft_consensus.cc:2368] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:25.653972 21932 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:25.665521 26130 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:25.670442 21926 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:25.670889 26130 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:25.671365 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.666149 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:25.671416 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.594939 27179 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.672338 22588 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:25.672343 21926 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:25.673619 21899 leader_election.cc:400] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:25.674108 21899 leader_election.cc:403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:07:25.649117 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 98531us
I20260709 13:07:25.679850 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.653461 22005 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:25.642740 21858 rpcz_store.cc:276] 0709 13:07:24.174406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.738829 (+564423us) service_pool.cc:224] Handling call
0709 13:07:24.739075 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.968958 (+229883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.540767 (+571809us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54003d3e54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:25.540980 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527286656}
W20260709 13:07:25.642093 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 116204us
W20260709 13:07:25.642930 27328 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:07:25.620100 26591 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:07:25.691598 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.671686 26130 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.692291 21853 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:25.641554 27318 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:25.694756 21926 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:25.672960 27179 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:25.696524 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52352.3us
I20260709 13:07:25.631104 27256 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.645219 26758 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:25.683187 27064 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:25.700146 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:25.651858 27026 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.712073 21974 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.657582 27296 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.703847 27329 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:25.647506 27069 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.667279 27202 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.712108 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.690207 22006 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:25.658357 27020 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.724841 27285 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.727097 27261 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.695484 27094 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:25.693715 27338 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:25.738796 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.698292 27095 consensus_queue.cc:1243] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:07:25.731948 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43761.6us
I20260709 13:07:25.732784 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.742456 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.706506 27256 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:25.749835 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2244) took 1267 ms. Trace:
I20260709 13:07:25.736029 22005 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:25.739814 21926 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:25.742514 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.756560 27001 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:25.749898 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:25.758095 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.755434 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:25.733594 27152 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:25.758915 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.759747 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.766006 21927 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.768002 27338 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:25.768388 27069 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:25.768424 27338 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:25.769834 27338 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:25.770614 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.772789 26758 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.775008 21927 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:25.776607 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:25.755227 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.794047 22005 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:25.797037 22002 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:07:25.765072 27347 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:25.705090 27127 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.798591 27127 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.778630 21859 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:25.780164 27296 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:25.780694 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.800511 26591 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:25.800870 27286 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.313s	user 0.004s	sys 0.000s
I20260709 13:07:25.765329 26995 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:25.781811 27297 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.802320 26995 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:25.803292 26995 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:25.803907 27297 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:25.716605 26931 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:25.804162 27338 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.804949 26931 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:25.805338 26931 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.805357 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2619) took 2057 ms. Trace:
I20260709 13:07:25.806231 26931 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.787595 27065 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.788722 27276 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:25.740873 27270 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:07:25.806268 21854 rpcz_store.cc:276] 0709 13:07:23.748010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.748121 (+   111us) service_pool.cc:224] Handling call
0709 13:07:23.748409 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.748702 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.748730 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.755073 (+  6343us) log.cc:844] Serialized 26 byte log entry
0709 13:07:24.115677 (+360604us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:24.115733 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.115819 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.800331 (+1684512us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:25.805062 (+  4731us) raft_consensus.cc:1685] finished
0709 13:07:25.805083 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.805300 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":758,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"spinlock_wait_cycles":9947904,"thread_start_us":321725,"threads_started":1,"wal-append.queue_time_us":672417}
I20260709 13:07:25.804661 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:25.815644 27276 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:25.819135 27065 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.808699 22006 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:25.819715 27020 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:25.816618 27276 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.820569 27276 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:25.820564 21928 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:07:25.821579 22006 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:25.822129 27202 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:25.825146 27297 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:25.826135 27179 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:25.826594 27179 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:25.827486 27179 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:25.811201 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:25.815223 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:25.830632 21926 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:25.830950 21859 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:25.837598 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 144221us
W20260709 13:07:25.842200 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.802268 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.850875 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:25.852303 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:25.854027 21927 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.855487 22005 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:25.830056 27338 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:25.873782 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:25.886482 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.886188 21927 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:25.888684 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:25.889516 21927 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:25.898366 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:25.879673 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.898381 21930 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:25.900499 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:25.911441 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:25.913475 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 111271us
I20260709 13:07:25.914384 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.927517 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:25.912628 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:25.957293 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:25.959266 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:25.871835 27329 consensus_queue.cc:1175] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8f3b35a35257416db3eaaf364cf6b52b" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 13 index: 12 } }
I20260709 13:07:25.896782 27315 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.898006 27305 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:25.904958 26931 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:25.871865 27065 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:07:25.948146 27131 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:25.962970 27020 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:25.945400 26989 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:25.959865 27338 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:07:25.966624 27335 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:07:25.977854 27347 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:25.996614 27060 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 13:07:25.963665 27084 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:25.980320 26591 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:25.972674 21930 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:07:25.978581 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:25.985376 27177 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:26.052271 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 334079us
I20260709 13:07:26.091408 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:26.045818 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 118370us
I20260709 13:07:25.998172 27305 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:25.979540 27315 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.002470 27131 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:26.093807 26130 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.006929 26989 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:26.124477 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:26.130038 21858 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:26.017707 27347 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:26.127149 26130 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.144477 21859 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:07:26.144959 21856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2631) took 2030 ms (client timeout 2199 ms). Trace:
I20260709 13:07:26.145860 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:26.145588 21856 rpcz_store.cc:269] 0709 13:07:24.114178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.258837 (+144659us) service_pool.cc:224] Handling call
0709 13:07:26.144628 (+1885791us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400017a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:26.144875 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705724160}
I20260709 13:07:26.077929 27065 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:26.087728 27343 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:07:26.169246 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:26.132277 27069 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:26.080646 27060 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.178687 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:26.157752 27186 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.003s
W20260709 13:07:26.182533 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.176853 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:26.187529 22586 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:26.188691 26130 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:26.190116 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.153633 27152 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:26.205060 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 292489us
I20260709 13:07:26.174180 27297 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:26.155400 26909 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:07:26.126516 27177 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:26.248296 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.204718 27020 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:26.207919 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:26.366608 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 158768us
I20260709 13:07:26.307413 27348 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:26.200037 27084 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:26.186510 27069 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.279850 21934 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:26.171316 27065 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:07:26.246750 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 247565us
I20260709 13:07:26.085819 27229 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:26.287714 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 940) took 1815 ms (client timeout 1583 ms). Trace:
W20260709 13:07:26.287920 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 118749us
I20260709 13:07:26.251834 27152 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:26.256835 27297 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:26.249923 27131 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.306327 27330 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:26.399102 27330 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:26.393631 27229 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.400005 27330 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.400729 27229 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:26.393151 27001 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:26.287020 26992 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:26.402217 27001 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.404412 27001 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:26.405308 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.405879 27365 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:26.408030 27365 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:26.310961 21926 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:26.323602 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 345101us
I20260709 13:07:26.312734 26995 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.410554 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:26.411689 26992 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:26.413462 27364 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:26.414100 27364 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:26.416374 27355 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:07:26.417268 27362 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:26.417395 27348 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:26.419087 27364 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:26.422431 27364 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:26.356218 21927 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:07:26.358656 27179 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:26.368016 26130 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:26.200469 27186 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:26.250804 27305 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.298923 27347 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.466949 22002 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:26.374189 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:26.375545 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:26.378350 22593 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:26.258034 26989 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:26.389667 21932 rpcz_store.cc:269] 0709 13:07:24.472389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.472644 (+   255us) service_pool.cc:224] Handling call
0709 13:07:26.287531 (+1814887us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540017e154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:26.287634 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670764288}
I20260709 13:07:26.390651 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:26.391487 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.371193 27349 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:07:26.394984 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:26.387873 27084 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:26.401620 27330 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:26.409719 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:26.410400 26995 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:26.419777 27356 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:07:26.421099 27358 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:26.421619 27315 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:26.422784 27060 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:26.424465 27177 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:26.425387 27020 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.523399 27309 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:26.548147 27297 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:26.428879 27127 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.004s
I20260709 13:07:26.369136 27338 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:26.332391 27062 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:26.354115 27268 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:26.550280 27268 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:26.355984 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2245) took 1823 ms. Trace:
I20260709 13:07:26.550474 27338 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.551297 21928 rpcz_store.cc:276] 0709 13:07:24.532592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.532731 (+   139us) service_pool.cc:224] Handling call
0709 13:07:24.533052 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.821909 (+1288857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.355636 (+533727us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400228654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:26.355919 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2960315264}
W20260709 13:07:26.330871 27353 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:07:26.433583 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1031) took 1116 ms. Trace:
I20260709 13:07:26.388774 27069 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:26.384615 27200 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:26.428404 27229 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.436395 26130 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:26.437199 27218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.012s	sys 0.006s
I20260709 13:07:26.506456 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:26.394004 27152 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:26.508462 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.508790 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 118198us
I20260709 13:07:26.579133 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:26.510561 21856 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:26.580571 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:26.512653 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.581797 22002 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:07:26.513288 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2744) took 2263 ms (client timeout 1943 ms). Trace:
W20260709 13:07:26.469522 27256 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.000s
I20260709 13:07:26.439755 27364 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.590598 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:26.509961 27249 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:07:26.511992 27354 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:26.565263 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2632) took 2107 ms. Trace:
W20260709 13:07:26.576649 27218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.012s	sys 0.009s
I20260709 13:07:26.561748 27200 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:26.561685 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 149187us
I20260709 13:07:26.599079 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:26.582839 22001 rpcz_store.cc:269] 0709 13:07:24.241287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.252690 (+ 11403us) service_pool.cc:224] Handling call
0709 13:07:26.504341 (+2251651us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540038b3d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:26.504493 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294542720}
I20260709 13:07:26.585367 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:26.600606 22007 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:26.586128 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:26.575567 27086 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:26.554498 22590 rpcz_store.cc:276] 0709 13:07:25.316786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.316949 (+   163us) service_pool.cc:224] Handling call
0709 13:07:25.337192 (+ 20243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.353447 (+ 16255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.353479 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:25.353980 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:07:26.154400 (+800420us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b180034e2e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:26.264470 (+110070us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:26.264532 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.264653 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:26.432919 (+168266us) raft_consensus.cc:1685] finished
0709 13:07:26.432936 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.433213 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16498,"raft.queue_time_us":16341,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":9286,"spinlock_wait_cycles":9524992,"thread_start_us":109836,"threads_started":1,"wal-append.queue_time_us":1020703}
I20260709 13:07:26.591792 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:26.592523 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:26.594409 26130 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:26.595377 21851 rpcz_store.cc:276] 0709 13:07:24.458076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.854408 (+396332us) service_pool.cc:224] Handling call
0709 13:07:24.854691 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.854879 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:26.343636 (+1488757us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b180040a140. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf
0709 13:07:26.343853 (+   217us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.563729 (+219876us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.564736 (+  1007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.564808 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:26.564878 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.564980 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.565178 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56118,"prepare.queue_time_us":63160,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":46582,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":27023,"spinlock_wait_cycles":4278528,"thread_start_us":226126,"threads_started":3,"wal-append.queue_time_us":235792}
W20260709 13:07:26.568463 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 136705us
W20260709 13:07:26.597672 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 128732us
W20260709 13:07:26.553524 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.602560 21932 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:07:26.575210 27375 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:26.586793 27168 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:26.579866 27152 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:26.701468 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 97999.3us
I20260709 13:07:26.606120 27315 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.703161 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:26.704406 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.705017 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:26.705760 22000 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:07:26.700835 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 97229.7us
I20260709 13:07:26.594976 27309 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:26.597384 27127 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:26.723379 22007 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:26.603020 27200 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:26.602317 27249 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:07:26.763813 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 331650us
I20260709 13:07:26.699669 27176 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.725781 27095 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:07:26.768417 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:26.775799 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:26.595549 27256 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.744500 27309 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.767869 27348 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:26.777451 27172 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:26.731351 27361 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:26.773579 27365 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:07:26.606063 27084 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.785537 27374 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:26.610036 27354 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.950304 26130 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:26.774549 27020 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
W20260709 13:07:26.896466 27276 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.075s	user 0.006s	sys 0.006s
W20260709 13:07:26.966691 27386 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:07:27.046069 27069 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.045921 27095 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:27.044961 27179 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.607s	user 0.001s	sys 0.005s
I20260709 13:07:27.039673 26995 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.142216 27069 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.149151 27354 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.149762 27200 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:26.769667 27375 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:26.949714 27361 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:27.056602 27373 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.515s	user 0.000s	sys 0.001s
W20260709 13:07:27.063861 27376 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:07:27.065208 27364 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.068320 27229 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:26.729027 27378 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.187s
I20260709 13:07:26.887661 27309 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:27.099260 26130 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.051951 27065 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:27.132107 26931 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.123s	user 0.011s	sys 0.003s
I20260709 13:07:27.183895 27200 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.283071 26130 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:27.283670 27376 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:07:27.306393 27256 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.316361 27375 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:27.298897 27384 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.514s	user 0.001s	sys 0.001s
I20260709 13:07:27.314095 27361 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:27.331615 27330 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.789s	user 0.008s	sys 0.006s
I20260709 13:07:27.333583 27172 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:27.392864 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.016s	sys 0.012s
W20260709 13:07:27.393487 21928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.012s	sys 0.012s
W20260709 13:07:27.394433 21928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.013s	sys 0.013s
W20260709 13:07:27.395385 21824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 3.001s (SENT) [suppressed 5 similar messages]
I20260709 13:07:27.398988 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.400323 21933 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:07:27.393903 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.017s	sys 0.012s
I20260709 13:07:27.407475 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2310) took 2413 ms. Trace:
W20260709 13:07:27.407915 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 673388us
I20260709 13:07:27.408174 21860 rpcz_store.cc:276] 0709 13:07:24.993801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.157496 (+163695us) service_pool.cc:224] Handling call
0709 13:07:25.157836 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.292802 (+134966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.406863 (+2114061us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5400259c54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:27.407228 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39698816}
I20260709 13:07:27.411973 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:27.413237 21901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.006s	sys 0.001s
W20260709 13:07:27.414191 21901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.006s	sys 0.002s
W20260709 13:07:27.415602 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.419359 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2742) took 1761 ms. Trace:
I20260709 13:07:27.419948 21926 rpcz_store.cc:276] 0709 13:07:25.657459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.871489 (+214030us) service_pool.cc:224] Handling call
0709 13:07:25.871834 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.409361 (+537527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.418346 (+1008985us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540042a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:27.419055 (+   709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009306368}
I20260709 13:07:27.421932 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:27.423570 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.424631 21926 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:27.426446 21926 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:27.427116 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2747) took 1768 ms. Trace:
I20260709 13:07:27.427767 21927 rpcz_store.cc:276] 0709 13:07:25.658752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.890671 (+231919us) service_pool.cc:224] Handling call
0709 13:07:26.355872 (+465201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.431333 (+ 75461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.426717 (+995384us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400228650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:27.426942 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069024512}
I20260709 13:07:27.420553 21928 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:27.428781 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.751149 21931 rpcz_store.cc:276] 0709 13:07:24.482688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.486683 (+  3995us) service_pool.cc:224] Handling call
0709 13:07:24.487126 (+   443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:24.494895 (+  7769us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:24.685476 (+190581us) log.cc:844] Serialized 14 byte log entry
0709 13:07:24.926445 (+240969us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:25.721544 (+795099us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:25.722085 (+   541us) log.cc:844] Serialized 52 byte log entry
0709 13:07:25.722233 (+   148us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:25.729673 (+  7440us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.729999 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.742058 (+ 12059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.749589 (+  7531us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.749637 (+    48us) raft_consensus.cc:1685] finished
0709 13:07:25.749648 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.749769 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87620,"prepare.queue_time_us":254203,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":85937,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":19997,"spinlock_wait_cycles":23562880,"thread_start_us":277836,"threads_started":2,"wal-append.queue_time_us":221970}
W20260709 13:07:27.430212 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.434456 22006 raft_consensus.cc:2368] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16.
I20260709 13:07:27.435689 26931 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.438436 26931 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:27.459019 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.459950 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.460530 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.461156 26931 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:27.467072 22590 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:27.469559 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:27.475075 22586 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.475212 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.480475 21931 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:27.470373 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.487702 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.488247 26130 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.489567 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:27.489297 26130 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.488776 21932 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:27.497752 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1093) took 2640 ms. Trace:
I20260709 13:07:27.504930 27020 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.506323 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:27.507248 22007 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:27.505997 27020 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.508740 26995 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.510432 26995 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.510761 27026 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:27.514127 27026 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.514894 22000 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:27.518343 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:27.516403 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.519364 22005 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:27.509326 26978 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.498284 22590 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:27.521724 26978 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.498296 22004 rpcz_store.cc:276] 0709 13:07:24.857285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.868288 (+ 11003us) service_pool.cc:224] Handling call
0709 13:07:25.062540 (+194252us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:25.096953 (+ 34413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.096981 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:25.097603 (+   622us) log.cc:844] Serialized 52 byte log entry
0709 13:07:25.249346 (+151743us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:25.249413 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.249555 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.647355 (+397800us) raft_consensus.cc:1685] finished
0709 13:07:25.647377 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.497185 (+1849808us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54003890d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:27.497523 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":32360,"spinlock_wait_cycles":445761408,"thread_start_us":118951,"threads_started":1,"wal-append.queue_time_us":309983,"wal-append.run_cpu_time_us":2712,"wal-append.run_wall_time_us":1287482}
I20260709 13:07:27.526868 26978 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.527179 27026 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.528409 26130 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:27.529277 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.002s	sys 0.006s
I20260709 13:07:27.530232 26995 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.531394 27396 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:07:27.555042 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.003s	sys 0.006s
I20260709 13:07:27.512725 27020 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.565547 27095 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.565909 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:27.566670 27095 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:27.569977 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.640021 22588 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:07:27.641261 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:27.641673 26591 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.556s	user 0.015s	sys 0.012s
I20260709 13:07:27.645455 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:27.645824 22586 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:27.638180 27095 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.646457 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:27.646993 22000 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:27.647585 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:24.873545 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.648784 27256 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.649358 27179 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.650060 27249 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:27.649571 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 74370.4us
I20260709 13:07:27.650969 27176 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.653295 22007 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:27.655522 27360 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:27.656221 27338 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:27.657593 27365 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.658315 27276 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:27.659752 27375 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.660312 27026 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.660575 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.661098 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:27.661274 27340 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.661754 27084 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.681641 22004 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:27.656260 27348 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.705152 22589 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:27.692384 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.693414 21931 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:07:27.697924 27292 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 3.096s	user 0.005s	sys 0.019s
I20260709 13:07:27.698076 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.698642 22593 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:27.699102 27396 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:27.701130 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:27.699137 27276 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.704419 27315 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.704564 26697 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.705137 27245 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.706239 26978 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:27.710927 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:27.712548 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.713203 27172 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:27.716588 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:27.691061 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:27.717806 27249 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.720469 27338 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.719300 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 949) took 2206 ms. Trace:
W20260709 13:07:27.704823 27152 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.930s	user 0.003s	sys 0.015s
I20260709 13:07:27.721038 22585 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:27.690989 26591 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:27.716632 27168 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.972s	user 0.019s	sys 0.014s
I20260709 13:07:27.716774 27179 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.758810 21929 raft_consensus.cc:2410] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:27.759707 27096 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.760377 27179 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:27.760248 27096 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.760895 27338 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.716598 27340 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.763586 27340 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.733417 22587 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:27.734882 27084 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.766404 27084 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.735925 21928 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:27.768204 27309 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.736671 27200 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:27.773280 27348 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.776192 22003 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:27.776530 27348 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.737949 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:27.737603 27172 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.738540 27026 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:27.733453 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.728245 21933 rpcz_store.cc:276] 0709 13:07:25.508033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.816255 (+308222us) service_pool.cc:224] Handling call
0709 13:07:25.816572 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.407643 (+1591071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.690138 (+282495us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000983d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:27.714544 (+ 24406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3639471488}
I20260709 13:07:27.737962 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.733134 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2722) took 4353 ms. Trace:
W20260709 13:07:27.770664 27357 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.794s	user 0.020s	sys 0.095s
I20260709 13:07:27.736411 26697 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.778807 27309 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:27.779202 27276 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.779727 27315 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.780534 26758 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:27.737401 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:27.788254 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60460us
I20260709 13:07:27.718659 27396 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:27.793895 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2771) took 1203 ms. Trace:
I20260709 13:07:27.793982 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2322) took 2085 ms. Trace:
I20260709 13:07:27.794566 27249 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:27.794497 21853 rpcz_store.cc:276] 0709 13:07:25.708213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.708326 (+   113us) service_pool.cc:224] Handling call
0709 13:07:25.708640 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.761099 (+ 52459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.761153 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.761209 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:25.761259 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.781606 (+2020347us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b5400259754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:27.781679 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.793745 (+ 12066us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120055680}
I20260709 13:07:27.795469 27315 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.721400 27360 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.796659 27340 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.797390 27340 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.797544 27064 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:27.797958 27360 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.724460 27256 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.799093 26697 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.799666 27340 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.800451 27064 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:27.802405 21931 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:27.794405 22002 rpcz_store.cc:276] 0709 13:07:26.587084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.737539 (+1150455us) service_pool.cc:224] Handling call
0709 13:07:27.790670 (+ 53131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:27.801235 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.815723 27396 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.816902 22002 pending_rounds.cc:85] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 8
I20260709 13:07:27.802596 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:27.818186 22000 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:27.817365 22002 pending_rounds.cc:107] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:07:27.813827 21860 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:27.817200 27396 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.820232 21928 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:27.800763 27256 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.836954 22005 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:27.837674 27069 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.819832 22002 raft_consensus.cc:2889] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:27.822692 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.006s	sys 0.010s
I20260709 13:07:27.844574 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:27.845537 21853 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:27.800906 27064 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:27.849742 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:27.849756 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:27.802690 27318 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:27.851414 22004 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:27.744976 27176 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.853762 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:27.854925 21859 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:27.855216 27176 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:27.855875 27176 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:27.856346 27176 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:27.801622 27270 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:07:27.850759 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.006s	sys 0.011s
I20260709 13:07:27.857195 22007 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:27.819862 27249 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:07:27.858392 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.15602e+06us
I20260709 13:07:27.858973 27249 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:27.849419 27095 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.860047 27095 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.860272 27249 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:27.862411 27095 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.862566 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:27.865535 27176 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.789132 27152 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.869319 27069 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.871230 27152 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.871661 27069 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:27.872028 27020 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.872628 27020 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.873304 27026 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.874359 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.874794 27020 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:27.875746 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66777.3us
I20260709 13:07:27.876024 22005 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:27.876407 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:27.852782 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50319.4us
I20260709 13:07:27.875937 21824 leader_election.cc:400] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16.
I20260709 13:07:27.879268 21824 leader_election.cc:403] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:07:27.801580 27172 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.883527 27249 raft_consensus.cc:2764] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:27.886965 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:27.802091 27086 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:07:27.801863 27363 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:07:27.847996 27179 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.801301 27365 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.801095 27245 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.900374 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.901126 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:27.902014 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:27.902366 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:27.783852 21935 rpcz_store.cc:276] 0709 13:07:23.379252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.379407 (+   155us) service_pool.cc:224] Handling call
0709 13:07:23.402657 (+ 23250us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:07:23.772147 (+369490us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:23.970864 (+198717us) log.cc:844] Serialized 14 byte log entry
0709 13:07:24.217670 (+246806us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:07:24.218634 (+   964us) log.cc:844] Serialized 104 byte log entry
0709 13:07:24.218816 (+   182us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:24.219099 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:07:24.219350 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.223587 (+  4237us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:26.261716 (+2038129us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180024da20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:27.648159 (+1386443us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:27.732597 (+ 84438us) raft_consensus.cc:1685] finished
0709 13:07:27.732617 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.732896 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74397,"prepare.queue_time_us":276170,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":4583,"raft.queue_time_us":37759,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":79506,"spinlock_wait_cycles":30870016,"thread_start_us":670140,"threads_started":3,"wal-append.queue_time_us":471937}
I20260709 13:07:27.911031 21859 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:27.875945 27096 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.912863 22005 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:27.913121 21926 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:27.818380 27375 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:27.915518 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.917775 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.926698 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.894163 27152 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:07:27.912820 27256 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.931478 21851 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:07:27.933571 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57864.3us
I20260709 13:07:27.914903 27338 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:27.923035 27086 consensus_queue.cc:1243] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:27.951004 27410 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.951782 27410 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.957104 26798 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.958016 26798 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.960328 26798 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.961108 27064 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:27.964301 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 218438us
W20260709 13:07:27.965150 21898 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.652s (SENT)
I20260709 13:07:27.965523 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:27.935288 27318 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.967649 27405 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.968402 27348 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.969280 27348 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:27.971727 27348 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.914744 27249 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:27.972851 27249 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.968426 27318 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.978503 22557 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:27.978824 27245 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:27.979446 27245 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:27.979622 21853 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:27.980764 27256 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.982366 27338 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.923463 27179 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:27.987277 27179 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:27.988303 22005 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:27.991730 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:27.997380 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.000347 22005 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:28.001437 27064 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:28.034809 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56472.5us
I20260709 13:07:28.036011 27134 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:28.003952 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.037483 27338 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.037889 21927 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:28.038977 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.039609 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:28.005522 21935 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:07:28.040607 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.006734 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:28.011621 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46557.7us
I20260709 13:07:28.015192 27249 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.017300 27375 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.017743 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:28.020126 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 119203us
I20260709 13:07:28.022502 26909 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:28.058748 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.059926 22587 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:28.068878 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.069954 22585 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:28.070869 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.071996 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.072645 21933 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:28.073865 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:28.083384 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.110713 21931 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:28.111227 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.113160 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.106468 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:28.110666 27305 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:28.116139 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.108841 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:28.118153 22004 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:28.117759 27305 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.119839 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.120756 21855 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:28.121573 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.122095 27305 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.122439 21855 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:07:28.122681 27131 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:28.126098 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 56762us
I20260709 13:07:28.126900 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:28.127630 21855 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.124150 22585 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:28.132359 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:28.113898 27360 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.136392 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:28.136948 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 92272.2us
I20260709 13:07:28.138428 21926 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:28.118708 27245 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:28.115212 27315 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:28.151867 21855 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:28.126132 27131 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.142788 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:28.154382 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:28.163971 22004 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:28.155042 27096 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:28.157301 22589 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:28.159036 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2773) took 1552 ms. Trace:
I20260709 13:07:28.133636 27179 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:28.169426 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:28.160362 27408 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.004s
W20260709 13:07:28.160650 27407 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.116s	user 0.003s	sys 0.000s
W20260709 13:07:28.168875 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 233271us
I20260709 13:07:28.155663 27270 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:28.159865 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.143116 27338 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:28.165592 22001 rpcz_store.cc:276] 0709 13:07:26.603943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.608306 (+  4363us) service_pool.cc:224] Handling call
0709 13:07:26.608593 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.608778 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:26.609035 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.736196 (+1127161us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b1800193ea0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:28.110834 (+374638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.123089 (+ 12255us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:28.136271 (+ 13182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.143218 (+  6947us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.156473 (+ 13255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116132,"prepare.queue_time_us":200,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":5178,"raft.queue_time_us":175380,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4001,"spinlock_wait_cycles":3057408,"thread_start_us":179875,"threads_started":1,"wal-append.queue_time_us":1383929}
I20260709 13:07:28.181980 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.179167 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:28.184069 22005 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:28.184211 27086 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.184909 27086 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.187320 27086 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.191497 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:28.194213 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.196936 27405 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16.
I20260709 13:07:28.197192 27131 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.197767 26798 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:28.198197 26798 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:28.198576 26798 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:28.183092 27338 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:28.201383 27338 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:28.205372 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88390.1us
I20260709 13:07:28.206388 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:28.171916 21855 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:28.169667 27179 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.205433 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:28.180030 27270 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.215113 22004 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:07:28.216372 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 81712us
I20260709 13:07:28.215245 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.143865 27360 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.180567 27096 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:28.193888 27084 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:28.214305 27179 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:28.176385 27245 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.227782 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:28.231935 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:28.233227 27095 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:28.233657 27095 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.234031 27095 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.234624 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.236003 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.236622 21859 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:28.219967 27270 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:28.224543 27396 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.188372 27315 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.237223 27152 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:28.239521 22591 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:28.219699 26909 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:28.230057 27131 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.247627 27094 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 13:07:28.251765 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45513.9us
I20260709 13:07:28.245805 27396 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.224821 27084 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.252396 26989 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
W20260709 13:07:28.260968 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.262506 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:28.237972 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:28.270196 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 820) took 2807 ms (client timeout 2547 ms). Trace:
I20260709 13:07:28.254571 27152 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.263494 22004 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:07:28.256597 26909 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:28.280330 21859 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:28.258224 27131 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.267891 27396 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.283825 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 942) took 3753 ms. Trace:
W20260709 13:07:28.280257 21852 rpcz_store.cc:269] 0709 13:07:25.462957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.463899 (+   942us) service_pool.cc:224] Handling call
0709 13:07:28.269806 (+2805907us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540025a654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:28.270001 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232446208}
I20260709 13:07:28.284906 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.282372 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.278786 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.291802 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.292524 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.292678 21935 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:28.292932 21856 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:28.293565 27096 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.294828 27315 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.303807 21853 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:28.302428 21934 rpcz_store.cc:276] 0709 13:07:24.530313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.530504 (+   191us) service_pool.cc:224] Handling call
0709 13:07:25.526685 (+996181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.387941 (+861256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.738395 (+1350454us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:28.283444 (+545049us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400206550. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:28.283641 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3864295168}
I20260709 13:07:28.295603 27360 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.281574 27094 consensus_queue.cc:1243] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:28.296751 26792 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:07:28.274115 27084 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:28.316565 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.324580 21933 raft_consensus.cc:2410] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:07:28.316550 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:28.285514 27152 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:28.290287 26909 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:28.340763 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.344579 21935 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:28.315809 27405 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.320864 26130 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:28.301564 27131 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.301375 27276 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.329170 26792 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:28.330302 27363 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:28.359596 27121 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:28.331858 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.333289 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:28.337579 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.350028 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.353917 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:28.355360 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 130398us
I20260709 13:07:28.330787 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.363737 21927 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:28.364921 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.365541 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:28.351274 27405 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.383138 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2326) took 2297 ms. Trace:
I20260709 13:07:28.387066 21858 rpcz_store.cc:276] 0709 13:07:26.008914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.009102 (+   188us) service_pool.cc:224] Handling call
0709 13:07:26.085673 (+ 76571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:26.137893 (+ 52220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.259731 (+2121838us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b540025a154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:28.306035 (+ 46304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101264768}
W20260709 13:07:28.391959 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.282357 27338 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.401567 21859 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:28.377128 27405 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:28.404920 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 120264us
W20260709 13:07:28.438874 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.448307 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:28.449429 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65094.8us
I20260709 13:07:28.449644 26989 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:28.450275 26989 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:28.450376 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:28.451529 26989 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.452508 27064 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:28.368000 27276 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.465744 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:28.469048 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67218.7us
I20260709 13:07:28.494812 21853 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:28.471446 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:28.458279 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.488452 22586 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:28.496073 21852 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:28.461145 27402 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:28.549147 27094 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:28.555455 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 97270.5us
I20260709 13:07:28.562773 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.575196 26130 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:28.442121 27363 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.154s
I20260709 13:07:28.452970 27064 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.586546 27063 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:28.586823 27202 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:28.555994 26591 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:28.557392 27402 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.441928 27095 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.597608 27410 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.577266 27094 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.470566 27375 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.118s
W20260709 13:07:28.607424 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 274291us
I20260709 13:07:28.599098 27064 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:28.603925 27245 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.599344 27063 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:28.605724 27402 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:28.616828 27410 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:28.618659 27375 consensus_queue.cc:1243] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:28.644068 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:28.648198 21852 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:28.628283 26591 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.658704 27179 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.662122 27276 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.664502 26798 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.674655 27094 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:28.675249 27377 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.959s	user 0.006s	sys 0.020s
I20260709 13:07:28.676041 26130 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:28.675791 27063 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:28.645692 26758 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.004s
I20260709 13:07:28.640476 27410 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.693925 27095 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.677685 26591 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.711911 26591 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.712067 27095 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:28.715750 27270 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.724913 26591 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.701498 27375 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.726536 27375 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.704099 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2779) took 1147 ms. Trace:
I20260709 13:07:28.703927 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.727770 22006 rpcz_store.cc:276] 0709 13:07:27.574334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.647408 (+ 73074us) service_pool.cc:224] Handling call
0709 13:07:27.647920 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.036749 (+388829us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:28.039793 (+  3044us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.178841 (+139048us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.179695 (+   854us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.189522 (+  9827us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.193157 (+  3635us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:28.193207 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.703713 (+510506us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54002dafd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:28.703765 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.703888 (+   123us) raft_consensus.cc:1685] finished
0709 13:07:28.703902 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.704035 (+   133us) inbound_call.cc:177] Queueing success response
0709 13:07:28.722548 (+ 18513us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800354580. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f
0709 13:07:28.724344 (+  1796us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4800130158. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
Metrics: {"prepare.queue_time_us":1399,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":1885,"raft.run_cpu_time_us":4539,"raft.run_wall_time_us":545602,"spinlock_wait_cycles":1026877824,"thread_start_us":137404,"threads_started":1,"wal-append.queue_time_us":167393}
I20260709 13:07:28.729631 27399 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:28.711388 21855 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:28.732152 27375 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.725824 26798 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.730224 27399 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:28.712656 27095 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:28.699812 27405 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:28.712133 27427 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:07:28.733603 27410 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:28.750396 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 187778us
W20260709 13:07:28.751870 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.752475 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.753481 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.753921 22591 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:28.754127 22587 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.754281 22589 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:28.756500 27176 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:28.757225 27176 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:28.758484 27176 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:28.761487 26758 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.761897 26758 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.762255 26758 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.764367 27363 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.764678 22587 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:28.764957 27363 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.766450 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:28.767275 27363 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.767460 27176 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:28.768733 27176 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:28.770373 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.770059 27176 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.772588 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.773272 22590 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:28.774386 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.775086 22584 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:28.775218 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.771456 26758 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.771905 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.775936 22590 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:28.772181 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.778008 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.778715 22585 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:28.778801 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.779109 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.779536 22584 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:28.779769 27086 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.780025 22590 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:28.779788 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.780556 27086 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.780849 27363 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.782828 26758 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.782912 27086 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.782398 27363 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.783788 27375 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.784209 27375 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.785169 27363 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.786612 27375 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.787078 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.787598 22591 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.787837 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.789098 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.789546 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.791193 27375 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:28.791733 27375 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.791970 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.792341 27375 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:28.793108 27363 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:28.793227 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.793531 27363 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:28.793924 27363 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.795264 21825 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:28.796027 27086 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:28.796396 27086 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:28.796744 27086 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:28.802116 27363 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.803660 27375 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.805168 26758 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:28.804966 27086 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.807307 27375 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.808735 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.809448 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.809454 22590 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:28.810153 22585 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:28.807734 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.813282 22584 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.817147 22585 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:28.817312 27371 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.818087 27371 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.818825 27249 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.819404 27249 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.820583 27371 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:28.821314 27249 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:28.821899 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.822686 26758 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:28.823184 26758 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:28.824671 22584 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:28.824450 26758 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.804963 27363 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.807361 27086 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.828059 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.825647 22590 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:28.830528 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.831516 27363 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:28.831991 27363 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:28.833105 22591 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:28.832793 27363 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.835139 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:28.836159 27375 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:28.836829 27375 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:28.838093 27375 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.743929 27095 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:28.840052 27086 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:28.847249 27086 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:28.743676 27338 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.749298 27063 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:28.865630 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.870332 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.891868 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.868016 27249 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.882850 26792 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:28.895421 27062 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.904151 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.906704 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.908382 22584 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:28.924130 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.909730 22585 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:28.909888 27405 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:28.909427 27399 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.864639 27086 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.911763 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:28.912964 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 288020us
I20260709 13:07:28.862313 27268 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:28.933298 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.934926 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:28.935609 22590 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:28.936859 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.938011 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.944489 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:07:28.959233 27396 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.002s
I20260709 13:07:28.959707 27396 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.945588 27268 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:28.962486 22008 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:28.962935 22008 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:28.964110 22008 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:28.946458 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:28.947861 27402 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:28.948446 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.889766 27360 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:07:28.985836 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.967141 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:28.982236 22005 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:28.982944 26909 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.005s
W20260709 13:07:29.008466 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 128637us
I20260709 13:07:29.008955 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:29.012494 21859 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.012746 22008 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.941172 27179 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:28.902740 27249 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:29.027333 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80954us
I20260709 13:07:29.004477 27276 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:29.039170 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:29.017351 27152 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.001s
I20260709 13:07:29.014171 27444 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:28.940234 27062 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.040992 22590 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:29.021091 26758 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.020970 27094 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.050508 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.054108 21898 leader_election.cc:400] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:29.054505 21898 leader_election.cc:403] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:07:29.054302 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.038125 26591 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.051694 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:29.048326 27261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.086s	user 0.049s	sys 0.016s
I20260709 13:07:29.043180 27444 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:29.019256 26909 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.072785 27162 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:29.073405 27162 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.073817 27162 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.074225 27062 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:29.074970 27062 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:29.076057 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2646) took 1172 ms. Trace:
I20260709 13:07:29.076486 21858 rpcz_store.cc:276] 0709 13:07:27.909836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.469899 (+560063us) service_pool.cc:224] Handling call
0709 13:07:28.470323 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.576149 (+105826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.582559 (+  6410us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.615938 (+ 33379us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.890145 (+274207us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:28.890178 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.890252 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.075790 (+185538us) raft_consensus.cc:1685] finished
0709 13:07:29.075813 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.075964 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":241512,"threads_started":1,"wal-append.queue_time_us":380710}
W20260709 13:07:29.064697 27261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.286s	user 0.050s	sys 0.017s
I20260709 13:07:29.082217 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:29.078513 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.092865 22590 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:29.043642 27276 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.043265 27152 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.095513 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:29.096434 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:29.101938 22590 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:29.104005 22590 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:29.065269 27176 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:29.106384 27249 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:29.069468 27444 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.115211 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:29.092130 27131 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
I20260709 13:07:29.119747 27060 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:07:29.120141 27410 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.125780 27176 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:29.126331 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.127044 27176 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:29.129936 27445 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:07:29.127669 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.055529 27245 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:07:29.116815 27131 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.121102 27060 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:29.128923 27433 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:07:29.066100 26758 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.157305 22584 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:29.157677 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 62250.4us
I20260709 13:07:29.134020 27396 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:29.152567 27062 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:29.170313 22008 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:07:29.170838 22008 pending_rounds.cc:85] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 6
I20260709 13:07:29.171231 22008 pending_rounds.cc:107] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:07:29.171641 22008 raft_consensus.cc:2889] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:29.161124 27121 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.791s	user 0.005s	sys 0.010s
I20260709 13:07:29.189493 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.152689 27060 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:29.167167 27084 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.813s	user 0.002s	sys 0.014s
I20260709 13:07:29.192325 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.198789 27276 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:29.205715 27441 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:29.206979 27410 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:29.191838 27172 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:29.211196 27329 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.211380 27172 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.212004 27026 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:29.212467 27026 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:29.213948 27172 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:29.214795 22000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2778) took 1698 ms (client timeout 2157 ms). Trace:
W20260709 13:07:29.215190 22000 rpcz_store.cc:269] 0709 13:07:27.516836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.518011 (+  1175us) service_pool.cc:224] Handling call
0709 13:07:29.214591 (+1696580us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b540038b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.214700 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197930880}
I20260709 13:07:29.208249 26931 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:29.217605 26931 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.219286 27270 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.220005 26931 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:29.222620 27400 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.465s	user 0.010s	sys 0.000s
I20260709 13:07:29.224704 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 829) took 1440 ms. Trace:
I20260709 13:07:29.225139 21856 rpcz_store.cc:276] 0709 13:07:27.784215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.796140 (+ 11925us) service_pool.cc:224] Handling call
0709 13:07:28.292405 (+496265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.305237 (+ 12832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.224503 (+919266us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540025a150. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.224647 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136169856}
I20260709 13:07:29.225847 27131 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:29.226389 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.226758 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.227392 22587 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.169422 26591 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.189447 27402 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:29.198813 27440 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:29.280027 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2765) took 3846 ms. Trace:
I20260709 13:07:29.280458 22008 rpcz_store.cc:276] 0709 13:07:25.434119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.662956 (+228837us) service_pool.cc:224] Handling call
0709 13:07:25.663282 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.962204 (+3298922us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b54001a6e20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:07:29.263040 (+300836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.279970 (+ 16930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178264832}
I20260709 13:07:29.261675 26591 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:29.220017 27270 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.282519 27270 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.283671 26591 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:29.237556 26758 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:29.285204 27152 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.284691 27270 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:29.211974 27329 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:29.289078 27331 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:07:29.210215 27084 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.294633 22009 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:29.295272 22587 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:29.296114 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.296706 22589 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:29.301738 27451 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:07:29.292768 27084 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.310058 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2645) took 1407 ms. Trace:
I20260709 13:07:29.310495 21859 rpcz_store.cc:276] 0709 13:07:27.903064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.447801 (+544737us) service_pool.cc:224] Handling call
0709 13:07:28.448201 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.016835 (+568634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.309810 (+292975us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b540010ca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:29.309997 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1296960256}
I20260709 13:07:29.311698 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.312418 21859 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:29.319250 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.008s	sys 0.012s
W20260709 13:07:29.319643 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.008s	sys 0.012s
W20260709 13:07:29.319916 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 854178us
I20260709 13:07:29.326534 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2666) took 1117 ms. Trace:
I20260709 13:07:29.329109 22591 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:29.338002 22589 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
W20260709 13:07:29.229758 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 175770us
I20260709 13:07:29.274055 27444 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.276639 27162 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.238977 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.327046 21855 rpcz_store.cc:276] 0709 13:07:28.209026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.214968 (+  5942us) service_pool.cc:224] Handling call
0709 13:07:29.326278 (+1111310us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54002b9b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.326445 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133960064}
I20260709 13:07:29.328378 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:29.340859 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:29.343750 21858 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:07:29.348065 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 966) took 1688 ms. Trace:
I20260709 13:07:29.348096 22007 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:07:29.348626 27094 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:29.348869 22007 pending_rounds.cc:85] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 10
I20260709 13:07:29.349313 22007 pending_rounds.cc:107] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:07:29.349627 22007 raft_consensus.cc:2889] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.350629 27444 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.351158 27162 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:29.351444 27064 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.001s
I20260709 13:07:29.352151 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.348783 21932 rpcz_store.cc:276] 0709 13:07:27.659370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.346979 (+1687609us) service_pool.cc:224] Handling call
0709 13:07:29.347490 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.347691 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.347729 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:29.347789 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:29.347819 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.347882 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.348001 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.353713 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.355027 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.356595 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.357338 21859 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:29.358011 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.358325 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.358986 21859 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:29.359428 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.360857 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.361500 21855 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.366972 21859 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:29.368175 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.368811 21859 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:29.369673 21926 pending_rounds.cc:85] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 7
I20260709 13:07:29.370251 21926 pending_rounds.cc:107] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:07:29.370672 21926 raft_consensus.cc:2889] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.371330 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.372072 21859 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:29.372586 27127 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.373211 27127 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.373556 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:29.374547 27315 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:29.374989 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.375077 27315 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.375715 27315 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:29.376901 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.377650 21935 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:29.378681 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.379854 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.380723 21935 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:29.380962 21927 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:07:29.381440 21927 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 8
I20260709 13:07:29.381728 21927 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:07:29.381779 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.382110 21927 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.383476 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.383606 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.384197 21935 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:29.351933 27064 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.385241 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.386544 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.386857 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.388722 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.389590 21934 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:29.389829 27189 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:29.390079 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.390489 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.390683 27189 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:29.390952 26130 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:29.391408 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.392117 21859 raft_consensus.cc:2376] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12. Re-sending same reply.
I20260709 13:07:29.387228 27064 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:29.387761 26909 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.387702 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:29.326613 27454 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:07:29.391139 21934 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:29.384912 27315 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.388754 21935 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.392606 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.397810 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:29.399276 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71015.3us
I20260709 13:07:29.399650 21855 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:29.417810 22589 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:29.404173 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.419194 21858 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:29.209789 27396 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.189s
I20260709 13:07:29.425357 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.424796 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.424542 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.426543 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.427892 21859 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:29.428606 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.440873 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:29.252300 27276 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.194s
I20260709 13:07:29.442567 26909 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:29.442916 27062 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:29.443475 27062 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:29.443924 26909 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:29.444314 26909 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:29.444428 27062 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.407126 26931 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:29.437525 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:29.446817 27121 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:29.447319 27121 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:29.448199 27121 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.393671 27095 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.407732 27152 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:29.438251 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.463858 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.455523 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.467249 21934 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:29.467473 21932 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:29.469444 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:29.470187 21935 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:29.471493 22589 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:07:29.471622 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 305778us
I20260709 13:07:29.474952 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:29.476054 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.478411 27399 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:29.480487 22584 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:29.469664 27152 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.484771 22587 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:29.475430 27189 raft_consensus.cc:2764] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:29.465376 26931 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:29.493564 21855 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:29.504045 21927 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:29.476261 27270 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:29.507169 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.510674 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.510167 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:29.512540 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.502962 27152 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.514110 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.515116 21927 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:29.516685 27084 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:29.517845 27084 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.521524 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:29.524215 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.501649 27399 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:29.530701 21927 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.500797 27268 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 13:07:29.511986 27202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.005s	sys 0.008s
I20260709 13:07:29.506472 27270 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:29.542176 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.544328 21935 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:29.471277 26909 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.532104 27084 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:29.539583 27179 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:29.551883 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.548909 26909 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:29.544562 27202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.006s	sys 0.008s
I20260709 13:07:29.533531 27444 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:29.558247 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.556774 21859 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:29.511969 27245 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:07:29.545615 27270 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.563321 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.565043 21935 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:07:29.565475 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 117052us
I20260709 13:07:29.551666 27315 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.570366 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.558837 27202 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:29.583088 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.584511 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.569557 27095 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.585551 21859 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:29.586815 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.588414 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.566236 27444 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.561321 27179 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.599761 27179 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.599295 27152 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.604094 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.616711 21858 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
W20260709 13:07:29.617852 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 60943.8us
I20260709 13:07:29.619874 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.592869 27444 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.621196 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.623267 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.623658 21858 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 9.
I20260709 13:07:29.622341 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.625691 21859 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:29.623807 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:29.621062 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.620731 21927 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:29.629945 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.629029 21927 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:29.634137 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:29.578292 26931 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.635553 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.636569 21932 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:29.637404 21855 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:07:29.642474 22009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 2756) took 4245 ms (client timeout 3001 ms). Trace:
W20260709 13:07:29.642916 22009 rpcz_store.cc:269] 0709 13:07:25.397444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.522827 (+125383us) service_pool.cc:224] Handling call
0709 13:07:29.642226 (+4119399us) spinlock_profiling.cc:243] Waited 3.77 s on lock 0x7b54001982d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.642385 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8671086848}
I20260709 13:07:29.643846 27296 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:29.645511 27152 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:29.647193 21859 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:29.647902 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2653) took 1632 ms. Trace:
I20260709 13:07:29.648202 21859 rpcz_store.cc:276] 0709 13:07:28.015417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.638976 (+1623559us) service_pool.cc:224] Handling call
0709 13:07:29.647025 (+  8049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.647631 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.647820 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.633368 27457 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:29.649785 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2655) took 1633 ms. Trace:
I20260709 13:07:29.624387 27462 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:29.650424 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1115) took 1800 ms. Trace:
I20260709 13:07:29.650835 22007 rpcz_store.cc:276] 0709 13:07:27.850334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.896817 (+ 46483us) service_pool.cc:224] Handling call
0709 13:07:27.921170 (+ 24353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.350155 (+1428985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.650224 (+300069us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540042bdd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:29.650369 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3280584448}
I20260709 13:07:29.650138 21859 rpcz_store.cc:276] 0709 13:07:28.016366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.648940 (+1632574us) service_pool.cc:224] Handling call
0709 13:07:29.649268 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.649459 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.649490 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:29.649534 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:29.649564 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.649615 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.649710 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.653951 21935 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:29.654879 21934 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:29.653877 21927 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.656415 21859 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:29.659318 21858 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:29.660499 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2656) took 1643 ms. Trace:
I20260709 13:07:29.660921 21858 rpcz_store.cc:276] 0709 13:07:28.016742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.651092 (+1634350us) service_pool.cc:224] Handling call
0709 13:07:29.651374 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.660089 (+  8715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.660443 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.662119 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.662636 21858 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:07:29.664219 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45335.8us
I20260709 13:07:29.663475 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2657) took 1646 ms. Trace:
I20260709 13:07:29.664788 21859 rpcz_store.cc:276] 0709 13:07:28.017115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.653048 (+1635933us) service_pool.cc:224] Handling call
0709 13:07:29.653918 (+   870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.663196 (+  9278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.663419 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.667960 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.601930 27399 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:29.670060 21859 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:29.672462 21934 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:29.673141 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2662) took 1456 ms. Trace:
I20260709 13:07:29.673830 21858 rpcz_store.cc:276] 0709 13:07:28.207690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.661985 (+1454295us) service_pool.cc:224] Handling call
0709 13:07:29.662300 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.664346 (+  2046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.664547 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.675244 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.675417 21927 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:29.684638 27457 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:29.689746 27457 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:29.685902 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.695281 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2665) took 1462 ms. Trace:
I20260709 13:07:29.695946 21859 rpcz_store.cc:276] 0709 13:07:28.208701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.665721 (+1457020us) service_pool.cc:224] Handling call
0709 13:07:29.666014 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.671143 (+  5129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.671403 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.700657 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1051) took 1622 ms. Trace:
I20260709 13:07:29.701159 22586 rpcz_store.cc:276] 0709 13:07:28.068695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.070573 (+  1878us) service_pool.cc:224] Handling call
0709 13:07:29.690869 (+1620296us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54005325d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.691011 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320456448}
I20260709 13:07:29.710975 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.717259 21859 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:29.720942 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.633148 27202 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.715436 26792 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:29.687851 27245 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:07:29.723227 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.755122 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.755699 21859 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.755941 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.760910 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.768174 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:29.769152 22008 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:29.771493 22000 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:29.774158 21934 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.780087 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.780174 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.772903 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.782809 21859 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.788475 22009 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:29.792052 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.793689 22000 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:07:29.795138 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.795930 22006 raft_consensus.cc:2368] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:29.795138 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.798709 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.799456 21931 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:29.802568 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.751698 26792 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:29.807967 22004 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:29.773264 27152 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:29.811436 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.818993 22008 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:29.819444 21931 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:29.824344 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 828) took 2115 ms (client timeout 1998 ms). Trace:
I20260709 13:07:29.824796 21851 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:29.824800 21860 rpcz_store.cc:269] 0709 13:07:27.714612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.716190 (+  1578us) service_pool.cc:224] Handling call
0709 13:07:29.824157 (+2107967us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b540018e554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.824264 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169053440}
I20260709 13:07:29.826141 21859 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:07:29.827195 21852 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.831033 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.832679 21928 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:29.835081 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2775) took 3098 ms. Trace:
I20260709 13:07:29.810778 27245 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:29.837532 21851 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:29.838738 26792 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:29.835491 22003 rpcz_store.cc:276] 0709 13:07:26.736711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.736879 (+   168us) service_pool.cc:224] Handling call
0709 13:07:26.737317 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.776669 (+1039352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.809783 (+2033114us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540038aed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:29.835017 (+ 25234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388092672}
I20260709 13:07:29.845831 21851 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:07:29.846451 21851 pending_rounds.cc:85] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 11
I20260709 13:07:29.846776 21851 pending_rounds.cc:107] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:07:29.847078 21851 raft_consensus.cc:2889] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.848924 27365 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:29.849787 21860 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:29.853163 22008 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:07:29.854916 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.845396 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.867239 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.868121 21928 raft_consensus.cc:2368] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17.
I20260709 13:07:29.879230 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:29.880713 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.885509 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.842558 27245 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:29.832701 27152 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:29.892350 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 63597.5us
I20260709 13:07:29.882705 27296 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.902009 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:29.890066 27462 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:29.905442 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.923543 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:29.899194 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:29.895496 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.933449 21931 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:29.934334 26130 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:29.937517 22003 raft_consensus.cc:2410] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:29.938284 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:29.938678 21928 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:29.937572 22001 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:29.955050 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:29.961371 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 963) took 2487 ms. Trace:
I20260709 13:07:29.963551 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.964257 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.965046 22008 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:29.932096 27064 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:29.962984 21930 rpcz_store.cc:276] 0709 13:07:27.491584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.495060 (+  3476us) service_pool.cc:224] Handling call
0709 13:07:27.516301 (+ 21241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.516655 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.516680 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.516732 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:27.516761 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.516869 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.961124 (+2444255us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b54003070d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.961286 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12628224}
W20260709 13:07:29.967360 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2712) took 6661 ms (client timeout 1976 ms). Trace:
I20260709 13:07:29.941773 27365 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:29.970464 21928 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:29.970271 21929 rpcz_store.cc:269] 0709 13:07:23.300367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.300489 (+   122us) service_pool.cc:224] Handling call
0709 13:07:29.936402 (+6635913us) spinlock_profiling.cc:243] Waited 4.33 s on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:29.962048 (+ 25646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9957046656}
I20260709 13:07:29.961954 27399 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:29.981732 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:29.982376 21931 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.988426 22008 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:29.994326 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:29.995122 22003 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:29.996520 26992 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:29.996899 26992 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:29.995165 27470 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:07:30.012159 21927 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:07:30.016681 21927 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:07:30.006669 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70937.9us
I20260709 13:07:29.973523 27365 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.019172 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.021039 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.022867 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.023157 21928 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:30.023540 22556 leader_election.cc:400] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17.
I20260709 13:07:30.023820 22556 leader_election.cc:403] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:07:30.022959 27365 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.024271 27462 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:07:30.022908 27064 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
W20260709 13:07:30.029801 27116 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.098s	user 0.039s	sys 0.025s
W20260709 13:07:30.038552 27116 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.167s	user 0.039s	sys 0.026s
I20260709 13:07:30.032151 27462 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:30.043881 21931 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:30.044725 26697 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:30.044029 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2639) took 2142 ms. Trace:
I20260709 13:07:30.045418 21854 rpcz_store.cc:276] 0709 13:07:27.901119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.925408 (+ 24289us) service_pool.cc:224] Handling call
0709 13:07:27.965326 (+ 39918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.982988 (+ 17662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.983039 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.983102 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:27.983149 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.043693 (+2060544us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:30.043739 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.043864 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39897344}
I20260709 13:07:30.039892 27462 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:30.037779 27064 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:30.052321 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2671) took 1767 ms. Trace:
I20260709 13:07:30.053346 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.074294 22003 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:30.021337 27084 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:30.012859 27475 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:07:30.012111 27245 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.010542 27270 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.060649 27062 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:30.064271 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 109344us
I20260709 13:07:30.064651 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:30.065464 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.080587 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.081602 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.082196 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:30.045145 26697 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:30.035310 27256 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:30.061962 27064 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:30.086581 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.071597 27480 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17.
I20260709 13:07:30.089932 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:30.086570 27375 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:07:30.091463 27479 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:07:30.092694 27442 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:30.092253 27064 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.093446 27442 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.094197 27152 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.094900 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.095777 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.096135 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.097026 22000 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:30.097138 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.097914 22009 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:30.098716 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.098794 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.099143 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.099735 22005 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:30.076812 27084 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.101418 27084 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.066247 26992 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.105098 26758 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:30.101460 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.109045 22008 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:30.109350 22007 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:30.110893 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2663) took 1902 ms (client timeout 2247 ms). Trace:
W20260709 13:07:30.111274 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65424.5us
W20260709 13:07:30.111395 21853 rpcz_store.cc:269] 0709 13:07:28.208080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.315992 (+107912us) service_pool.cc:224] Handling call
0709 13:07:30.108207 (+1792215us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54003d3954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:30.110811 (+  2604us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349687552}
I20260709 13:07:30.108906 22005 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:30.116343 22003 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:30.115679 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:30.083240 21858 rpcz_store.cc:276] 0709 13:07:28.279103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.704706 (+1425603us) service_pool.cc:224] Handling call
0709 13:07:29.705042 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.720053 (+ 15011us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:29.805259 (+ 85206us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.046453 (+241194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.046504 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:30.046550 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.046619 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.046728 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":172721,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":1363,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9779,"thread_start_us":242790,"threads_started":2,"wal-append.queue_time_us":252826}
I20260709 13:07:30.116765 22003 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:30.108460 26758 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.121521 27375 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:30.121975 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.131335 22003 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.123397 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2652) took 2108 ms. Trace:
I20260709 13:07:30.148336 21852 rpcz_store.cc:276] 0709 13:07:28.015105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.650615 (+635510us) service_pool.cc:224] Handling call
0709 13:07:29.825220 (+1174605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.830153 (+  4933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.123195 (+293042us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540018e550. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:30.123337 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2658217728}
I20260709 13:07:30.124578 22004 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:30.150223 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.126569 27256 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:30.130769 27481 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:07:30.138235 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.098784 27442 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.131071 27475 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:07:30.152462 22003 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.105988 27245 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.152985 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.155224 22585 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:30.147149 27391 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:30.094239 27064 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.111981 27480 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:07:30.146248 26758 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.161319 22009 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:07:30.166671 22007 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:30.167050 22006 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:30.167627 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.147006 27375 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.156105 22584 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:30.087656 27462 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.162770 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.172494 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.184530 22005 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:30.170893 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:30.185315 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 110361us
I20260709 13:07:30.173324 27331 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.177757 21929 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:30.188865 27457 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:30.172391 27487 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.181540 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.198988 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.202929 22003 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:07:30.203547 22003 pending_rounds.cc:85] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 8
I20260709 13:07:30.203846 22003 pending_rounds.cc:107] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:07:30.204142 22003 raft_consensus.cc:2889] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:30.199924 22009 raft_consensus.cc:2376] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14. Re-sending same reply.
I20260709 13:07:30.207322 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.208694 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.210527 26909 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:30.182921 27375 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.183305 27481 consensus_queue.cc:1243] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:30.156697 26697 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.238371 21926 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:30.185883 27480 consensus_queue.cc:1243] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:30.238828 26697 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:30.239742 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.240316 21852 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.241285 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.241508 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.242074 22009 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:07:30.185890 27063 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:07:30.176853 27391 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.243680 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.224517 26792 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:30.245486 26792 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.245864 26792 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:30.246469 27094 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:30.246901 27094 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:30.247766 27094 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.223248 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.250073 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2640) took 2348 ms. Trace:
I20260709 13:07:30.250417 21857 rpcz_store.cc:276] 0709 13:07:27.901477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.926125 (+ 24648us) service_pool.cc:224] Handling call
0709 13:07:30.147910 (+2221785us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b58001b1598. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:30.250009 (+102099us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54937344}
I20260709 13:07:30.252533 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.254002 22008 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:07:30.224287 27442 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:07:30.226306 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.263062 27396 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:30.264387 27396 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:30.267081 27457 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:30.267812 27457 consensus_queue.cc:1243] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:30.273274 27391 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:30.274235 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.226790 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.228148 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.278148 21858 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:07:30.278410 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51711.1us
I20260709 13:07:30.227530 27487 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.230727 27270 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.185698 27152 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:07:30.233496 26909 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:30.281946 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 153898us
I20260709 13:07:30.234107 27296 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:30.237813 21859 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:30.280498 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.282495 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.292397 22006 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:30.285063 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.293200 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.293905 22003 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:30.294687 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.295310 22009 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:07:30.304203 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.305270 21852 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:30.305485 22005 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:30.308029 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.309183 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.308688 22009 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:30.279556 27442 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.311911 22586 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:30.291352 27094 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:30.312932 22007 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:30.313694 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.317205 26909 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.319792 27375 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:30.321532 27270 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.316779 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.339319 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.339820 22005 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:30.267092 27365 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.323025 27026 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:30.325196 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:30.360364 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.325534 27152 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.336521 26758 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.305883 27329 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:30.313052 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.346256 22002 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
W20260709 13:07:30.347972 27469 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.481s	user 0.003s	sys 0.000s
I20260709 13:07:30.332443 26792 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.344199 27256 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:30.350034 27084 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:30.350994 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.353163 27365 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:30.356182 27375 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:30.368100 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55194.5us
I20260709 13:07:30.387055 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.375397 27329 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.375758 22005 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:30.378458 21859 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:30.378727 27060 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.379444 22004 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:30.390158 22006 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:30.364344 27270 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.411964 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.376750 27492 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:30.400926 27026 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:30.416345 21858 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:30.417486 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:30.419157 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.419523 22005 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:30.420610 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.421492 22005 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:30.422406 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.423324 22005 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:07:30.424583 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.391386 27442 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:30.429746 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 160990us
I20260709 13:07:30.426992 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.432734 21859 raft_consensus.cc:2368] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15.
I20260709 13:07:30.432662 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.376850 27152 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.438470 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.441555 21857 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:30.441766 27060 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.439124 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.443059 21859 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.444213 21853 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:30.426538 27329 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:30.448292 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:30.448329 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61395.9us
I20260709 13:07:30.449018 22006 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.449797 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.451732 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.454942 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.455876 21853 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:30.461930 26792 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.463351 27084 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:30.417510 27256 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.466265 27331 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:30.468076 21857 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:30.468344 27152 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:30.468853 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.447685 27329 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.469758 27329 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:30.470623 27487 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:30.426926 27492 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.435216 27442 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.473434 27492 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:30.474922 27442 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.414636 27270 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.477124 27296 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:30.479738 22005 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:30.481564 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.482121 21859 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:30.482600 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.487335 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.489363 22008 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:30.489596 22006 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:30.490625 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.491487 22006 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:30.501111 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.507201 27127 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:30.508293 26992 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:30.513707 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.524928 27481 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:30.525218 27329 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.527549 27329 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.535948 22007 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:30.537140 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:30.497262 26909 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:30.543354 21859 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:30.547747 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.548928 22006 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:30.476956 27270 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.558395 22003 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:30.554467 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:30.562718 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.563741 27496 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:07:30.564953 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.565646 22005 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:30.500057 27296 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:07:30.569531 27296 consensus_queue.cc:1243] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:30.539167 27442 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.541314 27176 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:30.573992 27176 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:30.574455 27176 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:30.588331 27480 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:30.588960 27480 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.593254 26931 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:30.593824 26931 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:30.594290 21856 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:30.594801 26931 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.596138 21855 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:30.596616 26931 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.518586 26792 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:30.597800 26792 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.598292 26792 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.597627 27116 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:07:30.598783 26130 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:07:30.602131 21856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2678) took 1816 ms (client timeout 2203 ms). Trace:
I20260709 13:07:30.598315 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2661) took 2391 ms. Trace:
I20260709 13:07:30.603071 21855 rpcz_store.cc:276] 0709 13:07:28.207319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.658677 (+1451358us) service_pool.cc:224] Handling call
0709 13:07:30.595802 (+937125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.596478 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.598136 (+  1658us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b540008b6d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:30.598258 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2154329984}
W20260709 13:07:30.603713 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66627.4us
I20260709 13:07:30.599874 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:30.602898 21856 rpcz_store.cc:269] 0709 13:07:28.785504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.226095 (+440591us) service_pool.cc:224] Handling call
0709 13:07:30.595682 (+1369587us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:30.602065 (+  6383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3127167744}
I20260709 13:07:30.612437 27474 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:30.608500 27329 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:30.614055 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.615295 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:30.615866 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61485.4us
I20260709 13:07:30.616645 27462 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:30.617995 21935 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:30.621249 27026 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:30.606894 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.616588 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.633968 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.637324 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.638334 21933 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:30.639034 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:30.639727 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.616945 21824 leader_election.cc:400] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:30.641937 21824 leader_election.cc:403] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:30.645514 21858 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:30.649029 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.653755 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:30.654677 21852 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:30.672837 21855 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:30.674504 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.676210 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.651919 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.681702 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.683930 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.653755 27474 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15.
I20260709 13:07:30.685905 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.686612 21851 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:30.685961 27065 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:30.687637 27065 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:30.688006 27065 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:30.689615 22006 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.690464 27457 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.686615 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.691399 22004 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:30.691757 21898 leader_election.cc:400] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15.
I20260709 13:07:30.692265 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:30.693480 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.694527 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:30.695454 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.697572 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.698747 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.701738 27315 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:30.702469 27315 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:30.703586 27315 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.675778 27202 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:07:30.689656 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.709506 21928 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:30.714514 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.715456 21931 raft_consensus.cc:2410] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:07:30.717913 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:30.717245 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162871us
I20260709 13:07:30.643846 27480 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:30.642709 27329 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:30.716863 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.724720 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:30.648672 27116 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.738973 21852 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:30.741127 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.644070 27060 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:07:30.741532 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:30.746575 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.736843 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.753700 21852 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:30.754953 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:30.756453 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.761957 22006 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:07:30.762508 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.763746 21928 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:30.767413 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.766202 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.725291 26931 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:30.779148 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.780349 21860 raft_consensus.cc:1217] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.9->[10.10-14.12]   Dedup: 10.10->[12.11-14.12]
I20260709 13:07:30.780857 21860 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [10.10-14.12]
I20260709 13:07:30.730664 27495 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.729395 27457 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:30.777540 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.775206 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.793392 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.727694 27474 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:07:30.728650 27402 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:07:30.786119 21851 raft_consensus.cc:2368] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16.
I20260709 13:07:30.787976 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.788934 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.742419 26697 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:30.741956 27162 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:30.834175 21928 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:30.765509 27202 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:30.784160 27360 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:07:30.834322 22004 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:30.775646 27475 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:30.729182 27305 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:07:30.730597 27270 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:07:30.777263 27256 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:30.782382 27491 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:30.764923 27026 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:07:30.787240 27296 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.784505 27501 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:30.831272 27152 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:30.786928 26931 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.837229 27490 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.787194 27084 raft_consensus.cc:2764] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:07:30.799574 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.766136 26909 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
W20260709 13:07:30.855306 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 89209.1us
I20260709 13:07:30.855931 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:30.740213 27116 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.831848 27331 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:30.846138 27315 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:30.845695 26697 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:30.839049 27162 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.798107 27063 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:07:30.889670 26697 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:30.846711 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60691.6us
I20260709 13:07:30.851747 27360 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.859421 27296 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.870620 27329 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.857191 27475 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:30.832386 27127 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
W20260709 13:07:30.881682 27502 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:07:30.799326 27495 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.893816 21933 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:30.879522 27065 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:30.900032 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 100563us
W20260709 13:07:30.902499 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 165723us
I20260709 13:07:30.914328 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:30.916929 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:30.913849 26992 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:07:30.898660 27331 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:30.918434 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48250.3us
I20260709 13:07:30.862562 27491 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:30.918710 27331 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.859156 27297 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
I20260709 13:07:30.861405 27152 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:30.840436 27461 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:07:30.872761 27315 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:30.912441 27475 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.892338 27202 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:30.891613 27162 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:30.790668 27457 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.926239 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:30.921615 27506 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:07:30.913713 27396 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.931209 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.897365 27064 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:07:30.932073 27315 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:30.941457 27296 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:30.932286 27202 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:30.942955 27202 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.943725 27462 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.944309 27462 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.947986 27026 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:30.950712 22593 raft_consensus.cc:2410] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:30.950691 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:30.930013 27152 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.906797 27360 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:30.960027 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:30.918634 26792 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.934290 27457 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:30.962733 27026 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:30.926841 27492 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:07:30.928596 27176 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.971025 22009 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:30.920079 27487 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:07:30.950129 27505 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:30.974824 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.970192 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:30.938860 27474 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:07:30.981393 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:30.986161 21851 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:30.982353 27116 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:30.982564 26792 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:30.922348 27491 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:30.957365 27509 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.984411 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:30.992381 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.993038 21928 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.994547 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:30.995193 21926 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:30.974061 27493 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:30.987788 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 68515.4us
I20260709 13:07:30.965493 27482 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:07:30.989899 27176 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:30.990370 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 76123us
I20260709 13:07:30.984295 27505 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.013711 22584 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:31.017665 22586 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:07:31.021299 27511 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.026023 27360 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.037508 21928 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:31.039588 21926 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:31.033607 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:31.060549 27360 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:31.062528 22584 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:31.065712 22586 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:31.067587 27202 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.069039 27315 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.071535 21853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:07:31.072075 27315 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.072393 21853 raft_consensus.cc:606] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:31.072750 21853 raft_consensus.cc:614] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:31.069813 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 143662us
W20260709 13:07:31.074095 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 104013us
I20260709 13:07:31.074361 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:31.011763 27505 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.078462 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:31.075636 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:31.017588 27065 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.088900 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:31.090642 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:31.090814 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:31.069536 27493 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:31.092908 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:31.094472 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 83117.4us
I20260709 13:07:31.075845 27360 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.092603 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:31.098693 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:31.099073 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.099979 22585 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:07:31.094707 27491 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.077221 27315 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:07:31.093628 27461 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:31.095168 27509 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:31.085345 27482 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:07:31.123418 27482 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.129788 27202 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.107152 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.119762 27399 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.111724 27360 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.134737 27026 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:31.134586 27360 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.142230 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 973) took 1423 ms. Trace:
I20260709 13:07:31.142517 27475 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.107240 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.132778 27152 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.091938 27525 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:31.142829 21934 rpcz_store.cc:276] 0709 13:07:29.718466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.777522 (+ 59056us) service_pool.cc:224] Handling call
0709 13:07:29.777869 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.794401 (+ 16532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.794452 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.817529 (+ 23077us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.962394 (+144865us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:29.979032 (+ 16638us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.979457 (+   425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.979552 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.526916 (+547364us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b18003f63a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:31.139438 (+612522us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b18003f63a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:31.141934 (+  2496us) raft_consensus.cc:1685] finished
0709 13:07:31.141965 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.142147 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19141,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":39,"raft.queue_time_us":319,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":219,"spinlock_wait_cycles":12145280,"thread_start_us":114064,"threads_started":1,"wal-append.queue_time_us":153225}
I20260709 13:07:31.095677 27493 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:31.109817 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:31.119455 27518 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:31.121197 21854 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:07:31.107662 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.093053 27474 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:31.102837 27496 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.154320 27525 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:31.155615 26758 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.156548 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:31.168476 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1265) took 1011 ms. Trace:
I20260709 13:07:31.158428 27296 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:31.159126 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:31.159562 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60915.9us
I20260709 13:07:31.175595 22584 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:31.168927 22589 rpcz_store.cc:276] 0709 13:07:30.157512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.157635 (+   123us) service_pool.cc:224] Handling call
0709 13:07:30.172779 (+ 15144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.614376 (+441597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:30.614418 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.614468 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:30.614505 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.168262 (+553757us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54005618d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:31.168314 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.168421 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8453,"raft.queue_time_us":17184,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":982486912}
I20260709 13:07:31.162331 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:31.112947 27521 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:31.167209 27511 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:31.165799 27493 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.170039 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:31.182500 22584 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:07:31.210887 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 118045us
I20260709 13:07:31.189103 27462 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.184572 27162 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.189589 22007 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:31.180030 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:31.193385 27524 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.202945 22556 leader_election.cc:400] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16.
I20260709 13:07:31.226552 22556 leader_election.cc:403] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:31.219298 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:07:31.224788 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54784.7us
I20260709 13:07:31.165918 27518 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:31.175189 27296 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:31.230234 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:31.230692 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50745.5us
I20260709 13:07:31.174170 27525 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:31.168255 27496 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.232262 27492 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.210453 27521 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:31.234050 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.177726 27501 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:31.234791 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:31.236117 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:31.250362 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:31.251125 22006 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:31.251773 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:31.253592 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:31.227584 27515 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:31.254354 22007 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:31.255254 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:31.223723 26992 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:07:31.264029 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.264868 21926 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:31.235661 27176 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:07:31.230439 27360 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.242102 27521 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.248915 27442 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:31.273563 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.274639 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.269220 21929 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:31.268203 27297 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:31.270668 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.237710 27529 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:31.277510 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.248613 27501 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:31.255900 27511 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:31.319128 21932 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:31.317830 22585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:31.261981 27116 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:31.279889 21935 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:31.285385 21931 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:31.322050 22585 raft_consensus.cc:606] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:31.335150 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.360407 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:31.361670 21926 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:31.351137 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:31.362993 21929 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.268193 27270 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:31.318493 27529 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:31.372113 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 111330us
I20260709 13:07:31.380259 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:31.278179 27152 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:31.254034 27528 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:07:31.318305 27360 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.380321 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:31.401921 21935 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:31.349390 27116 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.356912 27511 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:31.388046 21927 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:31.321121 27501 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.321223 27442 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:31.350211 26697 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.320875 27515 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:31.387431 21926 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:31.415680 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:31.441308 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:31.443485 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.384498 27529 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.408525 26992 consensus_queue.cc:1243] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:31.269048 27457 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.413080 27116 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.465691 26758 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:31.466095 27402 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:07:31.481653 21930 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:31.472149 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:31.431387 27474 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:31.432535 27315 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.489691 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:31.494077 22007 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:31.409451 27496 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.504122 21927 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:07:31.372406 27296 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.389431 27505 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.512859 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.514695 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:31.513170 21930 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:31.414498 27360 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.499085 27297 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:31.517405 27297 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.481590 27026 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:31.526541 27531 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:07:31.527667 27536 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:07:31.529314 27296 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.530753 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:31.377270 27518 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.537196 21855 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:31.468109 27528 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:07:31.494527 27474 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:31.555509 27457 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.557811 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.560403 27505 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.562985 27360 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.567863 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.557927 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1125) took 1196 ms. Trace:
I20260709 13:07:31.602540 22009 rpcz_store.cc:276] 0709 13:07:30.361786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.361980 (+   194us) service_pool.cc:224] Handling call
0709 13:07:30.966118 (+604138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.981859 (+ 15741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.557669 (+575810us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b540045b0d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:31.557857 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388612480}
I20260709 13:07:31.492841 27399 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:31.580152 27513 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:31.507323 27315 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.578357 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:31.450237 26697 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.582067 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.588284 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:31.634162 22591 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:31.594728 21935 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:31.596513 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:31.636947 22589 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:31.573746 27270 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:31.631258 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:31.585912 26992 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.634346 22590 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:31.574476 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.591432 27162 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.609874 27026 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:31.646270 27504 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.948s	user 0.005s	sys 0.000s
I20260709 13:07:31.658229 22585 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:31.662657 21932 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:07:31.664937 22590 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:31.665772 21931 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:31.631925 27513 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:31.673626 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.628021 27360 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:31.675575 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:31.657691 27026 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:31.593876 27442 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.709599 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.654012 26992 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.720022 27525 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.722865 27501 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.721583 27094 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.723970 22005 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:31.646759 27515 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.723999 27094 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:31.726841 27537 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:07:31.727340 27094 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:31.728505 27535 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.082s	user 0.001s	sys 0.001s
I20260709 13:07:31.730964 22002 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:31.734239 27315 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.735366 27513 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.736493 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.739404 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.627409 27511 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.736537 27513 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:31.742355 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:31.746019 22002 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:31.748355 22002 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:31.749110 27270 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:31.750273 27492 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
I20260709 13:07:31.750625 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2734) took 1435 ms. Trace:
I20260709 13:07:31.751451 27501 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:31.753239 22589 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:31.758787 26697 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:31.760399 27560 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:31.761571 27026 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.763309 27026 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.766574 27064 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:31.767169 27064 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:31.769943 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2692) took 1268 ms. Trace:
I20260709 13:07:31.770349 21857 rpcz_store.cc:276] 0709 13:07:30.501453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.501717 (+   264us) service_pool.cc:224] Handling call
0709 13:07:30.502078 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:30.528344 (+ 26266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:30.528379 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:30.528857 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:07:30.599670 (+ 70813us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:30.599732 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.600820 (+  1088us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:31.552681 (+951861us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:31.769590 (+216909us) raft_consensus.cc:1685] finished
0709 13:07:31.769612 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.769875 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":64588032,"thread_start_us":70367,"threads_started":1,"wal-append.queue_time_us":124968}
I20260709 13:07:31.771642 27495 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.773522 26931 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.751225 22008 rpcz_store.cc:276] 0709 13:07:30.315407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.315599 (+   192us) service_pool.cc:224] Handling call
0709 13:07:30.315878 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.489756 (+173878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.750153 (+1260397us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540040c254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:31.750522 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398493056}
I20260709 13:07:31.777112 27475 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.779378 27245 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:31.779829 27245 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:31.780682 27245 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.781785 27245 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.782649 27399 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.783174 27518 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.783324 27399 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.787058 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.787434 21928 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:31.788323 21929 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:07:31.788630 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.791458 21855 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:31.795574 21853 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:07:31.797254 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.798233 21853 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:31.799489 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.801559 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.810099 21852 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:31.815829 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:31.824110 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.828109 21928 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:31.829308 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.831640 27528 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:31.833473 27481 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:31.833911 27481 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:31.835183 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.680680 27474 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.836527 27474 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.839807 21859 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:31.843261 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.702666 27360 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:31.838728 27493 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.000s
I20260709 13:07:31.768271 27064 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.750687 27492 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16.
I20260709 13:07:31.856298 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.809705 27525 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:31.858982 22586 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.002s	sys 0.014s
W20260709 13:07:31.873832 22586 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.002s	sys 0.014s
I20260709 13:07:31.860632 22588 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:31.864099 26130 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:31.870149 22593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1260) took 1715 ms (client timeout 2009 ms). Trace:
I20260709 13:07:31.871687 22590 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:31.875806 26130 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:31.876572 22593 rpcz_store.cc:269] 0709 13:07:30.154379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.154538 (+   159us) service_pool.cc:224] Handling call
0709 13:07:31.862598 (+1708060us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b54003791d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:31.870055 (+  7457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1759365504}
I20260709 13:07:31.849783 27063 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.885795 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:31.853670 26697 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.887352 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.895890 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:31.850550 27528 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:07:31.834707 27481 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.899415 26130 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:31.899610 27530 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:31.889096 27521 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.001s
I20260709 13:07:31.857672 27475 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:07:31.862720 27480 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:07:31.944136 27480 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:31.944447 27509 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.000s
I20260709 13:07:31.944584 27480 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:31.945885 27480 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.947119 27480 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:31.847352 27490 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:07:31.915516 27492 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.919296 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:31.914650 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:31.951663 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.951968 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.952184 27487 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.952600 21858 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:31.952692 27487 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.953488 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.921370 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:31.923481 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:31.954947 27487 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:31.955133 21852 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.957145 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:31.929435 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 253984us
I20260709 13:07:31.930122 27127 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:31.870713 27493 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.934136 26992 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.935998 27360 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.937043 27399 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:31.937966 27549 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:31.939436 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:31.939800 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:31.940176 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 97342.4us
I20260709 13:07:31.940373 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:31.863639 27495 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:07:31.941810 22000 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:31.942210 27501 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.948241 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.015300 22005 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:32.016657 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:31.894716 27064 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:31.957980 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.010s
W20260709 13:07:32.019537 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.001s	sys 0.010s
I20260709 13:07:31.963809 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:32.024638 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:31.960085 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.922013 27116 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.027276 27116 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.961297 22004 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:31.931264 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.028088 22001 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:31.854334 27529 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:31.965811 21852 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:31.996729 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:32.034458 21927 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:31.997690 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:32.001956 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:32.036314 22587 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:31.948961 27509 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:31.893766 27063 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:31.895990 26697 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.025126 27564 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.014336 27549 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.036449 22006 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.003599 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:31.985384 27492 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.038524 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:32.029260 22584 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:07:31.948403 27521 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.054244 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:32.057214 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50529us
W20260709 13:07:32.042759 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 196121us
W20260709 13:07:32.057793 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 54317us
I20260709 13:07:32.060736 22002 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:32.061414 21851 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:32.061578 21933 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:32.068075 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.061832 21931 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:32.072772 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.066635 22006 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:32.076519 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.078337 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:32.079454 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:32.082433 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.041499 27176 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.082048 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.084633 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:32.145699 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:32.146136 22000 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:32.084906 21926 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:07:32.041201 27481 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.086241 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:32.089807 21927 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:32.148449 27116 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:32.149107 27116 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:32.092208 27065 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:32.150321 27116 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.152488 22590 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:07:32.059950 27539 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.576s	user 0.004s	sys 0.000s
W20260709 13:07:32.155452 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 73483.1us
I20260709 13:07:32.156566 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.090209 27495 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:32.147554 26758 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:07:32.162217 27521 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:32.051263 27564 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.167788 27548 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:32.168177 27493 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.098562 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.058364 26931 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:32.089718 27528 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:07:32.170454 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:32.172031 22003 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:32.090322 27462 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.173429 22003 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:32.174582 21857 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:07:32.055632 27549 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:32.057281 27063 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:32.050987 26697 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.059837 27492 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:31.949837 27530 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.042753 27501 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.197789 27121 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:32.196507 26931 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:07:32.200106 27501 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:32.107702 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:32.200884 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 102151us
I20260709 13:07:32.168830 27548 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 13:07:32.202759 27572 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:32.203835 22004 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:32.203984 27529 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.084322 27486 raft_consensus.cc:2764] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:07:32.205024 27486 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:32.205399 27486 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.205718 27486 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.177299 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.143378 21934 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:32.147341 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61148.9us
I20260709 13:07:32.210363 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.211180 27492 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:32.211807 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.212980 27492 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:32.191220 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45612.1us
I20260709 13:07:32.185099 22002 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:32.216850 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.218953 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2688) took 1780 ms. Trace:
I20260709 13:07:32.219810 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.222112 26931 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:32.223024 27329 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.223271 26931 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:32.219414 21854 rpcz_store.cc:276] 0709 13:07:30.438718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.439755 (+  1037us) service_pool.cc:224] Handling call
0709 13:07:30.447954 (+  8199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:31.167042 (+719088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.218623 (+1051581us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b540010ca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:32.218865 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547664768}
I20260709 13:07:32.226246 21974 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.235240 22000 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:32.244694 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.246556 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.257714 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.265455 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.266522 22590 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:32.268896 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:32.252679 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 221854us
I20260709 13:07:32.274060 22585 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:07:32.231178 27515 raft_consensus.cc:2764] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:07:32.278674 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52551.4us
I20260709 13:07:32.279835 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.282143 26130 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.282800 26130 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.283560 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.286481 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.287992 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:32.289374 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.251636 27490 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:32.145300 27442 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.134s
I20260709 13:07:32.298851 26758 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:32.176832 27176 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.286628 27548 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:32.305151 27297 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:32.306697 27152 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:32.307819 27152 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:32.308166 27152 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:32.327440 26130 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:32.289288 27516 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:32.293757 27553 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:32.336423 26758 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.338943 26758 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.286634 27493 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:07:32.288805 27509 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:07:32.321223 27127 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:32.281424 27475 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:07:32.340886 22002 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:32.338191 27516 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:32.289352 27549 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:07:32.351950 26758 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.294278 26909 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:07:32.353883 27524 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:32.359159 27563 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.292s	user 0.004s	sys 0.000s
I20260709 13:07:32.330540 27573 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:32.361084 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 988) took 1103 ms. Trace:
I20260709 13:07:32.289161 27457 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:07:32.338975 27402 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:07:32.373705 21933 rpcz_store.cc:276] 0709 13:07:31.257302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.257616 (+   314us) service_pool.cc:224] Handling call
0709 13:07:31.257874 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.071526 (+813652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.360690 (+289164us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400229a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:32.360981 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847934720}
I20260709 13:07:32.375612 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.374670 26909 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:32.368708 27564 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:32.308938 27152 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:32.382544 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 213425us
I20260709 13:07:32.342630 27577 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:07:32.384331 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.387210 22591 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:32.388247 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.372448 27524 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:32.379473 26909 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:32.374776 27573 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:32.417701 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.418597 22591 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:32.429435 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.416667 27577 consensus_queue.cc:1243] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:32.386201 27528 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:32.416900 27573 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:32.435715 27575 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:07:32.440383 22009 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:32.441332 27516 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.452574 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2808) took 1202 ms. Trace:
I20260709 13:07:32.453022 21934 rpcz_store.cc:276] 0709 13:07:31.250455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.250572 (+   117us) service_pool.cc:224] Handling call
0709 13:07:31.250899 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.209331 (+958432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.452345 (+243014us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400354dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:32.452518 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934480256}
W20260709 13:07:32.478701 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.420387 21900 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.432658 27501 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:32.446133 27152 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:32.483189 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 62941.5us
I20260709 13:07:32.484624 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.487054 27457 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:32.445426 27528 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:32.487490 27457 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:32.487891 27457 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:32.488313 27528 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.490718 27524 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.491791 27524 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.492233 27399 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:32.492894 27399 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:32.493788 27581 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:07:32.494046 27399 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.495954 27399 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.499070 26909 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.500485 26909 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.500519 26697 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:32.501113 26697 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:32.501929 26697 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.502928 26697 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.503621 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.508509 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:32.512071 27176 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:32.515914 27486 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.519194 27530 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:32.525377 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.525929 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.482030 27563 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:07:32.528196 27084 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:32.567721 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 264123us
I20260709 13:07:32.570744 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.571635 22000 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:32.574175 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:32.576305 22591 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:32.578598 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.578454 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.579715 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:32.583707 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.585156 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.586107 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:32.586720 22591 raft_consensus.cc:606] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:32.587170 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.587541 22591 raft_consensus.cc:614] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:32.587302 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 236807us
I20260709 13:07:32.587805 22586 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:32.589111 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.590276 21860 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:32.590195 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:32.576624 27457 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.593924 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:32.594676 21933 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:32.596585 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.597150 21933 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:07:32.597558 21933 pending_rounds.cc:85] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 8
I20260709 13:07:32.597899 21933 pending_rounds.cc:107] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:07:32.598157 21933 raft_consensus.cc:2889] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:32.601903 27509 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:32.602506 27509 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.602887 27509 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.602954 27577 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:32.603540 27577 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.603737 27511 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:32.603951 27577 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.604347 27511 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:32.604801 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.605288 27511 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.606985 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.607736 22591 raft_consensus.cc:2368] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14.
I20260709 13:07:32.608043 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:32.559201 27084 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.585515 27270 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:32.623646 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.623224 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:32.625628 21933 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:32.626605 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:32.630239 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:32.588680 27530 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:32.582036 27580 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:32.630864 21935 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:32.627287 21900 leader_election.cc:400] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14.
I20260709 13:07:32.631551 26758 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:32.575057 27549 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:07:32.593787 27524 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:32.622854 26931 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.643476 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.625800 27256 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:32.650692 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:32.635771 27524 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.632433 27530 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.656466 27127 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:32.664215 27591 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:32.637116 27296 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:32.644805 27493 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:32.648133 27297 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:32.686300 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2826) took 1314 ms. Trace:
W20260709 13:07:32.686609 27583 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:07:32.687119 27590 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:32.691514 21930 rpcz_store.cc:276] 0709 13:07:31.371479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.540202 (+168723us) service_pool.cc:224] Handling call
0709 13:07:31.540658 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:31.569757 (+ 29099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:31.569801 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:31.570382 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:07:31.793348 (+222966us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:31.793384 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.680682 (+887298us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b5400429ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:32.680733 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:32.680781 (+    48us) raft_consensus.cc:1685] finished
0709 13:07:32.680793 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:32.686220 (+  5427us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":12232,"spinlock_wait_cycles":65814912,"thread_start_us":182652,"threads_started":1,"wal-append.queue_time_us":316121}
I20260709 13:07:32.668481 27530 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.668859 27127 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.668103 27524 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.657732 27486 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:32.639648 27576 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:07:32.649588 26931 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.718984 22588 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:32.701164 27590 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.701409 27493 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:32.726363 27529 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:32.742077 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 91470.8us
I20260709 13:07:32.711427 27127 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:32.739455 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 101576us
I20260709 13:07:32.699296 27296 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.753083 21860 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:07:32.742856 27481 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.566s	user 0.006s	sys 0.006s
W20260709 13:07:32.758872 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.753747 21860 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:32.756734 21930 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:32.741696 27590 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.766983 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.760797 21860 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.779635 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:32.781203 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:32.781911 21853 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:32.783352 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:32.785121 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.753046 27529 consensus_queue.cc:1243] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:07:32.790017 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.791270 21860 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:32.791312 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.794552 21853 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:32.801419 27084 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.790798 27553 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:32.762575 27116 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:32.805517 27116 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.810093 27577 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.811175 27549 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.811784 27549 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.812404 27577 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.813179 26909 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.813759 26909 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.816375 26909 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.817855 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.818981 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.819676 21855 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:32.820375 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:07:32.821046 21934 raft_consensus.cc:606] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:32.821435 21934 raft_consensus.cc:614] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:32.821569 27587 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:07:32.823524 27296 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:32.768142 27256 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:07:32.829941 27509 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.830485 27176 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:32.830386 27493 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.831197 27176 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.833238 27152 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:32.833977 27457 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.833869 27152 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:32.836364 27595 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:07:32.837630 27442 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:07:32.838421 27593 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:07:32.774497 27591 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
W20260709 13:07:32.806141 27597 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:32.797175 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:32.845283 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57447.3us
I20260709 13:07:32.843837 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.849180 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.844858 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:32.825714 27573 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.839668 27442 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:32.868203 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71087.7us
I20260709 13:07:32.872573 21860 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:32.874478 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:32.875140 21934 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:32.875751 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.876947 22007 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:32.879618 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.879781 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:32.880846 21927 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:32.883121 21931 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:32.883028 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.885922 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:32.886658 21856 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:32.884155 27475 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:07:32.879020 27577 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:07:32.865381 27116 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:32.865753 27152 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.892560 27529 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:32.882516 27457 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:32.881304 27296 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:32.882197 26697 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:32.881058 27442 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.865984 27297 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 13:07:32.898494 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 51687.9us
I20260709 13:07:32.906699 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1086) took 1051 ms. Trace:
I20260709 13:07:32.911895 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.909675 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:32.910208 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.894074 27521 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:32.914354 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73476us
I20260709 13:07:32.912371 22587 rpcz_store.cc:276] 0709 13:07:31.855610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.857481 (+  1871us) service_pool.cc:224] Handling call
0709 13:07:32.906490 (+1049009us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54006630d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:32.906627 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320444160}
I20260709 13:07:32.902235 27116 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.901770 27296 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.936866 27461 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.950929 27461 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.951637 27461 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.949632 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.955144 27511 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.955508 27461 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:32.957065 22584 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:32.960665 27511 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:32.907125 27329 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:32.972561 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.932168 27296 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.985612 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:32.917140 27521 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.917333 27524 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.927711 27256 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:07:32.967993 27176 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.999387 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:32.980181 27329 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:32.966084 27116 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:32.981889 27457 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:33.009616 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:33.010150 22587 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:33.008404 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.984853 27577 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:33.034130 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 185553us
I20260709 13:07:32.998144 26931 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.034584 22588 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:32.986196 27509 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:32.999521 26697 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:33.047624 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:33.036900 27399 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.004s
I20260709 13:07:33.053260 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.054889 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:33.056742 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.051347 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.057407 21852 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:33.035386 27495 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:33.065656 21855 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:07:33.067245 21855 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 8
I20260709 13:07:33.067533 21855 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:07:33.042852 27060 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:33.070688 21854 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:33.030886 27511 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:07:33.067704 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.074167 21927 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:33.067786 21855 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:33.075825 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.066870 27461 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.047251 27509 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:07:33.061659 27176 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.044713 27577 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.008402 27127 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.074069 27495 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.077219 22587 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:33.132112 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.135440 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.136926 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.135911 22009 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:33.134307 22005 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:33.170023 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:33.137074 27060 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.176651 22008 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:07:33.167878 27608 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:07:33.180745 21974 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:33.120386 27116 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.119603 27524 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.078601 27521 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:33.187006 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 153048us
I20260709 13:07:33.175420 27442 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.133162 26697 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.175238 27462 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:33.121655 27127 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.077695 27461 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.094383 27549 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:33.229980 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:33.098750 27176 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:33.232558 27568 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:07:33.251691 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 452979us
I20260709 13:07:33.265841 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.185405 27060 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.290540 22006 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:33.288149 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:33.288600 21856 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:33.289263 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.290032 21852 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:33.292333 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.291786 21860 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:33.291750 21858 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:33.293995 21852 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:33.287935 27606 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:07:33.261780 27511 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:33.301201 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.309285 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.260869 27614 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:33.310154 21853 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:33.302129 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:33.263538 27329 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.308028 22591 raft_consensus.cc:1217] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.9->[11.10-13.11]   Dedup: 11.10->[13.11-13.11]
I20260709 13:07:33.322702 22591 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:07:33.311213 21856 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:33.316942 21858 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
W20260709 13:07:33.317724 27612 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:07:33.318480 27611 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:07:33.266816 27026 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
W20260709 13:07:33.320027 27609 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:07:33.294129 27256 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:07:33.237089 26697 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:33.348877 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 168201us
I20260709 13:07:33.236392 27462 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:33.360044 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:33.319319 27605 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:33.365103 21858 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:33.301137 27457 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:33.287498 27490 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:07:33.321630 27600 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.321810 27625 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:33.301331 27626 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:33.378821 27577 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.368403 27621 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.371606 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:33.391848 21856 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:33.375525 27614 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:33.376386 27116 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:07:33.388572 27515 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.357793 27511 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:33.453612 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46916.7us
W20260709 13:07:33.475916 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:33.475392 27590 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.000s
I20260709 13:07:33.478854 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1090) took 1177 ms. Trace:
I20260709 13:07:33.479251 22589 rpcz_store.cc:276] 0709 13:07:32.301579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.607642 (+306063us) service_pool.cc:224] Handling call
0709 13:07:32.607975 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:32.608330 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:32.608377 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:32.634581 (+ 26204us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.004318 (+369737us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:33.004359 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.004429 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:33.478644 (+474215us) raft_consensus.cc:1685] finished
0709 13:07:33.478664 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:33.478797 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8588,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5938,"thread_start_us":335108,"threads_started":1,"wal-append.queue_time_us":519586}
W20260709 13:07:33.417464 27622 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:07:33.414873 27462 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.462275 27626 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:33.475364 27515 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.478885 27590 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.489035 22000 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:07:33.477048 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:33.415791 21758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 192) took 31033 ms (client timeout 15000 ms). Trace:
W20260709 13:07:33.417699 27530 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.705s	user 0.011s	sys 0.000s
I20260709 13:07:33.408242 26992 raft_consensus.cc:2764] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:07:33.408378 27613 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:07:33.508451 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:33.400874 27600 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.471679 27629 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:07:33.416014 27026 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.484020 27060 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.407368 27577 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:33.404860 27495 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.484122 27614 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.522774 27629 consensus_queue.cc:1243] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:33.526290 27627 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.537395 27627 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.539484 27627 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.418881 27621 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.541908 27621 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.541977 27026 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.506290 27501 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:33.544492 27501 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:33.544809 27516 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:33.545374 27516 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:33.546470 27516 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.548018 27516 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:33.463153 27524 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:07:33.549697 27524 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:33.550140 27524 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:33.550943 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1274) took 1232 ms. Trace:
I20260709 13:07:33.553124 27176 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:33.559027 27296 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.000s
W20260709 13:07:33.531127 27633 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:07:33.551458 22592 rpcz_store.cc:276] 0709 13:07:32.318687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.667118 (+348431us) service_pool.cc:224] Handling call
0709 13:07:32.667497 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:32.768025 (+100528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:32.768060 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:32.792314 (+ 24254us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.027975 (+235661us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:33.028019 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.411425 (+383406us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54005618d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:33.411514 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:33.550530 (+139016us) raft_consensus.cc:1685] finished
0709 13:07:33.550559 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:33.550867 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230581632,"thread_start_us":190583,"threads_started":1,"wal-append.queue_time_us":255172}
W20260709 13:07:33.561841 27620 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.193s	user 0.003s	sys 0.000s
I20260709 13:07:33.562281 27590 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.533627 27600 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.510780 27462 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:33.563570 27026 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.551196 27524 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.566020 27524 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:33.566443 27577 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:33.497927 21758 rpcz_store.cc:269] 0709 13:07:02.386263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.397948 (+ 11685us) service_pool.cc:224] Handling call
0709 13:07:02.541041 (+143093us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:02.557354 (+ 16313us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b5400011210. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6ab640 00007f3b76582b76 00007f3b7668cc3e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:33.415533 (+30858179us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:18.116432 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:18.117167 (+   735us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:18.117220 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:07:18.117241 (+    21us) tablet.cc:662] Decoding operations
0709 13:07:18.199021 (+ 81780us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:18.199157 (+   136us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:18.199186 (+    29us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:07:18.201204 (+  2018us) tablet.cc:701] Row locks acquired
0709 13:07:18.201256 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:07:18.201437 (+   181us) write_op.cc:270] Start()
0709 13:07:18.201625 (+   188us) write_op.cc:276] Timestamp: P: 1783602438201368 usec, L: 0
0709 13:07:18.201667 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:07:18.337331 (+135664us) log.cc:844] Serialized 7519 byte log entry
0709 13:07:19.469752 (+1132421us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cae28 00007f3b74205581 00007f3b74204d58 00007f3b742099f1 00007f3b742099a9 00007f3b74209939 00007f3b74209901 00007f3b74208b2d 000000000038f354 000000000038f199
0709 13:07:19.852968 (+383216us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:20.589906 (+736938us) op_driver.cc:464] REPLICATION: finished
0709 13:07:20.856903 (+266997us) write_op.cc:301] APPLY: starting
0709 13:07:20.857053 (+   150us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:21.584669 (+727616us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b180008dd60. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:21.960117 (+375448us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:21.960171 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:24.280534 (+2320363us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:24.280927 (+   393us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:07:24.280993 (+    66us) write_op.cc:312] APPLY: finished
0709 13:07:24.389973 (+108980us) log.cc:844] Serialized 172 byte log entry
0709 13:07:24.565905 (+175932us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:24.572820 (+  6915us) write_op.cc:454] Released schema lock
0709 13:07:24.598192 (+ 25372us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:24.673625 (+ 75433us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:24.676067 (+  2442us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10366469,"reported_tablets":35,"spinlock_wait_cycles":111450368,"child_traces":[["op",{"apply.queue_time_us":248644,"apply.run_cpu_time_us":75852,"apply.run_wall_time_us":3791170,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":180898,"prepare.run_cpu_time_us":34945,"prepare.run_wall_time_us":1361034,"raft.queue_time_us":422031,"raft.run_cpu_time_us":61159,"raft.run_wall_time_us":394843,"replication_time_us":2326869,"spinlock_wait_cycles":62635392,"thread_start_us":740171,"threads_started":5,"wal-append.queue_time_us":522255,"wal-append.run_cpu_time_us":38637,"wal-append.run_wall_time_us":3735562}]]}
I20260709 13:07:33.563740 27461 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.577387 27573 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:33.488004 27626 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.417245 26697 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.518175 27530 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.410404 27474 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:33.590157 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 331847us
W20260709 13:07:33.592517 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 84151.2us
I20260709 13:07:33.559490 27296 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.596364 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.591600 27462 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:33.590001 27495 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.610760 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:33.613065 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.405591 27625 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.620203 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.612357 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:33.619132 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 256119us
I20260709 13:07:33.621415 22004 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.622291 22590 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:33.616544 21927 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:07:33.622713 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:33.617476 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.620854 22003 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:33.633198 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:33.667614 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.594357 27577 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:33.671967 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.679275 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.679593 22004 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:07:33.682296 27613 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:33.626505 27495 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.682982 27530 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:33.592593 27474 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.695216 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.695598 27629 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.695868 21855 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:33.696369 27474 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.696182 27629 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.698057 27296 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.698446 27629 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.699074 27600 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:33.699580 27600 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.699888 27296 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.701691 27600 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:33.705886 27399 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:33.706312 27399 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:33.707638 26992 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.707485 27457 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:33.685791 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.672295 27634 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:33.689666 27625 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:33.690510 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.692083 27495 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.667230 27621 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:33.594764 27529 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:07:33.667059 27256 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.715907 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:33.707280 27399 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.709267 21935 raft_consensus.cc:2410] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:07:33.686226 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:33.692557 27501 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.710813 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:33.709197 21927 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:33.728080 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:33.686282 27515 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.700500 27296 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.716540 22592 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:33.719489 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:33.729599 22585 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:33.732599 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:33.755585 27493 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.006s	sys 0.014s
W20260709 13:07:33.756399 27493 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.006s	sys 0.015s
I20260709 13:07:33.757580 21860 raft_consensus.cc:1217] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Deduplicated request from leader. Original: 8.9->[10.10-14.12]   Dedup: 10.10->[12.11-14.12]
I20260709 13:07:33.758279 21860 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [10.10-14.12]
I20260709 13:07:33.775668 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.780572 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:33.781301 21855 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.782712 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.708132 26992 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.784045 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.794050 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.794790 21856 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:33.795881 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.797232 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.788183 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.799361 21856 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:33.801474 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:33.802347 21856 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:33.803715 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.805405 21854 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
W20260709 13:07:33.805791 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.807103 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.811600 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.777020 27399 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.752702 27515 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:33.815546 21854 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:33.830200 21855 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:33.810768 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:33.816731 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:33.832466 21852 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:33.826642 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.831542 21856 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:33.818234 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 114054us
I20260709 13:07:33.818950 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.737795 27461 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:07:33.820359 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.820905 22002 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:33.725687 27256 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.788326 27573 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.839108 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.821759 22592 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:33.825886 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.836409 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.869927 22000 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:33.870466 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.821614 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:33.810361 27645 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:33.811699 27640 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:33.859902 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.853289 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.859357 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:33.871932 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.875344 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.882287 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.822806 27653 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.882598 22000 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:33.873498 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:33.893875 21931 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:33.880479 22009 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:33.894465 22007 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:33.911926 22587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:33.912083 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.912711 22587 raft_consensus.cc:606] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:33.872350 27116 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.914027 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:33.917116 22009 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:33.918605 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:33.913471 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:33.921247 27493 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:33.877653 27646 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.922431 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:33.877291 27573 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.880328 27645 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:33.880455 27490 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.925450 21931 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:33.892261 27084 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.853426 27515 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.926941 21932 raft_consensus.cc:2376] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16. Re-sending same reply.
I20260709 13:07:33.931432 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:33.930946 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57544.2us
W20260709 13:07:33.933699 27641 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:33.910228 27649 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:33.894275 27653 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.937681 27654 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:33.942677 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.943650 21929 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:33.935228 27644 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:07:33.968885 27522 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:33.969408 27515 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:33.891390 27640 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.925954 27116 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:33.965953 21853 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:33.966670 27649 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:33.967921 27648 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:33.977782 22001 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:33.979564 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:33.982923 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.000784 21932 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:33.998611 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:34.002943 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:34.003520 22591 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:34.003670 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:34.005298 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.006284 22009 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:33.985929 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.007297 21934 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:34.001417 27297 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:07:34.008324 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.009371 27640 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:34.009523 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:33.973580 27501 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:34.010205 21932 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:34.012276 21932 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:07:33.938799 27256 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.005493 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:33.963812 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.015601 21823 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.018718 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:34.020164 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:34.020346 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.030359 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.038249 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:34.039175 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:34.040011 22591 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:34.041682 22591 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:07:34.042061 22591 pending_rounds.cc:85] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 10
I20260709 13:07:34.042349 22591 pending_rounds.cc:107] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:34.042660 22591 raft_consensus.cc:2889] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:34.043038 22591 pending_rounds.cc:107] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:07:34.043253 22591 raft_consensus.cc:2889] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:34.038769 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:33.979074 27511 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:34.048354 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 68897.6us
I20260709 13:07:33.972863 27528 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:33.939301 27646 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.049328 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:34.012287 27116 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:34.051303 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:33.952049 27084 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:33.950443 27490 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.022332 27614 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:07:34.027212 27667 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:34.038751 27256 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:34.058936 27632 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.059571 27653 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.075810 27627 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:34.050879 27529 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:34.062114 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.105031 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.105588 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.003260 27625 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:07:34.014051 27615 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:07:34.062806 27643 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:07:34.113375 27116 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.063510 27474 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:34.064016 27402 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:34.066211 27600 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.066705 27601 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:34.077404 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:34.080379 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.034794 27530 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:34.104282 27627 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.019897 27522 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:07:34.061498 27296 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.096002 27621 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:34.105798 21926 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:34.106906 27573 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.048207 26909 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:07:34.096724 27256 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.096511 27667 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.114027 27116 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.102420 27590 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:34.046489 27640 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:34.107877 27121 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.120710 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.188135 22002 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:34.108949 27529 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.114800 26992 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:34.125186 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 246015us
I20260709 13:07:34.046613 27645 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.134078 22007 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:34.136842 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:34.104601 27632 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.119812 27530 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.124024 27627 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.195971 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:34.216799 21926 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:34.142666 27256 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:34.141600 27667 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:34.255165 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 177809us
W20260709 13:07:34.226264 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 175015us
I20260709 13:07:34.243368 22587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bcf30627084357a4745445324a2430"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:34.257216 22587 raft_consensus.cc:606] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.257814 22587 raft_consensus.cc:614] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:34.191409 27634 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:34.187328 27640 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:34.193131 27529 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:34.266419 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70555.1us
I20260709 13:07:34.278973 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:34.268357 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:34.259598 27645 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:34.295502 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.296375 21852 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:34.225370 27530 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:34.135069 27621 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.290505 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.304957 21851 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:34.292006 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.293144 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:34.265483 27637 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:07:34.257325 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 242231us
I20260709 13:07:34.305606 21854 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:34.311048 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:34.312371 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.314527 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:34.277454 27634 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.312403 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:34.258158 27462 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.008s	sys 0.007s
W20260709 13:07:34.317799 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 223156us
I20260709 13:07:34.310808 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.318413 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:34.319152 21926 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:34.275341 27296 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:34.322328 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.297447 27615 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:34.283711 27640 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.349993 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:34.286396 27638 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.424383 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:34.424939 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 112580us
I20260709 13:07:34.425904 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.426360 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:34.322932 27462 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.008s	sys 0.007s
I20260709 13:07:34.302210 27521 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.266100 27658 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.268s
I20260709 13:07:34.287786 27591 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:07:34.321746 27634 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:34.314252 27649 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.305254 27674 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.446063 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.352001 27522 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.446277 27524 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.454037 27673 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:34.431746 27462 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:34.455966 27646 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.446645 27521 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.448503 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:34.457715 27673 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:34.457894 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.458282 27673 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:34.458803 21931 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:34.459705 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.432036 27656 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:34.461588 27656 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.463058 27521 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.463675 27521 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.464190 27521 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.432812 27653 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:34.459311 27673 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.351853 27474 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:34.351889 27615 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:34.454931 27677 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:07:34.446909 21926 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:34.428637 27509 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.460835 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:34.512288 22593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:34.444013 27516 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:34.318408 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:34.520948 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 471627us
I20260709 13:07:34.454352 27524 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.536729 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:34.549261 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.467373 27490 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.551796 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:34.552809 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:34.553197 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 234843us
I20260709 13:07:34.559499 21930 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:34.553323 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.552908 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.561808 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.552753 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.570210 27646 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:34.570739 27646 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:34.571136 27646 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:34.579161 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1167) took 1725 ms. Trace:
I20260709 13:07:34.579602 22590 rpcz_store.cc:276] 0709 13:07:32.853645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.909948 (+ 56303us) service_pool.cc:224] Handling call
0709 13:07:34.578954 (+1669006us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54006266d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:34.579101 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629425408}
I20260709 13:07:34.583283 27399 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:34.583840 27399 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:34.584532 27121 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:34.546612 26909 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.455965 27462 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.469473 27084 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.509802 27615 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:34.596093 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:34.599125 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.454136 26992 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
W20260709 13:07:34.604290 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51642.8us
I20260709 13:07:34.612533 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:34.620296 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.620160 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:34.623078 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 300381us
W20260709 13:07:34.609872 27652 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.728s	user 0.006s	sys 0.002s
I20260709 13:07:34.625420 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:34.625790 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:34.633718 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:34.543458 27509 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.616425 27522 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:34.648213 27647 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.729s	user 0.003s	sys 0.005s
I20260709 13:07:34.653234 27683 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.609422 26909 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.654084 27656 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:34.654811 27675 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:07:34.655866 26909 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:34.656555 27692 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:34.656987 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1006) took 1046 ms. Trace:
I20260709 13:07:34.658231 27693 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:34.653847 27701 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:34.657673 21935 rpcz_store.cc:276] 0709 13:07:33.610273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.610491 (+   218us) service_pool.cc:224] Handling call
0709 13:07:34.655254 (+1044763us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b5400307ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:34.656922 (+  1668us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224331008}
W20260709 13:07:34.662777 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 130247us
I20260709 13:07:34.584838 27399 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.640937 27490 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.669147 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:34.671149 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:34.655894 27678 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:07:34.658912 27693 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.676597 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.679575 22003 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:34.662639 27701 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.676836 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:34.666608 27691 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.690518 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:34.696359 27656 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:34.698977 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:34.674041 27116 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.678977 27629 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:34.677528 27632 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:34.708884 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.690037 27399 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:34.653885 27683 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.713192 27680 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:34.686152 27701 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.693421 27621 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.727594 27695 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:34.698028 27696 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:34.731802 27695 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.712273 27658 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.705372 27656 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:34.741250 27487 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:34.741806 27487 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:34.742861 27487 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.738360 27695 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:34.748893 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:34.749377 27682 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:07:34.749691 22588 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:34.750880 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.751616 21931 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:34.750877 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.753305 21928 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:34.754235 27676 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.617s	user 0.008s	sys 0.000s
I20260709 13:07:34.668562 27474 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:34.693120 27529 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.718189 26992 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.723328 27685 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:34.728096 27680 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.766103 27121 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:34.736536 27696 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:34.702244 27691 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.767827 27457 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.798645 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.790320 27658 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.805330 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:34.799213 27121 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.807595 27529 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:34.808219 27667 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.002s
I20260709 13:07:34.810132 27084 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.810621 27638 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.811305 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.812000 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.812790 27614 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:34.816596 27626 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.805392 27692 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:34.777889 27685 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:34.783887 27680 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:34.827519 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.828300 27509 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:34.829564 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.832056 27060 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:07:34.832695 27530 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.000s
I20260709 13:07:34.780185 27521 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:34.833402 22006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2771) took 1701 ms (client timeout 2029 ms). Trace:
W20260709 13:07:34.923303 22006 rpcz_store.cc:269] 0709 13:07:33.131667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.131796 (+   129us) service_pool.cc:224] Handling call
0709 13:07:34.833211 (+1701415us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540040d654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:34.833342 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287757696}
I20260709 13:07:34.833817 27683 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:34.834081 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:34.834821 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.835613 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2859) took 1010 ms. Trace:
I20260709 13:07:34.840516 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.926272 21856 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:34.927114 22004 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:34.841991 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.843621 26697 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:34.844300 22002 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:07:34.931054 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 97052.4us
I20260709 13:07:34.844311 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:34.928118 22003 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:34.932459 22001 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:34.845265 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:34.846454 27487 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:34.847167 27121 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:34.790573 27696 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.853304 21929 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:34.854995 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1100) took 1123 ms. Trace:
I20260709 13:07:34.886389 27616 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:34.857877 27693 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:34.859098 27670 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.733s	user 0.005s	sys 0.007s
I20260709 13:07:34.858553 27656 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.858944 27701 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.870360 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:34.871259 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:34.873654 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.953406 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:34.875979 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.872962 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:34.879560 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 209201us
W20260709 13:07:34.837522 27627 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.003s
W20260709 13:07:34.884298 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 215156us
I20260709 13:07:34.905645 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.879571 27653 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:34.780525 26992 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.924767 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.883452 27614 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:34.926162 22005 rpcz_store.cc:276] 0709 13:07:33.825535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.835389 (+  9854us) service_pool.cc:224] Handling call
0709 13:07:34.834210 (+998821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.834425 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.834452 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.834496 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:34.834530 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.834592 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.835367 (+   775us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b540040d650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:34.835549 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296219008}
I20260709 13:07:34.933593 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:34.936795 22592 rpcz_store.cc:276] 0709 13:07:33.731404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.732274 (+   870us) service_pool.cc:224] Handling call
0709 13:07:34.854755 (+1122481us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b54007e0254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:34.854914 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203154816}
W20260709 13:07:34.856638 27645 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.000s
I20260709 13:07:34.941226 27493 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:34.871338 27457 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.909919 27529 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.941089 27684 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:34.954705 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 81862.7us
I20260709 13:07:34.926783 27530 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.883126 27084 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.960127 21856 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:34.966850 27495 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:34.959111 21853 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:34.996879 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.001863 21856 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:35.002017 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.002787 21853 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:35.002754 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.004572 21856 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:34.961912 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.882230 27667 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.909560 27626 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:34.984479 21854 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:34.986806 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.914825 27685 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.039091 22593 raft_consensus.cc:606] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:35.039666 22593 raft_consensus.cc:614] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:34.989274 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:35.041638 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 52455.9us
I20260709 13:07:35.043108 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:35.044009 27462 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:35.044768 27462 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.045238 27462 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.045662 27462 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.049816 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:34.948136 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:35.051139 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 103102us
I20260709 13:07:34.935724 26697 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.053336 26697 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.055090 21857 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:34.926581 27695 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.871269 27658 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.007592 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.058400 21930 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:35.008069 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.008941 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.012192 22591 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:35.009477 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.003875 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.913589 27692 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.884991 27638 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.001200 27495 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:34.962898 27627 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:35.023389 27528 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.952s	user 0.007s	sys 0.006s
I20260709 13:07:35.023883 27490 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:35.026217 27701 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.028211 27667 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:34.950791 27616 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.029513 27646 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:34.993913 27529 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.031265 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2783) took 1111 ms. Trace:
I20260709 13:07:35.127764 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:35.128286 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.135603 27516 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:35.127154 27152 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:35.130798 21852 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:35.145462 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:34.969081 27653 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:35.166577 27634 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.699s	user 0.005s	sys 0.004s
W20260709 13:07:35.166719 27256 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.000s
I20260709 13:07:35.202958 21859 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:35.211720 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:35.058885 27693 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:34.998706 27645 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:35.223117 27621 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:35.151523 27495 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.223554 22000 rpcz_store.cc:276] 0709 13:07:33.920120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.933101 (+ 12981us) service_pool.cc:224] Handling call
0709 13:07:33.933495 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.933844 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.933876 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.934365 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.026421 (+ 92056us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:34.026469 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.028891 (+  2422us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.858371 (+829480us) spinlock_profiling.cc:243] Waited 5.33 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:35.030912 (+172541us) raft_consensus.cc:1685] finished
0709 13:07:35.030939 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.031195 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5434,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":6433,"spinlock_wait_cycles":12256000,"thread_start_us":91438,"threads_started":1,"wal-append.queue_time_us":167693}
I20260709 13:07:34.996265 27521 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.225314 21858 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:35.127583 27706 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:35.251534 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:35.262458 21926 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:35.133579 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.229111 27516 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:35.258790 27625 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.273275 27629 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:35.306495 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55038.2us
I20260709 13:07:35.243877 27152 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.282191 27645 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.296586 27667 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.251267 27634 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.265233 27256 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.325476 22585 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:07:35.303828 27590 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.301635 27516 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.272351 27693 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.312969 27481 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:35.307670 27649 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:35.340569 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 212816us
I20260709 13:07:35.286113 27621 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.348953 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:35.343523 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 211541us
I20260709 13:07:35.324528 27490 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.327762 27645 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.350320 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.354132 21935 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:35.338354 27656 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:35.325187 27474 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:07:35.317530 27629 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.362820 27693 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.345257 27481 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:35.363751 27721 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:07:35.364427 27708 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:07:35.364342 27646 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.365168 27084 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:35.395687 27640 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.961s	user 0.008s	sys 0.005s
W20260709 13:07:35.365651 27720 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:07:35.366444 27713 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:07:35.366875 27152 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:35.368564 27256 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:35.438743 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 85337.5us
I20260709 13:07:35.442822 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.371284 27680 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.444906 27629 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:35.445101 27724 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:07:35.445360 27680 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.372248 27696 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.447533 27696 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:35.448192 27715 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:07:35.449483 27722 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:07:35.355729 27649 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.383639 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.342406 27590 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.385452 27490 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.310536 27625 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.337121 27667 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.312856 27706 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:35.403510 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.431931 27627 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.362820 27529 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.366914 27695 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:35.427995 27615 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.000s
I20260709 13:07:35.428479 27640 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.429522 27616 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:35.434165 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.369956 22587 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:35.442421 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.301208 27521 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.451635 22590 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:35.359372 27658 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.451818 22008 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:35.443349 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.474674 27521 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.387574 27738 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:35.401917 27481 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.394464 22000 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:35.460078 27121 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:07:35.365188 27717 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:07:35.402717 26992 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.410255 27638 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.435254 27626 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.450675 27635 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:35.505458 27635 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:35.505604 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2388) took 2653 ms (client timeout 1931 ms). Trace:
W20260709 13:07:35.506083 21859 rpcz_store.cc:269] 0709 13:07:32.852502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.868533 (+ 16031us) service_pool.cc:224] Handling call
0709 13:07:35.505430 (+2636897us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b540010c054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:35.505536 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5247438720}
I20260709 13:07:35.506505 27635 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.396695 27656 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.475149 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.459502 27590 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.449699 27691 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:35.456727 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 73344.9us
I20260709 13:07:35.515846 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.467000 27509 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:35.483889 27649 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.489665 27621 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.495589 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.479408 27680 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:35.521852 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46778.9us
I20260709 13:07:35.522657 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:35.496704 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.498660 27490 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.500339 27495 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.501991 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.507741 27635 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.518214 22591 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:07:35.496353 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.536717 22589 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:35.518362 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.482052 27692 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.469128 27653 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.471015 27615 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.540396 27553 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:35.546445 27615 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:35.541906 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.549036 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.526376 27121 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.550642 27121 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:35.558493 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:35.558861 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2837) took 1618 ms. Trace:
I20260709 13:07:35.559118 27121 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:35.559656 27634 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.559676 21933 rpcz_store.cc:276] 0709 13:07:33.940601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.940705 (+   104us) service_pool.cc:224] Handling call
0709 13:07:33.940981 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.941319 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.941357 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.941768 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.765558 (+823790us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18002e3960. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:34.944751 (+179193us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:34.944805 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.944875 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.558572 (+613697us) raft_consensus.cc:1685] finished
0709 13:07:35.558596 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.558803 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7125,"raft.queue_time_us":5875,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":1168,"spinlock_wait_cycles":26012544,"thread_start_us":179014,"threads_started":1,"wal-append.queue_time_us":1478396}
I20260709 13:07:35.561148 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:35.561800 22591 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:35.562196 27706 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.562741 27296 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:35.563256 27296 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.566020 27667 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:35.566530 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 1.45893e+06us
I20260709 13:07:35.569271 27706 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:35.569585 26697 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:07:35.532611 27674 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:35.575254 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.572399 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.572276 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.575251 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.577147 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.578101 22004 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:35.571389 22591 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:35.580174 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:35.582150 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.582895 22009 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:35.582751 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.584023 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.535270 26992 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:35.590198 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.617717 26992 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:35.618183 26992 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:35.590842 22005 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:35.620240 27625 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.620393 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.621490 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.621575 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.621718 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 875) took 1069 ms. Trace:
I20260709 13:07:35.622418 22005 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:35.622684 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.594017 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.593411 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:35.625916 27671 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.488s	user 0.003s	sys 0.003s
I20260709 13:07:35.623984 22005 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:35.564744 27614 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.481889 27462 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.630192 27462 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.526072 27509 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.632275 27509 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.561179 27640 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:35.595932 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 448209us
I20260709 13:07:35.638293 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:35.639776 27121 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:35.639967 27706 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:35.600449 27495 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:35.600703 27296 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:35.603647 27716 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:07:35.644244 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:35.645471 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2406) took 1310 ms. Trace:
I20260709 13:07:35.608785 22004 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:35.613890 27026 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:35.648021 27026 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:35.648820 27026 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.569957 26697 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.650038 27026 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.650197 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:35.650790 21931 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.614233 27084 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:35.651453 27084 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.652313 21934 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:35.605607 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.615415 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.653469 27084 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.653702 22008 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:35.616083 27521 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:35.592073 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:35.655170 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 63160.4us
I20260709 13:07:35.656299 21930 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:35.651688 26697 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.622260 21854 rpcz_store.cc:276] 0709 13:07:34.551876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.553094 (+  1218us) service_pool.cc:224] Handling call
0709 13:07:35.621398 (+1068304us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540018e554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:35.621656 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":973683968}
I20260709 13:07:35.646552 21858 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:35.591223 27732 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:35.639983 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:35.660727 27121 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:35.607092 27522 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:35.661352 27121 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.662056 27522 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.662135 27121 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.645803 21855 rpcz_store.cc:276] 0709 13:07:34.335063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.335239 (+   176us) service_pool.cc:224] Handling call
0709 13:07:34.335679 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.336386 (+   707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.336426 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:34.751898 (+415472us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b2c000e8e68. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:34.752330 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.971893 (+219563us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:34.971946 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.972038 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.603098 (+631060us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:35.645279 (+ 42181us) raft_consensus.cc:1685] finished
0709 13:07:35.645297 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.645419 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3470,"raft.queue_time_us":86756,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":2863,"spinlock_wait_cycles":94751232,"thread_start_us":219232,"threads_started":1,"wal-append.queue_time_us":374455}
I20260709 13:07:35.659276 27732 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.664386 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:35.665153 21852 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:35.592445 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:35.665844 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73500.2us
I20260709 13:07:35.667073 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.667452 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.667675 21855 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:35.668532 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.672060 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.672111 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.676262 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.676821 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:35.682739 21972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.931s (SENT) [suppressed 32 similar messages]
W20260709 13:07:35.541486 27685 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 13:07:35.684738 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:35.686344 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:35.699025 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.688092 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:35.665658 27732 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.688863 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.692587 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.693166 27692 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:35.702080 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.695137 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:35.704744 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.695998 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.696122 22004 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:07:35.705392 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:35.696859 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.706022 21855 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:07:35.706214 22589 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:35.706346 22006 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.697517 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.594425 27743 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:35.708101 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:35.708343 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.709465 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:35.710638 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.711475 21851 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:35.712275 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.705835 22004 pending_rounds.cc:85] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 12
I20260709 13:07:35.713482 22004 pending_rounds.cc:107] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:07:35.713761 22004 raft_consensus.cc:2889] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:35.698402 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.714615 27732 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:35.715096 27732 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.715315 27026 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:35.700575 21858 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:35.694015 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:35.715927 27026 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.697749 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.716536 27654 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:35.704770 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.713323 27521 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:35.717139 27654 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.718391 27521 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.719724 27654 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:35.715440 27732 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.722275 27026 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.722862 27457 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.700687 21931 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:35.723860 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:35.724850 21932 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:35.716949 21857 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:35.702700 22007 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:35.726552 22006 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:35.715941 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:35104 (request call id 22) took 1247 ms. Trace:
I20260709 13:07:35.727283 22593 rpcz_store.cc:276] 0709 13:07:34.455596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.467138 (+ 11542us) service_pool.cc:224] Handling call
0709 13:07:35.703089 (+1235951us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54006a04d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707c9853 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691 00007f3b6ed1d60c 00007f3b6ed1c8c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:35.703194 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209542912}
I20260709 13:07:35.727771 22000 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:35.694044 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:35.698079 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:35.705463 21856 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:35.732628 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:35.720562 27121 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.739151 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:07:35.737727 22000 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:35.739549 27621 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:35.740113 27621 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.735917 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.741611 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:35.743209 21856 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:35.749497 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1016) took 1001 ms. Trace:
I20260709 13:07:35.742276 27621 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:35.749933 21935 rpcz_store.cc:276] 0709 13:07:34.748451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.748608 (+   157us) service_pool.cc:224] Handling call
0709 13:07:35.749295 (+1000687us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400229a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:35.749430 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376884096}
I20260709 13:07:35.742246 27121 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:35.751797 21856 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:35.752352 27530 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.753931 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:35.755563 21856 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:35.756479 27577 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:35.745879 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 865) took 2452 ms. Trace:
I20260709 13:07:35.757053 27577 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:35.757066 21860 rpcz_store.cc:276] 0709 13:07:33.293729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.295798 (+  2069us) service_pool.cc:224] Handling call
0709 13:07:33.296231 (+   433us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:33.758603 (+462372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.745601 (+1986998us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54002b9b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:35.745817 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060398336}
I20260709 13:07:35.758541 27577 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.708123 27743 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:35.599282 27656 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.761795 21854 raft_consensus.cc:2368] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18.
I20260709 13:07:35.764031 27127 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:35.764086 27084 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:35.765522 27399 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:35.765882 27399 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:35.766822 27399 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:35.767764 27614 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.767885 27399 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.764626 27127 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:35.776572 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.781236 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:35.782810 21935 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:35.783985 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:35.784749 21927 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.789978 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:35.615933 27516 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:36.080659 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.086532 21929 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
W20260709 13:07:36.057880 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:36.087726 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.087906 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:36.087759 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.090744 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:36.091789 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.092298 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:36.092389 21929 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:36.093415 21855 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:36.093809 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:35.849423 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.762518 27743 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.838310 27481 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:36.010842 27176 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:36.098182 21931 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:36.089959 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:35.888142 21933 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:35.845160 22593 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:07:35.869966 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.101770 22591 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:36.102268 21930 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:07:36.103992 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:36.097692 21852 raft_consensus.cc:2393] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:36.105310 21930 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:36.009279 27402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.882s	user 0.027s	sys 0.056s
I20260709 13:07:35.873309 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:35.875862 27524 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:35.771037 27645 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:36.104106 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:35.879426 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2853) took 1804 ms (client timeout 2163 ms). Trace:
I20260709 13:07:35.871156 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1303) took 1076 ms. Trace:
I20260709 13:07:35.793051 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:36.107436 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:36.108330 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:35.883946 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.096248 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:36.109280 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:36.122978 21927 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:35.732800 27685 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:36.139986 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.209851 21857 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:36.220131 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:36.132290 21931 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:36.240087 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:36.243252 21929 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:36.248315 22591 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:36.126077 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.249986 21926 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:36.213363 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:36.253079 22000 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:36.238891 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:36.255551 21927 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:36.235667 21930 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:36.239009 22590 rpcz_store.cc:276] 0709 13:07:34.794486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.794614 (+   128us) service_pool.cc:224] Handling call
0709 13:07:34.794894 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.473212 (+678318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.870844 (+397632us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400664254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:35.871057 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509805312}
W20260709 13:07:36.238873 21934 rpcz_store.cc:269] 0709 13:07:34.074847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.827241 (+752394us) service_pool.cc:224] Handling call
0709 13:07:35.879196 (+1051955us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54004295d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:35.879339 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697269504}
I20260709 13:07:36.237499 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:36.235064 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:36.238818 22592 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:36.126991 27516 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:36.260799 27600 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:36.261679 27600 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:36.264343 27600 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:36.265249 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:36.254717 27511 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:36.266337 27511 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:36.268321 27511 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:36.272971 22003 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:36.273470 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:36.275059 27685 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:36.214017 27635 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.000s
I20260709 13:07:36.281037 26697 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:36.281718 27621 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:36.282979 27516 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:36.283370 27516 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:36.284375 27516 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:36.285351 21927 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:36.285583 27516 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:36.285849 27084 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.286366 27084 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:36.288719 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:36.289882 21851 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:36.289929 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55086us
I20260709 13:07:36.269631 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:36.291863 21932 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.778687 27127 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:36.124943 27402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.998s	user 0.028s	sys 0.057s
I20260709 13:07:36.283349 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:36.294767 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 74760.4us
I20260709 13:07:36.295987 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:36.297199 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:36.298162 21934 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
W20260709 13:07:36.323760 27522 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.661s	user 0.005s	sys 0.005s
I20260709 13:07:36.324553 21926 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: [11.11-15.12]
I20260709 13:07:36.300105 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:35.799733 27457 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:36.324317 27522 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:37.021303 21929 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.001s	sys 0.014s
I20260709 13:07:36.323796 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:36.327870 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44600.4us
W20260709 13:07:37.012243 21933 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.005s	sys 0.016s
W20260709 13:07:37.064376 21933 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.017s
W20260709 13:07:37.065359 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 811171us
I20260709 13:07:37.065434 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:36.919687 21935 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.006s	sys 0.015s
W20260709 13:07:37.070827 21935 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.008s	sys 0.016s
W20260709 13:07:36.531077 21972 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.000s	sys 0.016s
W20260709 13:07:37.013087 22009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.009s	sys 0.009s
W20260709 13:07:36.980187 21823 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.008s	sys 0.005s
W20260709 13:07:37.072780 21823 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.009s	sys 0.006s
W20260709 13:07:37.073298 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 835125us
W20260709 13:07:37.061448 21932 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.761s	user 0.013s	sys 0.010s
I20260709 13:07:37.074167 21932 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:36.344501 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:07:37.075284 22009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.010s	sys 0.009s
I20260709 13:07:36.215685 21753 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 6, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 6 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.062492 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2806) took 1477 ms. Trace:
I20260709 13:07:37.078531 22003 rpcz_store.cc:276] 0709 13:07:35.584849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.622464 (+ 37615us) service_pool.cc:224] Handling call
0709 13:07:37.062263 (+1439799us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54001e19d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:37.062418 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494112512}
W20260709 13:07:36.375226 21853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.009s	sys 0.053s
W20260709 13:07:37.079553 21853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.467s	user 0.017s	sys 0.066s
W20260709 13:07:36.777513 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.082015 21972 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.398s	user 0.003s	sys 0.020s
W20260709 13:07:37.082533 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.39969e+06us
W20260709 13:07:37.061448 22000 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.808s	user 0.006s	sys 0.015s
I20260709 13:07:37.083945 22000 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
W20260709 13:07:37.085917 21929 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.001s	sys 0.014s
W20260709 13:07:36.390784 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 55404.3us
I20260709 13:07:36.212929 27627 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.115541 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:37.117113 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 821170us
I20260709 13:07:37.117287 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:36.138335 27481 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.118491 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:37.164227 22007 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:37.006143 27745 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.354s	user 0.000s	sys 0.000s
W20260709 13:07:37.006716 27747 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.407s	user 0.000s	sys 0.001s
I20260709 13:07:37.183635 21859 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:36.463943 27577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.008s	sys 0.030s
W20260709 13:07:37.034343 27516 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.003s
W20260709 13:07:36.342694 27590 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.816s	user 0.010s	sys 0.000s
W20260709 13:07:36.531282 27521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.003s	sys 0.017s
W20260709 13:07:37.194819 27521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.473s	user 0.010s	sys 0.021s
I20260709 13:07:37.195609 21928 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:37.195650 27521 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:37.196218 27521 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.196609 27521 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:36.370553 27511 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:37.197404 27511 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.197573 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2347) took 1524 ms. Trace:
I20260709 13:07:37.197860 27511 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:37.197971 21928 rpcz_store.cc:276] 0709 13:07:35.672804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.672942 (+   138us) service_pool.cc:224] Handling call
0709 13:07:35.750875 (+ 77933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.196154 (+1445279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.196756 (+   602us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400229a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:37.197299 (+   543us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b5400229a50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:37.197509 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3500606464}
W20260709 13:07:37.191761 27577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.429s	user 0.014s	sys 0.044s
W20260709 13:07:36.341683 27614 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.003s
I20260709 13:07:36.367877 27461 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.821s
W20260709 13:07:36.502604 26992 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.000s
I20260709 13:07:37.180449 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1121) took 1342 ms. Trace:
I20260709 13:07:37.181340 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.216778 22587 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:37.217294 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1182) took 1124 ms. Trace:
I20260709 13:07:37.217612 22587 rpcz_store.cc:276] 0709 13:07:36.092434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.092571 (+   137us) service_pool.cc:224] Handling call
0709 13:07:36.325401 (+232830us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:37.181216 (+855815us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aae9c 00007f3b6d6aaa5b 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:37.217235 (+ 36019us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39128320}
W20260709 13:07:36.817531 27399 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.049s	user 0.010s	sys 0.002s
I20260709 13:07:37.005751 27402 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:36.463706 27475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.463s	user 0.014s	sys 0.028s
I20260709 13:07:37.118067 27457 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.072156 27629 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.786409 27638 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:36.325958 27600 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.545361 27176 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:37.255299 27475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.258s	user 0.017s	sys 0.045s
I20260709 13:07:37.269973 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1181) took 1252 ms. Trace:
I20260709 13:07:36.339625 27621 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:36.563742 27701 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:36.481637 27645 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:36.909361 27746 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.277622 21753 catalog_manager.cc:5697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.174803 27627 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.220247 22007 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:37.205255 27590 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.215703 22588 rpcz_store.cc:276] 0709 13:07:35.838307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.838450 (+   143us) service_pool.cc:224] Handling call
0709 13:07:36.326078 (+487628us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:37.179672 (+853594us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aae9c 00007f3b6d6aaa5b 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:37.179845 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.180023 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.180050 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.180169 (+   119us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:37.180203 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.180257 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.180392 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44600960}
I20260709 13:07:37.118920 27656 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:07:37.118834 27524 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
W20260709 13:07:37.206183 27127 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.865s	user 0.005s	sys 0.005s
W20260709 13:07:36.366514 27732 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.645s	user 0.004s	sys 0.004s
I20260709 13:07:37.217999 27577 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.220605 27699 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:37.253116 27745 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:37.272065 22585 rpcz_store.cc:276] 0709 13:07:36.017293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.018060 (+   767us) service_pool.cc:224] Handling call
0709 13:07:37.060970 (+1042910us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:37.254960 (+193990us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aae9c 00007f3b6d6aaa5b 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:37.258870 (+  3910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.259125 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.259166 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.269674 (+ 10508us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:37.269728 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.269792 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.269919 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50223616}
I20260709 13:07:37.274472 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2794) took 1702 ms. Trace:
I20260709 13:07:37.299230 22002 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:37.253407 27399 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.268661 27759 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:36.325289 27657 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:37.266832 27686 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.235760 26992 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:37.292126 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.276288 27629 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.275723 27621 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.274987 27084 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.293857 21753 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 5 to 8, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 8 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:37.221896 27614 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:37.299494 22001 rpcz_store.cc:276] 0709 13:07:35.571916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.572856 (+   940us) service_pool.cc:224] Handling call
0709 13:07:35.573280 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.574452 (+  1172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.574490 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:35.589324 (+ 14834us) log.cc:844] Serialized 26 byte log entry
0709 13:07:35.980677 (+391353us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:35.980738 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.980836 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.274229 (+1293393us) raft_consensus.cc:1685] finished
0709 13:07:37.274249 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.274409 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2150,"raft.queue_time_us":17580,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":390968,"threads_started":1,"wal-append.queue_time_us":917020}
I20260709 13:07:37.274199 27638 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.276539 27600 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.280174 27176 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.314118 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2417) took 1655 ms. Trace:
I20260709 13:07:37.330519 21859 rpcz_store.cc:276] 0709 13:07:35.658236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.659021 (+   785us) service_pool.cc:224] Handling call
0709 13:07:35.659285 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.184361 (+1525076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.313875 (+129514us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54003d3954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:37.314057 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3505040256}
I20260709 13:07:37.283821 27701 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.317715 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:37.305943 27457 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.288055 27746 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.321239 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1307) took 2256 ms. Trace:
I20260709 13:07:37.304663 27657 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.297412 27732 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:37.326959 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.316906 27621 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.298635 27127 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.323508 21753 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 2 to 8, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 8 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:07:37.302419 27577 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.361406 27176 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.307343 27399 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.333916 22584 rpcz_store.cc:276] 0709 13:07:35.062095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.064502 (+  2407us) service_pool.cc:224] Handling call
0709 13:07:35.158489 (+ 93987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.466511 (+308022us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:35.477741 (+ 11230us) log.cc:844] Serialized 14 byte log entry
0709 13:07:36.084643 (+606902us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:36.085295 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:07:36.085475 (+   180us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:36.085520 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.085628 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.317957 (+1232329us) raft_consensus.cc:1685] finished
0709 13:07:37.317976 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.318104 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2302,"prepare.queue_time_us":12037,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":370094,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":60072,"thread_start_us":663867,"threads_started":2,"wal-append.queue_time_us":1050318}
I20260709 13:07:37.302663 27699 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.308240 27686 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.295234 27402 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:37.362279 27751 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:07:37.334089 27522 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.320387 27759 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:07:37.395064 27399 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.392161 22589 raft_consensus.cc:2368] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15.
W20260709 13:07:37.397908 22589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1119) took 1621 ms (client timeout 1679 ms). Trace:
W20260709 13:07:37.398316 22589 rpcz_store.cc:269] 0709 13:07:35.776070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.776230 (+   160us) service_pool.cc:224] Handling call
0709 13:07:37.397713 (+1621483us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400531954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:37.397854 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713215744}
I20260709 13:07:37.329358 27614 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.400949 27614 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.401383 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2884) took 1705 ms. Trace:
I20260709 13:07:37.401754 22002 rpcz_store.cc:276] 0709 13:07:35.696318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.707593 (+ 11275us) service_pool.cc:224] Handling call
0709 13:07:35.707830 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.303230 (+1595400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.401154 (+ 97924us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540038a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:37.401328 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3655939840}
I20260709 13:07:37.363559 27657 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.403183 22000 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:37.402542 27600 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.382124 27457 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:37.389772 21753 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: config changed from index 6 to 2, term changed from 9 to 6, leader changed from <none> to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from VOTER to NON_VOTER. New cstate: current_term: 6 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:07:37.406029 27748 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 13:07:37.408784 27756 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:07:37.409307 27638 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.410457 27457 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:37.419878 27577 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.419710 27457 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.420714 27511 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.421568 27521 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.423125 27732 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.423539 21931 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:37.424219 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2863) took 1341 ms. Trace:
I20260709 13:07:37.424614 21931 rpcz_store.cc:276] 0709 13:07:36.083014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.241894 (+158880us) service_pool.cc:224] Handling call
0709 13:07:37.423040 (+1181146us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:37.423228 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.423942 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.424166 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6742272}
I20260709 13:07:37.423048 27176 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:37.423738 27521 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.441556 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2895) took 1157 ms. Trace:
I20260709 13:07:37.442049 22009 rpcz_store.cc:276] 0709 13:07:36.283874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.284035 (+   161us) service_pool.cc:224] Handling call
0709 13:07:36.284344 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:36.284542 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:36.284569 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:36.285083 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:07:37.075806 (+790723us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:37.075861 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.075954 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.441340 (+365386us) raft_consensus.cc:1685] finished
0709 13:07:37.441360 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.441497 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":790299,"threads_started":1,"wal-append.queue_time_us":990114}
I20260709 13:07:37.436010 27686 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.475565 27511 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.478149 27629 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.437505 27627 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.434834 27176 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.493105 26992 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.494333 27627 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.520709 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2870) took 1259 ms. Trace:
I20260709 13:07:37.521415 27701 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.521412 21929 rpcz_store.cc:276] 0709 13:07:36.260855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.260970 (+   115us) service_pool.cc:224] Handling call
0709 13:07:36.261315 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:36.261521 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:37.086895 (+825374us) log.cc:844] Serialized 14 byte log entry
0709 13:07:37.520322 (+433427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.520379 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:37.520422 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.520489 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.520635 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1009431,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":674,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":113,"thread_start_us":1257362,"threads_started":2}
I20260709 13:07:37.513799 27746 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.513597 27590 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.525552 27402 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.546551 27127 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.525935 27627 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.641264 27176 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:37.661763 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.005s	sys 0.002s
W20260709 13:07:37.663802 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.005s	sys 0.002s
W20260709 13:07:37.666889 21853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.001s	sys 0.005s
W20260709 13:07:37.667341 21853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.001s	sys 0.005s
I20260709 13:07:37.668771 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1322) took 1376 ms. Trace:
I20260709 13:07:37.669234 22586 rpcz_store.cc:276] 0709 13:07:36.292798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.119179 (+826381us) service_pool.cc:224] Handling call
0709 13:07:37.183136 (+ 63957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.668314 (+485178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.668359 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.668425 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:37.668469 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.668550 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.668699 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":203601,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":12640,"thread_start_us":484653,"threads_started":1}
W20260709 13:07:37.670956 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.001s	sys 0.005s
W20260709 13:07:37.671370 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.001s	sys 0.005s
I20260709 13:07:37.664842 27627 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:37.683665 21744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 9fc728770e5c4aa1ac36a662cebb8598 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 34f3b83da84448bf8aac62b37a1486eb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:37.685084 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.002s	sys 0.005s
W20260709 13:07:37.685695 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.002s	sys 0.005s
W20260709 13:07:37.686465 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.34191e+06us
W20260709 13:07:37.662992 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.006s	sys 0.001s
W20260709 13:07:37.690441 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.006s	sys 0.001s
I20260709 13:07:37.695421 27629 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.696175 27629 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.698644 27629 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.710137 27768 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:37.711313 27768 consensus_queue.cc:1243] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:37.712014 27402 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:37.714075 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:37.712503 27402 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:37.715147 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.036s	sys 0.039s
W20260709 13:07:37.715982 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.008s	sys 0.001s
W20260709 13:07:37.716037 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.036s	sys 0.039s
W20260709 13:07:37.714783 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.054s	sys 0.059s
W20260709 13:07:37.716969 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.054s	sys 0.060s
I20260709 13:07:37.719980 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:37.721115 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:37.715608 21826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.006s	sys 0.001s
W20260709 13:07:37.723078 21826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.007s	sys 0.002s
W20260709 13:07:37.724191 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.716650 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.008s	sys 0.001s
I20260709 13:07:37.727015 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:37.727347 22589 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:37.730548 21974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.011s
W20260709 13:07:37.731135 21974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.000s	sys 0.011s
I20260709 13:07:37.722308 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:37.731477 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 640246us
I20260709 13:07:37.731338 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:37.730966 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.734994 22586 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
W20260709 13:07:37.733421 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43747.5us
I20260709 13:07:37.741551 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:37.751653 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.753638 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:37.755329 21935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.002s	sys 0.011s
W20260709 13:07:37.756843 21935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.002s	sys 0.011s
I20260709 13:07:37.757429 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:37.758733 22002 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:37.761261 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:37.763043 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:37.714807 27627 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.742578 27686 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:37.730522 27176 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:37.730329 27577 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.730948 26992 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.730297 27701 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:37.743458 27127 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:37.778563 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.795368 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2411) took 2809 ms. Trace:
W20260709 13:07:37.755681 21823 connection.cc:256] RPC call timeout handler was delayed by -1.3840187119999428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
I20260709 13:07:37.779831 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:37.780118 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.744828 27521 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:37.761313 27576 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:37.798106 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.807448 22002 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:37.807505 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.808256 22004 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
W20260709 13:07:37.808677 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.007s	sys 0.007s
I20260709 13:07:37.799831 22003 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
W20260709 13:07:37.808971 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.007s	sys 0.007s
W20260709 13:07:37.793212 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.744900 27758 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.768962 27399 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:37.765376 27600 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:37.795707 22007 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
W20260709 13:07:37.809284 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 639363us
W20260709 13:07:37.809746 21933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.008s	sys 0.006s
W20260709 13:07:37.810000 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.003s	sys 0.011s
I20260709 13:07:37.768942 27522 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:37.802866 27645 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:37.767267 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:37.771871 27743 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:37.771533 27475 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:37.835501 27475 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.835101 27743 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.800866 26992 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.810442 27752 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.835891 27475 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.836236 27752 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.838276 26992 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.838481 27752 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.827008 27521 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.839072 27767 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:37.839488 27767 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.839428 27521 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.833600 27522 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.841785 27521 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.813769 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.815373 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.843791 22008 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:37.744768 27629 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:07:37.797109 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:37.845625 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78454us
I20260709 13:07:37.846184 22001 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:37.839931 27767 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.804939 27701 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.805083 27127 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.800443 27176 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.849278 27127 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.849861 27701 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.771878 27481 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.850760 27176 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.850526 27481 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.806567 27576 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:37.800562 27577 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.843075 22005 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:37.813544 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:37.813548 27399 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.774955 27627 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.818302 27758 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.857990 27576 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.860181 27576 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:37.854547 27629 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:37.862831 27629 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:37.841120 27522 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.819416 27402 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:37.867141 27457 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.870756 27481 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.872015 27577 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.853415 27524 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:37.856570 27699 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.873386 27524 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.873821 27524 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.854650 27621 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.876312 27621 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.846647 27645 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.872175 27657 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.877557 27759 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.878095 27759 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.851305 27176 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:37.879014 27645 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.856657 27590 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:37.880208 27590 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.858053 27399 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:37.880877 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56050.3us
I20260709 13:07:37.882562 27590 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.884007 27743 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.845326 27600 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:37.884883 27758 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.863989 27629 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.885877 27629 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.885653 27600 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.816231 27686 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:37.889848 27770 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:37.889622 27686 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.890386 27770 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.892429 27770 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.880905 27627 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.895450 27759 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.896231 27743 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:37.896924 27399 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.898586 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:37.899302 27399 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.900233 27657 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.902377 22584 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:37.904058 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.905738 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.906380 22584 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 13: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:37.907161 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.907734 22586 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:37.887616 27475 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.911290 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:37.911621 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:37.916631 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:37.917495 27770 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:37.909080 27743 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:37.897820 27522 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.908850 27776 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:37.879103 27176 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.923835 27176 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:37.923864 27776 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:37.924561 27776 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.924597 27767 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:37.925424 27475 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.918066 27759 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:37.926349 27524 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.927618 27767 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.928521 27524 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:37.931874 27522 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:37.934108 27776 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:37.936224 27776 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:37.796332 21823 connection.cc:256] RPC call timeout handler was delayed by -1.3534187119998933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
W20260709 13:07:37.972158 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.973664 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 178565us
I20260709 13:07:37.977758 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:37.825407 21933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.009s	sys 0.006s
I20260709 13:07:37.795789 21853 rpcz_store.cc:276] 0709 13:07:34.986019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.236967 (+250948us) service_pool.cc:224] Handling call
0709 13:07:35.612279 (+375312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.612498 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:35.612790 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:07:37.667722 (+2054932us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.667803 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:37.667880 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.667973 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.795167 (+127194us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54003d3e50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:37.795294 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56319,"prepare.queue_time_us":567,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":173073,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":862311680,"thread_start_us":1998095,"threads_started":2,"wal-append.queue_time_us":1580602}
I20260709 13:07:37.983527 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2868) took 1756 ms. Trace:
I20260709 13:07:37.984030 21933 rpcz_store.cc:276] 0709 13:07:36.227259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.290365 (+ 63106us) service_pool.cc:224] Handling call
0709 13:07:36.290650 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:36.290851 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:37.064888 (+774037us) log.cc:844] Serialized 14 byte log entry
0709 13:07:37.982442 (+917554us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.982612 (+   170us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:37.982768 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.982928 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.983267 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":989971,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"thread_start_us":1643466,"threads_started":2,"wal-append.queue_time_us":782389}
I20260709 13:07:37.987779 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2355) took 2171 ms. Trace:
W20260709 13:07:37.825037 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.004s	sys 0.011s
I20260709 13:07:37.988363 21935 rpcz_store.cc:276] 0709 13:07:35.816494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.816696 (+   202us) service_pool.cc:224] Handling call
0709 13:07:35.817119 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:36.105800 (+288681us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:37.071687 (+965887us) log.cc:844] Serialized 14 byte log entry
0709 13:07:37.986811 (+915124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.986961 (+   150us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:37.987121 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.987244 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.987528 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5796,"prepare.queue_time_us":1122472,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4357,"raft.queue_time_us":94074,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"thread_start_us":1873864,"threads_started":3,"wal-append.queue_time_us":797539}
W20260709 13:07:37.990106 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 925935us
W20260709 13:07:37.993548 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 238079us
I20260709 13:07:37.996949 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:38.018146 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.018707 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:37.998872 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.019236 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:38.001497 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 240674us
W20260709 13:07:37.997746 21823 connection.cc:256] RPC call timeout handler was delayed by -1.0064568339998914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:07:38.010555 22556 connection.cc:256] RPC call timeout handler was delayed by -1.7311036289999038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
I20260709 13:07:38.024042 27399 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.025789 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.025424 27399 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.026820 22584 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:38.033098 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:38.034396 22001 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:38.033360 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:38.038203 27765 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:38.039448 27765 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.020119 22585 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.014246 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.046386 27765 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:38.046015 27629 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:38.009644 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 254443us
I20260709 13:07:38.056456 27629 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.060859 27475 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:38.061528 27629 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.062300 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:38.063851 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:38.065263 27743 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:38.065927 27743 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:38.067440 27743 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.070627 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:38.071488 22584 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.073592 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:38.074635 22586 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.076452 27576 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:38.077422 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 335822us
I20260709 13:07:38.073634 27522 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:07:38.078858 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:38.079082 27522 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:38.073787 27577 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:38.079576 27522 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:38.079886 27577 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.080173 27789 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:38.080435 27577 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.080734 27789 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.081457 27475 consensus_queue.cc:1243] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:38.081393 27789 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.021910 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:38.013139 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:38.090271 27686 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:38.090503 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.097569 27686 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.100979 27576 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.090927 27475 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.105536 27475 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:38.107357 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 95781.2us
I20260709 13:07:38.108341 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:38.109117 27600 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.109365 27645 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:38.109875 27645 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:38.109908 27600 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.112596 27600 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.110376 27645 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:38.121738 27399 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:38.129590 27763 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:38.130178 21972 leader_election.cc:400] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18.
I20260709 13:07:38.130472 27763 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.130779 21972 leader_election.cc:403] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:07:38.131142 27763 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:38.133769 27657 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:38.134739 27657 consensus_queue.cc:1243] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:38.137409 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:38.138108 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 125333us
I20260709 13:07:38.138603 27657 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:38.139132 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:38.139647 27657 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:38.140130 27765 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:38.154563 27686 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:38.081617 27522 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.141252 27657 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:38.199868 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:38.200874 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78130.2us
I20260709 13:07:38.202919 22585 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:38.205209 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:38.206660 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 84075.8us
I20260709 13:07:38.207592 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:38.208514 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71327us
I20260709 13:07:38.211174 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:38.210141 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:38.215201 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.215705 22006 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:38.199059 27576 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:38.211453 27765 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:38.226233 27522 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.229836 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.230944 21853 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.233974 22586 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:38.214629 27475 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.242074 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:38.243739 27770 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:38.253319 27785 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.254556 27516 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:38.255101 27516 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:38.255517 27516 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.256906 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:38.254592 27614 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:07:38.258137 27521 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:38.258740 27782 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:38.259227 27782 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:38.259548 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 118663us
I20260709 13:07:38.260593 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:38.257086 27576 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:38.263459 27576 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.263679 27521 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:38.263924 27576 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.264626 21935 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:38.263554 27765 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:07:38.242656 27783 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.272521 21859 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:38.260655 27782 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:38.296367 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.297649 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.301956 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.302649 27782 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.303532 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.304248 21854 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:38.305541 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.306408 21857 raft_consensus.cc:2368] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19.
I20260709 13:07:38.307314 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.307910 21857 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:38.309094 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.310477 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.311743 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.312897 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:38.313364 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.313887 21856 raft_consensus.cc:2368] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:38.314981 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.315644 21856 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:38.316079 21853 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:38.316479 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:38.317363 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78044.1us
I20260709 13:07:38.318253 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:38.319190 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:38.320061 27399 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:38.320441 27399 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.320804 27399 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.321300 27686 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:38.321735 27686 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:38.322185 27686 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:38.323179 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71059.7us
W20260709 13:07:38.323743 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.325524 21974 leader_election.cc:400] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15.
I20260709 13:07:38.325440 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:38.325941 21974 leader_election.cc:403] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:38.326740 27127 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:38.328104 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:38.329383 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:38.329723 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.330317 21931 raft_consensus.cc:2368] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:38.321789 27789 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.317665 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.331188 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.278821 27629 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.335269 27779 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15.
I20260709 13:07:38.336472 27127 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:38.337792 27127 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:38.338155 27127 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:38.338781 27763 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.339610 27781 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:38.340083 27781 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:38.339207 27127 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.341729 27127 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:38.346292 27791 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:07:38.348917 21854 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
W20260709 13:07:38.365818 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 153118us
W20260709 13:07:38.370388 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.373706 21856 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:38.308689 27577 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.377614 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.377949 21935 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:38.378321 21854 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
W20260709 13:07:38.385423 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 60074.7us
I20260709 13:07:38.320591 27645 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.391275 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:38.401569 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 141127us
I20260709 13:07:38.401487 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.425465 22584 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:38.421382 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.433351 21860 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:38.434934 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.437263 27614 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.438026 27614 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.439739 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.440342 27614 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:38.441135 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.441845 21860 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:38.443990 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.445421 27402 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.401538 27763 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.456609 21851 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:38.455897 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:38.460197 21857 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.470047 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.470719 27781 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.471693 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.472409 27781 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:38.473832 21931 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:07:38.473971 27576 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.475275 21858 raft_consensus.cc:2368] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:38.477348 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.478104 21856 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.483511 27780 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.485303 27621 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.485930 27621 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.488322 27621 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.490830 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.491623 21928 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:38.492841 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.493955 21930 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:38.494489 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:38.495039 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.495270 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.496470 21926 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:38.502652 27516 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.503996 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.504874 21935 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:38.505314 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.506024 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.506669 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.507474 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.507925 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.508299 21932 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:38.510542 27765 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:38.511201 27765 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.512120 27743 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:38.512668 27577 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:38.511358 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 126465us
I20260709 13:07:38.514744 27789 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.511327 27614 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:38.516772 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.518199 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.519280 27576 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.484526 27780 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:38.520112 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:38.521313 27765 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.522200 21930 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.522200 27780 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.525579 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.526652 21934 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:38.527679 27645 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:38.462992 27481 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:38.528666 27481 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.529211 27481 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.508821 21854 raft_consensus.cc:2368] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17.
I20260709 13:07:38.530299 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.530572 27629 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:38.531164 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.531644 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.528587 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:38.531925 21926 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:38.529974 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.550128 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.552821 21860 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:38.529934 27516 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.554896 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:38.551048 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:38.556370 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.565153 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:38.566493 21859 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.568228 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.569345 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.570041 21935 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:38.571225 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:38.572368 27600 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:38.572908 27600 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:38.488547 27686 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.573423 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:38.573958 27600 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.574982 27686 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:38.575752 27686 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:38.576917 27686 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:38.620051 27686 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:38.581204 22590 raft_consensus.cc:2368] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
W20260709 13:07:38.621925 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 64708.5us
I20260709 13:07:38.581244 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:38.622754 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:38.583096 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.583413 21930 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:38.512631 27743 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:38.504774 27399 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:38.589787 22000 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:39.075999 21859 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:07:38.949868 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.189090 21858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.019s	sys 0.003s
I20260709 13:07:38.598491 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:38.715457 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 92777.5us
I20260709 13:07:38.721464 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:38.679415 22588 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:38.599181 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:39.005121 21973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.022s	sys 0.007s
I20260709 13:07:38.968909 21933 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
W20260709 13:07:39.193331 21973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.025s	sys 0.013s
W20260709 13:07:39.194165 21857 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.662s	user 0.018s	sys 0.021s
I20260709 13:07:39.194727 21857 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
W20260709 13:07:38.756699 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.012409 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.048713 21974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.025s	sys 0.101s
W20260709 13:07:39.197511 21974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.029s	sys 0.105s
W20260709 13:07:38.618232 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 51858.7us
W20260709 13:07:38.594185 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 136094us
W20260709 13:07:39.191028 21858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.020s	sys 0.004s
W20260709 13:07:38.579910 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.610889 21856 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:07:39.202211 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.714562 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.004333 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:38.617185 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.679631 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:38.624234 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:38.849157 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.193542 21933 pending_rounds.cc:85] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 10
W20260709 13:07:39.206818 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.597683 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:39.156688 27781 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.684s	user 0.016s	sys 0.000s
W20260709 13:07:39.098213 27402 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.652s	user 0.008s	sys 0.004s
I20260709 13:07:38.595988 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:39.187738 27791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.003s	sys 0.010s
I20260709 13:07:39.217337 21856 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:07:39.223168 21859 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:38.880621 27770 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:38.824079 27782 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.000s
I20260709 13:07:38.599143 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:38.715338 27775 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:39.206459 21933 pending_rounds.cc:107] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 15.11
W20260709 13:07:39.089147 27786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.027s	sys 0.100s
W20260709 13:07:39.221700 27127 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.879s	user 0.012s	sys 0.001s
W20260709 13:07:39.211284 27481 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.007s
I20260709 13:07:39.236951 22593 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:39.237259 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:39.225821 21857 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [11.10-14.12]
I20260709 13:07:39.226006 21855 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:07:38.592031 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:39.227541 21926 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:39.211062 27779 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:39.229205 21933 raft_consensus.cc:2889] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:39.230227 21860 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:39.217213 27791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.003s	sys 0.010s
I20260709 13:07:39.233287 21854 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 9.
I20260709 13:07:39.216439 27780 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.230757 27786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.029s	sys 0.116s
I20260709 13:07:39.242587 21931 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:39.239573 22586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.988s	user 0.010s	sys 0.027s
W20260709 13:07:39.251825 22586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.010s	sys 0.027s
I20260709 13:07:39.235217 27576 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.245954 27785 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.991s	user 0.005s	sys 0.008s
I20260709 13:07:39.236670 27762 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:39.262782 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:39.262547 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:39.263995 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:39.242952 27781 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.268652 22585 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:39.270875 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 647337us
I20260709 13:07:39.271939 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:39.272989 22593 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:39.244257 27614 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.276954 21935 raft_consensus.cc:1217] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 6.7->[7.8-14.12]   Dedup: 11.10->[12.11-14.12]
I20260709 13:07:39.285507 21935 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [7.8-14.12]
I20260709 13:07:39.242182 27481 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.286693 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.290105 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:39.290247 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:39.291285 21852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.006s	sys 0.010s
I20260709 13:07:39.294132 21933 raft_consensus.cc:2376] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16. Re-sending same reply.
W20260709 13:07:39.294356 21852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.006s	sys 0.010s
I20260709 13:07:39.295459 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2426) took 1023 ms. Trace:
I20260709 13:07:39.295627 27475 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:39.302268 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:39.304749 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:39.302209 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:39.247922 27770 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.307905 22585 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:39.277722 27785 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18.
I20260709 13:07:39.315039 21853 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:39.322917 21860 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:39.322784 21854 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:39.327047 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:39.327788 21858 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:39.328794 21933 raft_consensus.cc:2410] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:07:39.331979 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:39.330219 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.283706 27779 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.340327 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 9.
I20260709 13:07:39.345157 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.345587 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.280692 27743 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.306195 27576 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.334467 27786 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:39.360764 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88919.2us
I20260709 13:07:39.357470 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:39.373550 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:39.374610 21857 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:39.375530 22005 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:39.375594 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:39.380343 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.383311 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.370640 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2772) took 1030 ms. Trace:
I20260709 13:07:39.385237 27775 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:39.292438 27781 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.319905 27780 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.389482 21851 rpcz_store.cc:276] 0709 13:07:38.329754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.951685 (+621931us) service_pool.cc:224] Handling call
0709 13:07:39.246730 (+295045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.250984 (+  4254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:39.251011 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:39.251055 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:39.251092 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.251148 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.359914 (+108766us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b5400017050. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:39.360116 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73834880}
I20260709 13:07:39.357419 27399 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:39.344694 27127 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:39.392498 21857 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:39.399330 21858 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:39.395224 22593 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:39.332407 27614 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.400280 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:39.401518 21857 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:07:39.401770 27614 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:39.405534 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.404784 27127 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.410426 21860 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:39.403932 27686 raft_consensus.cc:2764] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:39.416320 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.421262 27779 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:39.405468 27785 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.412237 27127 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:39.415100 27776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.017s	sys 0.023s
I20260709 13:07:39.426285 22001 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
W20260709 13:07:39.429425 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 97565.9us
I20260709 13:07:39.433049 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1331) took 1194 ms. Trace:
I20260709 13:07:39.433420 22586 rpcz_store.cc:276] 0709 13:07:38.238902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.243372 (+  4470us) service_pool.cc:224] Handling call
0709 13:07:38.249535 (+  6163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.249977 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:38.250031 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:38.250801 (+   770us) log.cc:844] Serialized 26 byte log entry
0709 13:07:39.264623 (+1013822us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:39.264669 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.264749 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.432746 (+167997us) raft_consensus.cc:1685] finished
0709 13:07:39.432767 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.432916 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1001025,"threads_started":1,"wal-append.queue_time_us":1122784}
I20260709 13:07:39.434310 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.434840 27781 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:39.432545 27815 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.435778 27481 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:39.436484 27815 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:39.435122 22005 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:39.427762 27776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.018s	sys 0.023s
I20260709 13:07:39.437685 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.430276 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.439271 22000 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:39.439880 27785 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:39.440799 27785 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:39.441264 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.442022 22006 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:39.442042 27785 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.443373 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.444061 22004 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:39.442886 22002 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:39.450165 27770 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:39.435853 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.450866 27776 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:39.451352 27776 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:39.452250 27780 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:39.295809 21852 rpcz_store.cc:276] 0709 13:07:38.271947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.272750 (+   803us) service_pool.cc:224] Handling call
0709 13:07:38.273180 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.273477 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:38.375170 (+101693us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.295126 (+919956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:39.295177 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:39.295227 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.295302 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.295406 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10012,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":11121,"raft.queue_time_us":13366,"thread_start_us":973193,"threads_started":2}
I20260709 13:07:39.451416 22007 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:39.454574 22000 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:39.452350 27127 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:39.466727 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2435) took 1150 ms. Trace:
I20260709 13:07:39.467154 21853 rpcz_store.cc:276] 0709 13:07:38.303845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.316795 (+ 12950us) service_pool.cc:224] Handling call
0709 13:07:39.454426 (+1137631us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b540008b1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:39.454614 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2256556800}
I20260709 13:07:39.478664 22006 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:39.457716 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:39.460826 27809 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:07:39.485783 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.437244 27815 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:39.445706 27576 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:39.488184 22003 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:39.455822 27481 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.456166 27127 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:39.490696 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:39.488790 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58505.9us
I20260709 13:07:39.491519 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.495011 27779 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:39.541286 22006 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:39.500242 27812 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:07:39.491734 27127 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:39.532891 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.495052 27481 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:39.539949 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 208516us
I20260709 13:07:39.540699 21974 leader_election.cc:400] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:39.541378 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.452477 27776 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.535851 21972 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.549876 22001 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:39.551096 21974 leader_election.cc:403] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:07:39.551541 27475 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:39.578482 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.581365 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49013.9us
W20260709 13:07:39.592321 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.569986 27776 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.600345 21972 leader_election.cc:400] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:39.600682 21972 leader_election.cc:403] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:07:39.597508 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 364897us
I20260709 13:07:39.569270 27686 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.572774 27775 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:39.603377 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.606007 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:39.606766 21933 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:39.595253 27815 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.582821 27806 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:39.611770 27127 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.613298 27127 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:39.614246 27481 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:39.615023 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.617460 27822 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.618145 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.618337 27822 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:39.604386 27780 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:39.619777 27780 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.622017 27780 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:39.622988 27481 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:39.623206 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1132) took 1028 ms. Trace:
I20260709 13:07:39.623529 27481 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:39.623618 22589 rpcz_store.cc:276] 0709 13:07:38.594586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.598929 (+  4343us) service_pool.cc:224] Handling call
0709 13:07:39.623005 (+1024076us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54005dcad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:39.623150 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871633152}
I20260709 13:07:39.623075 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:39.625355 27818 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:39.626008 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.625923 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:39.626895 27818 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.627308 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.630172 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.630301 27818 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:39.630785 27818 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:39.631168 27818 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:39.631435 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.632274 27779 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:39.632400 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.624070 27481 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:39.638358 21898 leader_election.cc:400] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:39.638839 21898 leader_election.cc:403] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:39.639945 22003 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:39.640789 22006 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:39.648738 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2443) took 1101 ms. Trace:
I20260709 13:07:39.643913 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.652489 22007 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:07:39.652590 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49386.9us
I20260709 13:07:39.656735 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.657323 27804 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:39.657805 27804 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.658151 27804 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:39.661453 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:39.662340 22009 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:39.665163 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50238.8us
I20260709 13:07:39.666498 27824 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:39.667192 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.655722 21900 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.674700 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:39.678340 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:39.678712 22589 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:39.679296 22588 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:39.680475 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:39.681409 22587 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:39.682379 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:39.654937 27780 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:39.683271 22592 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:39.651031 27782 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:39.687577 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.649451 21859 rpcz_store.cc:276] 0709 13:07:38.547484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.229964 (+682480us) service_pool.cc:224] Handling call
0709 13:07:39.247532 (+ 17568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.284742 (+ 37210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:39.550074 (+265332us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.648302 (+ 98228us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:39.648361 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:39.648402 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.648465 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.648669 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2373,"prepare.queue_time_us":337894,"thread_start_us":262604,"threads_started":2}
I20260709 13:07:39.691624 22007 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:07:39.691779 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88651us
I20260709 13:07:39.692763 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.691944 27818 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.699738 22001 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:39.639921 27614 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:39.700235 21854 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:39.701690 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2442) took 1166 ms. Trace:
W20260709 13:07:39.701963 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61799.7us
I20260709 13:07:39.693315 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:39.707473 22591 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:07:39.702191 21856 rpcz_store.cc:276] 0709 13:07:38.534843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.227940 (+693097us) service_pool.cc:224] Handling call
0709 13:07:39.242445 (+ 14505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.267816 (+ 25371us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:39.539621 (+271805us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.701239 (+161618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:39.701320 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:39.701378 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.701465 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.701622 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46615,"prepare.queue_time_us":292701,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":1881,"thread_start_us":313579,"threads_started":2}
I20260709 13:07:39.712180 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:39.714771 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.714617 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.719659 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.728227 22584 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:39.643558 27770 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.064s
W20260709 13:07:39.733480 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79696.9us
W20260709 13:07:39.737169 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 104842us
I20260709 13:07:39.737726 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:39.633970 27686 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:39.715512 27824 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:39.745409 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 89747.8us
I20260709 13:07:39.746994 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.737169 27645 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:39.720264 27785 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:07:39.746752 27770 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:39.770176 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 103105us
I20260709 13:07:39.751382 27475 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:39.777752 22009 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:39.778887 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:39.771727 27645 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:39.780037 27838 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:39.787600 27834 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:39.788240 27834 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:39.775583 27770 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:39.763005 27815 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:39.770781 27820 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:39.795502 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 75132.1us
I20260709 13:07:39.795993 27820 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:39.796836 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.781136 27475 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.797854 27475 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.798163 27770 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.799441 27770 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:39.801617 27835 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:39.802119 27835 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:39.796689 27820 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:39.770663 27825 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:07:39.806782 27835 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.773509 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:39.770573 27827 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:39.826323 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 91835.8us
I20260709 13:07:39.774078 27522 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.839601 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.794355 27839 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:39.817708 27785 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.848323 21823 leader_election.cc:336] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.594s (SENT)
W20260709 13:07:39.852559 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 159809us
I20260709 13:07:39.853873 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.856792 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:39.828356 27827 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:39.859338 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 96410.2us
I20260709 13:07:39.833822 27835 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:39.860073 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.847630 27614 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:39.866053 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 149712us
W20260709 13:07:39.870455 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.863725 27762 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:39.875162 27846 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:39.876513 27815 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:39.879401 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 116985us
I20260709 13:07:39.861680 27827 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:39.870313 27825 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:39.890012 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.870018 27522 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:07:39.891193 27686 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:39.828521 27645 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.867978 27614 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:39.860234 27785 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.871825 27839 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:39.918633 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 180988us
I20260709 13:07:39.877375 27762 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:39.841109 27834 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:39.925716 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.954346 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 97647.6us
I20260709 13:07:39.955925 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:39.922851 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:39.958825 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.959924 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.922545 27614 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.928009 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.919083 27838 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:39.979061 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.965888 27836 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:39.972906 27820 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.986881 27836 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.987581 27836 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:39.988108 27838 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:39.958114 27762 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:39.955381 27818 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:39.989176 27820 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:39.921789 27780 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:39.966040 27839 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:39.981222 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 58454.4us
I20260709 13:07:39.991755 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:39.922017 27846 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:39.992657 27818 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:39.993881 27846 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:39.992378 27839 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:39.994534 27818 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:39.995515 27614 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:39.965380 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 131852us
I20260709 13:07:39.965561 27845 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:40.002096 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 74188.9us
I20260709 13:07:40.003043 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:40.004035 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:39.891211 27832 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:07:39.997900 27818 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:40.033638 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77789.1us
I20260709 13:07:40.033814 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:40.035780 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:40.035928 27833 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:07:40.035853 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 175840us
I20260709 13:07:40.020958 27845 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:40.039537 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:40.035342 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:40.036580 27686 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:40.064256 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:40.065119 27824 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:40.066709 27804 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.038388 27127 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.068553 27781 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:40.069447 27781 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:40.071091 27781 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.072243 27781 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.051403 27845 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:40.073534 27845 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:40.068610 27127 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.069303 27576 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:40.075765 27576 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:40.045727 27852 raft_consensus.cc:2764] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:40.046653 27847 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:40.077561 27576 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.065354 27475 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.045166 27782 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:40.081478 27782 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.084299 27860 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:40.085692 27860 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:40.090606 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.086323 27860 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:40.106932 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45269.4us
I20260709 13:07:40.111999 27852 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:40.115001 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:40.115902 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 111970us
I20260709 13:07:40.116263 27864 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:40.128882 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 71592.5us
I20260709 13:07:40.139067 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:40.108230 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:40.049837 27522 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:40.139806 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52642.5us
I20260709 13:07:40.098309 27475 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:40.148818 27857 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:40.161578 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 139012us
I20260709 13:07:40.149384 27804 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:40.165015 27839 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:40.165728 27847 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.166231 27127 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:40.166332 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.152443 27822 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:40.166249 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:40.152609 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44510.7us
I20260709 13:07:40.168928 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:40.171236 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.119408 27860 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:40.178948 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:40.179849 21856 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:40.198554 21859 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:40.190582 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:40.179132 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 57743.5us
W20260709 13:07:40.203454 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.181890 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 147459us
W20260709 13:07:40.184403 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.196254 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:40.208391 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:40.041834 27815 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.176656 27614 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:40.209230 22007 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.228575 21855 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:40.231242 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:40.151775 27864 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.232410 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:40.232772 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:40.233381 21853 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:40.186148 27820 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:40.234563 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:40.234231 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:40.237176 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:40.209713 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:40.209818 22590 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:40.246290 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:40.181902 27839 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:40.208379 27846 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:40.240012 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:40.233774 27855 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:07:40.191130 27522 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:40.259230 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:07:40.254112 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 54599.7us
I20260709 13:07:40.259573 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:40.267199 22002 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:40.272337 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:40.256330 21935 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:40.264523 27864 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:40.250242 27785 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:40.252307 27600 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.259341 27820 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:40.234192 27822 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:40.249274 27857 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.294871 21932 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:40.296159 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.296617 27851 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 13:07:40.297817 27869 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:40.210090 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:40.297637 27840 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:40.275413 27846 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.321331 27840 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:40.322422 27840 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.323352 27846 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:40.277712 27127 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:40.324062 27127 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.324460 27127 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:40.190105 27847 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.332445 27820 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.337491 27869 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:40.338331 27522 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.343468 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:40.306982 27600 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.345603 27822 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.349131 27858 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:07:40.345171 27866 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:40.353329 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59031.3us
I20260709 13:07:40.215010 27818 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:40.350859 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.344718 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:40.381044 22005 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:40.318470 22590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
W20260709 13:07:40.424604 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.513711 22590 raft_consensus.cc:606] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:40.410218 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:40.518200 22590 raft_consensus.cc:614] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:40.416414 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:40.402477 27835 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.519s	user 0.016s	sys 0.000s
W20260709 13:07:40.531687 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 115388us
I20260709 13:07:40.339505 27869 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:40.411933 27868 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:07:40.355163 27847 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:40.585575 27853 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:07:40.587366 27872 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:07:40.599776 21823 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.550s (SENT)
I20260709 13:07:40.600384 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:40.361369 27857 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.607333 27857 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:40.613708 27856 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.614677 27856 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:40.616101 27781 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.004s
I20260709 13:07:40.553905 27475 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:40.645092 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.647553 22007 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
W20260709 13:07:40.655874 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 230402us
I20260709 13:07:40.658460 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:40.664904 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:40.612394 27825 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.003s
W20260709 13:07:40.636025 27779 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.003s	user 0.012s	sys 0.004s
I20260709 13:07:40.560169 27645 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.123s
I20260709 13:07:40.702441 21851 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:40.722769 22002 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:40.635560 27475 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:40.725569 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 125841us
W20260709 13:07:40.729595 21823 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.580s (SENT)
I20260709 13:07:40.739042 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:40.710750 27762 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.006s
W20260709 13:07:40.709872 27827 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.003s
W20260709 13:07:40.713579 27882 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:07:40.714160 27782 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
I20260709 13:07:40.674853 27846 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:40.737354 27481 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.082s	user 0.013s	sys 0.005s
I20260709 13:07:40.724650 27779 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:40.731961 27475 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:40.698789 27825 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:40.756611 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.007s
W20260709 13:07:40.776844 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.009s
I20260709 13:07:40.789901 22588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f588420255042dda396ba6419abf6bc"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:40.790747 22588 raft_consensus.cc:606] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:40.791172 22588 raft_consensus.cc:614] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:40.769039 27836 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.781s	user 0.014s	sys 0.002s
I20260709 13:07:40.792125 27836 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.758762 27827 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.795693 27836 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:40.767675 27782 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:40.798730 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 565962us
I20260709 13:07:40.800230 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:40.755965 27762 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.802644 22585 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:40.797470 27856 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:40.814429 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.817898 27770 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.018s	user 0.018s	sys 0.000s
I20260709 13:07:40.818037 27875 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:40.819953 27762 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:40.820489 22593 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:40.800222 27782 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:40.764181 27846 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:40.812577 27849 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.660s	user 0.004s	sys 0.000s
I20260709 13:07:40.847728 27873 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:40.850745 21853 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:40.849213 22591 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:40.852867 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.796813 27827 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:40.855244 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 125682us
W20260709 13:07:40.856279 21823 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.961s (SENT) [suppressed 28 similar messages]
I20260709 13:07:40.848465 27846 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:40.851449 27858 consensus_queue.cc:1243] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:40.772533 27481 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.885481 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:40.884189 27127 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.555s	user 0.015s	sys 0.000s
I20260709 13:07:40.873066 27866 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:07:40.896831 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 96693.2us
W20260709 13:07:40.925645 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 209342us
I20260709 13:07:40.947150 21933 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:40.957440 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:40.958715 21930 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:40.923480 27127 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.960826 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:40.960062 22556 connection.cc:256] RPC call timeout handler was delayed by -1.5196566749999647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 13:07:40.961638 21929 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:40.961925 22556 connection.cc:256] RPC call timeout handler was delayed by -1.3918566749999854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:07:40.962721 22556 connection.cc:256] RPC call timeout handler was delayed by -1.2964566749999449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
W20260709 13:07:40.947409 27860 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.004s
W20260709 13:07:40.963415 22556 connection.cc:256] RPC call timeout handler was delayed by -1.2343566749999582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
W20260709 13:07:40.965029 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0603890630000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 13:07:40.965363 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0147566749999442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 13:07:40.966156 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:40.967412 21926 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:40.970027 27806 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:40.974604 27806 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:40.966285 27860 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:40.980000 27845 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.895s	user 0.011s	sys 0.004s
I20260709 13:07:40.986089 27845 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:40.991636 27845 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:40.992902 27127 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:40.998661 27860 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:40.976838 27806 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:41.005887 27839 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.020876 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.021966 27839 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:41.022480 21928 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:41.010457 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.010742 22556 leader_election.cc:400] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17.
I20260709 13:07:41.009708 27475 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:41.014393 22591 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:41.024017 22556 leader_election.cc:403] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:07:41.015712 27782 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.028770 21932 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
W20260709 13:07:41.034338 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48067.8us
I20260709 13:07:41.031134 27866 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17.
I20260709 13:07:41.041507 21929 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:41.041781 21927 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:41.042273 21928 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.029074 27782 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:41.046118 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.046623 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.047274 27782 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:41.048620 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:41.049259 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.049768 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.051344 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:41.052268 27657 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:07:41.053051 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.053535 27657 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:41.054188 27894 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:41.054622 27894 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:41.053949 27869 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:41.058063 27894 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.059878 27894 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:41.059049 27847 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:41.060748 27847 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:41.061753 27847 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.063372 27847 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:41.064254 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:41.053994 27657 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:41.049278 21932 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.068464 21926 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:41.065900 27657 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.073369 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:41.073590 27866 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:41.074038 21852 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:41.085276 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:41.085606 21972 leader_election.cc:400] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19.
I20260709 13:07:41.086022 21859 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:41.087205 21972 leader_election.cc:400] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
W20260709 13:07:41.090108 27898 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:41.093997 21926 raft_consensus.cc:1217] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Deduplicated request from leader. Original: 6.7->[7.8-14.12]   Dedup: 11.10->[12.11-14.12]
I20260709 13:07:41.094694 21926 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 14. Current term is 15. Ops: [7.8-14.12]
I20260709 13:07:41.096374 27870 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:07:41.096800 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.097761 27870 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:41.090317 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.102905 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:41.088651 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:41.110819 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.115072 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.119061 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:41.121846 21853 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:41.120548 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.122798 21926 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:41.123271 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:41.124209 21854 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:41.124578 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.118983 21859 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:41.126078 21852 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:41.118145 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:41.128446 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.129078 21926 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:41.127899 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:41.132179 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.134647 27890 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:41.135226 27890 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:41.136140 27890 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.137117 27657 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.140146 27657 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:41.140677 27869 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:41.141243 27869 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:41.142057 27782 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:41.142526 27869 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:41.146178 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 279509us
I20260709 13:07:41.133244 27127 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:41.109191 27870 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:41.159664 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.160225 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.160463 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.162565 22002 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:07:41.163053 22001 raft_consensus.cc:2368] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:41.110843 27866 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:07:41.164170 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.164839 22008 raft_consensus.cc:2368] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:41.165292 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.166810 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.123926 27824 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.167654 21926 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.168444 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.169617 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.169665 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.170668 22007 raft_consensus.cc:2368] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:41.170833 21926 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:41.172470 22007 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:41.172636 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.172830 22002 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:41.173511 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:41.179241 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 90680.5us
I20260709 13:07:41.183743 21855 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:41.185458 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.142620 27782 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:41.192135 21854 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:41.167282 27806 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.197531 27806 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.198472 27600 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:41.200237 27806 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:41.195165 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:41.202323 27896 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:41.202826 27896 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:41.203253 27846 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.203884 27896 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.167121 27127 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.207199 27127 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:41.208142 27870 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.196298 27873 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:41.209813 27782 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.212119 22001 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:41.212492 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.213083 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.220484 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:41.213526 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.221588 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.170596 27824 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.217985 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.214138 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.217041 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.222724 21855 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:41.221836 22003 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.195484 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.226137 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:41.226738 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.226833 22590 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:41.235095 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:41.237851 22009 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:41.240294 22003 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 10.
I20260709 13:07:41.240818 21934 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:41.252522 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.227387 27824 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.264518 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1217) took 1082 ms. Trace:
I20260709 13:07:41.264523 21855 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:07:41.266348 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45934.9us
I20260709 13:07:41.238323 27127 raft_consensus.cc:2764] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:41.275008 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:41.279294 27846 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:41.281733 22009 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:41.274233 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:41.283886 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.268112 27860 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.285646 27860 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.286056 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:41.287362 27852 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:41.287869 27860 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:41.288060 27843 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:41.288623 27843 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:41.245621 27576 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:07:41.273495 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.273904 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.293905 22002 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:41.272267 27834 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:41.307945 27834 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.304464 27824 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.310043 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.310686 21928 raft_consensus.cc:2376] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15. Re-sending same reply.
I20260709 13:07:41.311681 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.269621 22591 rpcz_store.cc:276] 0709 13:07:40.163390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.165988 (+  2598us) service_pool.cc:224] Handling call
0709 13:07:41.245934 (+1079946us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54006702d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:41.246045 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521961216}
I20260709 13:07:41.297695 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:41.299561 27843 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.301669 27576 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:41.302444 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.303244 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.305358 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.315354 21856 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:41.317493 22001 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:41.312835 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.274739 27870 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:41.334824 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.335960 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.287905 27852 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:41.337540 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.337735 27852 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.340052 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.340771 22004 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:41.289930 27850 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:07:41.341684 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.268792 27818 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 13:07:41.323117 21935 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:41.342902 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67962.5us
I20260709 13:07:41.344666 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:41.325840 22005 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:41.342280 22004 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:41.342896 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.352524 22006 raft_consensus.cc:2368] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:41.323503 22590 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:41.348047 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.349088 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.349450 27786 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:41.322342 21825 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:41.336879 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.367833 21928 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20.
I20260709 13:07:41.359671 21935 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:41.370425 22591 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:07:41.370841 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.371685 22002 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:41.350499 27576 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:41.373507 22004 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:07:41.373600 27127 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.374164 27576 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.374389 27127 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.376075 27576 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:41.377983 22009 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:41.378068 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.379733 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.380363 21928 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
W20260709 13:07:41.386853 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64594.7us
I20260709 13:07:41.374073 22004 pending_rounds.cc:85] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 10
I20260709 13:07:41.389030 22004 pending_rounds.cc:107] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:07:41.356025 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:41.387609 27600 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:41.366425 27399 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.389201 22008 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:41.396656 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.391536 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.370868 22591 pending_rounds.cc:85] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 11
I20260709 13:07:41.389353 22004 raft_consensus.cc:2889] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:41.397845 21935 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:41.397957 21856 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:41.421834 21929 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:41.422363 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 66452.2us
I20260709 13:07:41.391315 22002 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:41.601204 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.598273 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.467653 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:41.569803 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.572099 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.465852 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 121237us
I20260709 13:07:41.606112 22591 pending_rounds.cc:107] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:07:41.606608 22591 raft_consensus.cc:2889] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:41.604694 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:41.611527 22005 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:41.614238 22005 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:41.616787 22005 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:41.617550 22009 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:41.622668 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.623418 21928 raft_consensus.cc:2376] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16. Re-sending same reply.
I20260709 13:07:41.427891 27806 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:41.429219 27861 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:41.450591 27860 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.626895 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:41.627738 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.633256 21928 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 12: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:41.650013 21823 leader_election.cc:336] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.723s (SENT)
I20260709 13:07:41.650851 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:41.657711 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.422855 27399 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.665969 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.666695 21928 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.669401 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.428797 27657 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.201s
W20260709 13:07:41.660939 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56526.2us
I20260709 13:07:41.627161 27782 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:41.660065 21935 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.674161 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:41.698105 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 62158.7us
W20260709 13:07:41.699718 21823 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.729s (SENT)
I20260709 13:07:41.705713 27914 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:41.706929 27914 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:41.700466 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:41.670162 27881 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:41.728374 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.717533 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:41.729635 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.729884 21929 raft_consensus.cc:2376] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16. Re-sending same reply.
I20260709 13:07:41.672416 27824 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:41.719818 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.638350 27860 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:41.716857 27127 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:41.716495 27782 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:41.733376 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.734915 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.757797 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:41.764070 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.734061 21928 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:41.681625 27686 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:07:41.808550 21926 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.811260 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.812366 22003 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:41.824118 27779 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:41.841886 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.875568 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.876543 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:07:41.878685 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.887382 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.735209 27881 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:41.902763 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.903182 21928 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.903656 21929 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
W20260709 13:07:41.886247 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 227825us
I20260709 13:07:41.904520 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:41.904637 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 146946us
I20260709 13:07:41.905125 21929 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:41.745136 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.906787 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.910547 22005 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:41.911188 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 211583us
W20260709 13:07:41.911938 21823 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.759s (SENT)
I20260709 13:07:41.912497 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:41.924448 22003 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:41.928587 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.929955 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.936429 21928 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:41.942646 21929 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:41.938728 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:41.951614 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:41.959913 22003 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:41.953302 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:41.961843 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:41.945950 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:41.963493 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.888062 27861 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:41.910923 27852 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:41.912081 27481 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:41.948340 22005 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:41.953792 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:41.945577 21935 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:41.948108 27782 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:41.983153 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.964609 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.038480 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.039177 21928 raft_consensus.cc:2376] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15. Re-sending same reply.
I20260709 13:07:42.019893 22003 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
W20260709 13:07:42.073590 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.961426 27779 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:41.949409 27915 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:41.966406 27600 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:41.982489 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 70574.2us
I20260709 13:07:42.075856 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:42.074437 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:42.074887 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 147701us
I20260709 13:07:42.044840 21929 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:41.963109 27832 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:42.004129 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:42.000797 27843 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.012s	sys 0.011s
W20260709 13:07:42.084066 21823 leader_election.cc:336] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.800s (SENT)
I20260709 13:07:42.094131 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:42.072932 27908 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.247s	user 0.001s	sys 0.001s
I20260709 13:07:41.982139 27921 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.981249 27399 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:41.983826 27875 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:42.094426 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:42.095227 27843 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.013s	sys 0.016s
I20260709 13:07:42.094146 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:42.096588 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:42.106429 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:41.978302 27917 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:42.107086 21859 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:42.083489 27860 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:42.083477 27481 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.096662 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.103140 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.104892 21935 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:42.119100 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:42.119783 21929 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:42.102092 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.086658 27915 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:42.120779 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.121256 21928 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:42.072239 27923 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:07:42.097944 27881 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:42.116132 27576 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.740s	user 0.030s	sys 0.001s
W20260709 13:07:42.120049 27786 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.735s	user 0.004s	sys 0.013s
I20260709 13:07:42.123713 21851 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
W20260709 13:07:42.125713 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49767.2us
I20260709 13:07:42.121446 21855 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:42.171741 21859 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:42.171955 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:42.149921 21929 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:42.150933 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.128471 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.173511 21854 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
W20260709 13:07:42.149803 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.169641 21853 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:07:42.043381 27861 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.123646 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.175534 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.180205 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:42.193463 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.192625 21935 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:42.222558 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.222900 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.225915 21929 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:42.102051 27925 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
W20260709 13:07:42.195438 27912 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:07:42.128961 27860 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:42.128401 27926 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:42.223870 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 126245us
I20260709 13:07:42.226864 21935 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:42.118203 27832 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.239614 22003 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:42.229980 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.168579 27786 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:42.232270 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.223930 27845 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.234470 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.245880 21928 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:42.128589 27917 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:42.237450 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.248018 21935 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
W20260709 13:07:42.227707 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47592.4us
I20260709 13:07:42.223376 27916 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:42.238850 22005 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:07:42.239456 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.105105 27921 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.122059 27875 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.243697 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.250289 22002 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:42.253424 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:42.249174 21972 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:42.250834 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:42.228315 27889 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:42.251578 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.239595 27922 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:07:42.255044 21929 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:42.256913 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:42.258028 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.258620 22000 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:42.230381 27929 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.233152 27860 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:42.259882 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:42.260831 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 176786us
I20260709 13:07:42.262190 21899 leader_election.cc:400] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
W20260709 13:07:42.272539 21823 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.923s (SENT)
I20260709 13:07:42.262388 21931 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:42.262658 27779 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:42.263219 27935 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:42.263940 27921 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:42.264068 27870 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.290675 27870 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.266777 22585 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:42.269496 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.272486 21899 leader_election.cc:403] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:07:42.290623 21931 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:42.296010 21931 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:42.297030 27918 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:07:42.300123 27475 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:42.301327 27924 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:07:42.303093 27927 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
I20260709 13:07:42.303825 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:42.306783 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1060) took 1139 ms. Trace:
I20260709 13:07:42.307042 27937 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.285962 27935 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:42.309687 27935 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:42.310809 27875 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:42.252875 27916 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:42.284536 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.278782 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:42.307312 21927 rpcz_store.cc:276] 0709 13:07:41.167275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.667297 (+500022us) service_pool.cc:224] Handling call
0709 13:07:41.667662 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:41.668026 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:41.668053 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:41.668531 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.885559 (+217028us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:42.095372 (+209813us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.097413 (+  2041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.097529 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.306477 (+208948us) raft_consensus.cc:1685] finished
0709 13:07:42.306494 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.306700 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7061,"prepare.queue_time_us":357343,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":83746,"raft.run_cpu_time_us":496,"raft.run_wall_time_us":34440,"thread_start_us":393074,"threads_started":2,"wal-append.queue_time_us":331995}
I20260709 13:07:42.322638 21928 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:42.351351 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 72700.9us
I20260709 13:07:42.289187 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:42.289829 27930 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:42.355193 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:42.369975 21928 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:42.272471 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:42.371706 22005 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:42.307735 27937 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.314107 21931 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:42.331830 21932 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:42.346131 27917 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.284752 27936 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.275502 27860 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:42.371109 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:42.429489 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58482.5us
I20260709 13:07:42.377388 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2382) took 1051 ms. Trace:
I20260709 13:07:42.254403 27926 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:42.414829 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 142318us
I20260709 13:07:42.239321 27600 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.313648 27786 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:42.430517 21931 rpcz_store.cc:276] 0709 13:07:41.325636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.297571 (+971935us) service_pool.cc:224] Handling call
0709 13:07:42.297888 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.374789 (+ 76901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.377309 (+  2520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:42.425679 27938 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:07:42.433797 27931 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:42.430704 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:42.436745 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:42.441069 21823 leader_election.cc:336] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.942s (SENT)
I20260709 13:07:42.441462 21928 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.441519 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:42.351927 27916 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:42.416000 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:42.377214 27779 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:42.446353 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 173397us
W20260709 13:07:42.443233 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 158685us
I20260709 13:07:42.448406 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:42.449303 22005 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.441707 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:42.377944 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2909) took 1031 ms. Trace:
I20260709 13:07:42.443748 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.426008 27920 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.414306 27939 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:42.450043 21932 rpcz_store.cc:276] 0709 13:07:41.346217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.331020 (+984803us) service_pool.cc:224] Handling call
0709 13:07:42.331412 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.375170 (+ 43758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.377873 (+  2703us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:42.415071 27929 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.461484 22006 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:42.437922 27786 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.467366 21928 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:42.271766 27889 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.377661 27921 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:07:42.467034 21972 leader_election.cc:400] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:07:42.440845 27931 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:42.474164 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58283.1us
I20260709 13:07:42.473372 27786 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:42.266613 27915 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.443526 27845 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:42.477211 21972 leader_election.cc:403] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:07:42.478571 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.490216 21933 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
W20260709 13:07:42.491396 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.491643 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50632.5us
W20260709 13:07:42.492236 21823 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.957s (SENT)
I20260709 13:07:42.472952 21931 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:42.460675 27779 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.474640 27782 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:42.473510 27875 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:42.495875 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:42.438972 27936 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.499955 21853 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:42.502965 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.500795 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2921) took 1113 ms. Trace:
I20260709 13:07:42.482885 27932 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:42.507066 27933 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:07:42.508014 27838 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:42.483485 27931 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:42.512465 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 250434us
I20260709 13:07:42.513358 21899 leader_election.cc:400] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:42.503795 21933 rpcz_store.cc:276] 0709 13:07:41.378167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.383046 (+  4879us) service_pool.cc:224] Handling call
0709 13:07:42.478395 (+1095349us) spinlock_profiling.cc:243] Waited 908 us on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:42.490994 (+ 12599us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088192}
I20260709 13:07:42.507369 21852 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:42.489924 27845 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.529049 21931 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:42.502458 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:42.500466 27779 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:42.534973 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:42.537221 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:42.499982 27920 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
W20260709 13:07:42.541167 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48960.1us
I20260709 13:07:42.541293 21927 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:07:42.541427 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:42.479490 27940 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:07:42.514132 27915 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:42.531541 27845 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:42.545750 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.540194 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:42.531680 27832 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:42.543037 27770 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.008s	sys 0.011s
I20260709 13:07:42.543913 27838 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:42.559190 21852 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:42.560184 27782 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:42.560653 27934 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.562078 27481 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:42.562508 27481 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:42.563501 27481 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.565183 27943 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:42.565747 27943 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.567656 27832 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:42.569098 27834 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:42.564796 27481 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:42.571707 21930 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:42.572360 27943 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:42.573201 27839 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.573855 27839 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.578296 27936 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:42.578982 27937 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:42.553340 27770 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.009s	sys 0.011s
I20260709 13:07:42.543603 27932 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:42.572775 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2884) took 1453 ms (client timeout 1818 ms). Trace:
I20260709 13:07:42.587327 27889 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:42.588191 27944 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:42.591521 27935 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.595867 27939 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
W20260709 13:07:42.596791 27806 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.967s	user 0.010s	sys 0.004s
I20260709 13:07:42.634608 27806 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:42.602624 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:42.636451 22004 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:42.554572 27920 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:42.637542 27920 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:42.638674 27920 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.640285 27779 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.606509 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.647370 21934 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:42.610594 21930 rpcz_store.cc:269] 0709 13:07:41.119235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.119349 (+   114us) service_pool.cc:224] Handling call
0709 13:07:42.571525 (+1452176us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b54001435a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d2e18 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:42.572597 (+  1072us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540003c354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:42.572713 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538129408}
I20260709 13:07:42.628471 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:42.648792 22585 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:07:42.630348 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.555294 27926 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:42.609937 27932 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.642511 27942 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:42.643314 27936 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:42.559096 27838 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:42.656661 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.660959 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:42.648265 27940 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:07:42.668653 27839 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:42.671105 27916 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.645999 27860 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:42.681303 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 135447us
W20260709 13:07:42.664970 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 123590us
I20260709 13:07:42.677604 22587 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:42.659615 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:42.684839 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:42.682085 27936 consensus_queue.cc:1243] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:42.692684 27939 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:42.693977 27839 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:42.703007 27939 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.706064 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:42.707098 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.707288 27956 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.708122 27956 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.710861 27956 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:42.712066 27954 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:42.712085 27840 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:42.713104 27840 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:42.714849 27840 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.716427 27840 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.721856 27936 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:42.724220 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2891) took 1505 ms (client timeout 1961 ms). Trace:
I20260709 13:07:42.705749 27939 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.712617 27954 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:42.728039 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2817) took 1559 ms (client timeout 1547 ms). Trace:
W20260709 13:07:42.728454 22001 rpcz_store.cc:269] 0709 13:07:41.168099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.168213 (+   114us) service_pool.cc:224] Handling call
0709 13:07:42.727804 (+1559591us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b540040d654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:42.727934 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98595072}
I20260709 13:07:42.729763 27806 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:42.730683 27806 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.731463 27786 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.733283 22007 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:42.733421 27806 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:42.724694 21926 rpcz_store.cc:269] 0709 13:07:41.219214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.221332 (+  2118us) service_pool.cc:224] Handling call
0709 13:07:42.723856 (+1502524us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:42.724079 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348031232}
I20260709 13:07:42.734082 27952 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:42.734812 27952 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:42.735317 27957 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:42.735944 27954 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:42.733453 27786 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:42.729895 21934 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:42.737565 27952 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:42.738225 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2914) took 1390 ms (client timeout 1783 ms). Trace:
W20260709 13:07:42.738763 21934 rpcz_store.cc:269] 0709 13:07:41.347584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.366732 (+ 19148us) service_pool.cc:224] Handling call
0709 13:07:42.606332 (+1239600us) spinlock_profiling.cc:243] Waited 944 us on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:42.738163 (+131831us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171776}
I20260709 13:07:42.739984 27887 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:42.740448 27887 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:42.690726 27845 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.741317 27887 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.742489 27845 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:42.742672 27887 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:42.743156 27845 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:42.743726 27845 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.743829 21756 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 15, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 15 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.745703 21756 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 10, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 10 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.735247 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2945) took 1124 ms. Trace:
I20260709 13:07:42.748122 22003 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:42.744319 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.749441 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.749833 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.753366 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.747869 22007 rpcz_store.cc:276] 0709 13:07:41.611067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.611976 (+   909us) service_pool.cc:224] Handling call
0709 13:07:42.733103 (+1121127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.733717 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.735010 (+  1293us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540040d650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:42.735161 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577424640}
W20260709 13:07:42.769711 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 225737us
I20260709 13:07:42.771019 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.747219 21756 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 15, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 15 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:42.785996 21823 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.163s (SENT)
I20260709 13:07:42.786482 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:42.786291 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 234898us
I20260709 13:07:42.798194 21857 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:07:42.800849 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 94867.7us
I20260709 13:07:42.802098 21899 leader_election.cc:400] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16.
I20260709 13:07:42.802695 21855 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:42.765479 27938 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.766122 27852 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.807093 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.809392 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:42.810230 21851 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:42.810633 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.782435 27956 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:42.812987 21934 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [14.9-15.10]
I20260709 13:07:42.787602 27952 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:42.815958 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.792186 27839 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.816922 21855 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:42.815049 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:42.818054 21926 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:42.793807 27399 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.824000 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:42.824733 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:42.825767 22005 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:42.827234 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:42.828281 22007 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:42.820549 27956 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.811877 27938 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:42.811352 27881 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:42.834851 27956 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:42.886796 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.192544 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:42.886731 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:42.845108 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.065243 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.206571 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:43.073655 21851 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:43.207676 27127 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:43.200857 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 380526us
W20260709 13:07:42.916706 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.933719 21756 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.205695 22002 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
W20260709 13:07:43.200376 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 429543us
W20260709 13:07:43.206792 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.213757 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.872149 21926 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:42.842687 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.215440 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:43.079125 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2942) took 1469 ms. Trace:
I20260709 13:07:43.217029 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:42.928750 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1346) took 1596 ms. Trace:
I20260709 13:07:43.217864 21854 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:43.218461 22591 rpcz_store.cc:276] 0709 13:07:41.332234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.346207 (+ 13973us) service_pool.cc:224] Handling call
0709 13:07:41.346483 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.619163 (+272680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.928437 (+1309274us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b54003791d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:42.928672 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8695168}
I20260709 13:07:43.210628 21972 leader_election.cc:400] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
W20260709 13:07:42.871243 27906 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.049s	user 0.003s	sys 0.000s
W20260709 13:07:42.937419 27910 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.070s	user 0.000s	sys 0.003s
W20260709 13:07:43.226743 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.902920 27770 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.216084 22004 rpcz_store.cc:276] 0709 13:07:41.610057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.610175 (+   118us) service_pool.cc:224] Handling call
0709 13:07:41.610489 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.636895 (+1026406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.075536 (+438641us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400166054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:43.079054 (+  3518us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358502784}
I20260709 13:07:43.222535 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.194033 27964 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:43.208844 27869 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.237332 27869 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.238605 21930 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:43.241418 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.213891 27956 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.232089 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:42.888927 27845 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:42.886591 27889 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.189869 27920 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.911530 27779 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:42.837425 27935 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:42.813184 27852 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:42.826182 27940 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.838200 27860 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:43.236469 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:43.246927 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.277257 21854 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:43.280131 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.281086 21854 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:43.282001 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.282877 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46546.8us
I20260709 13:07:43.221782 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:43.250859 21857 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:42.826892 27952 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.252027 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:43.249738 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.245602 27943 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:42.840049 27931 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.265044 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:42.919946 27939 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.266913 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.251791 27956 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:43.270085 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 52171.8us
I20260709 13:07:42.837591 27881 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.271425 27938 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.259989 27850 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:43.285477 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.285980 21926 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:43.246820 27770 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.324644 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:43.326416 21857 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:43.250831 27964 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.293059 27896 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:43.362289 27896 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:43.362942 27952 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:43.363082 27896 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.364858 27956 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:43.367012 27931 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:43.367527 21852 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
W20260709 13:07:43.344197 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 123268us
I20260709 13:07:43.261162 27889 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.355902 27881 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.357776 27850 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:43.359445 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.265822 27920 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.392520 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:43.394403 21926 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:43.394790 21930 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:43.393399 21927 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:43.375784 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:43.396198 21932 raft_consensus.cc:2368] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:07:43.382808 21935 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:43.292742 27943 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:43.398389 27943 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.400205 27943 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.386544 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:43.401440 21929 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:43.311769 27887 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:43.402931 27887 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.406037 27887 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.277431 27940 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.323861 27970 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.409420 27970 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.371038 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.414768 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.416160 21857 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:43.420130 27970 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.429514 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:43.430367 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:43.430542 21931 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:43.432161 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 210285us
I20260709 13:07:43.433868 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:43.441434 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:43.445771 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51437.2us
I20260709 13:07:43.383574 27956 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:43.387526 27971 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:43.455829 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 107660us
I20260709 13:07:43.457595 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:43.449404 27840 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.733s	user 0.014s	sys 0.002s
W20260709 13:07:43.477252 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 152724us
I20260709 13:07:43.475446 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.469128 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:43.446630 27806 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:43.446996 27975 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:43.479135 21899 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:43.444609 27873 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:07:43.437492 27481 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:07:43.485137 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 269691us
I20260709 13:07:43.454870 27935 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:43.451748 27869 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.498235 27873 consensus_queue.cc:1243] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:43.485149 27939 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:43.447026 27973 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:07:43.450742 27887 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:07:43.511684 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:43.500970 21857 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:07:43.512207 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 122523us
I20260709 13:07:43.456233 27956 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.503487 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:43.521651 21856 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:07:43.517990 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.513113 21859 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:43.483032 27881 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:07:43.458417 27961 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:07:43.529361 21930 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:43.532161 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.541477 27982 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:43.541653 21858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:07:43.543183 21858 raft_consensus.cc:606] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:43.543767 21858 raft_consensus.cc:614] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:43.510591 27935 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.547175 27961 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:43.547761 27961 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:43.531358 27869 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:43.549501 27869 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:43.550004 27869 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:43.550861 27889 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:43.528081 27970 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:43.546098 27852 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:43.553346 27970 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:43.554194 27970 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:43.555001 27956 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.556198 27935 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:43.546307 27939 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.557369 27939 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.558586 27940 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:43.560781 27973 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:43.561498 27973 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:43.548796 27961 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.567025 27961 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:43.567169 27838 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.503033 27975 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.573019 27840 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:43.573873 27840 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:43.575260 27840 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.572885 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.578809 21934 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:43.564531 27973 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.579358 21934 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:43.579850 27973 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.580031 21857 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:43.580621 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.527889 27978 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:43.581974 27978 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:43.458449 27971 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.587157 21858 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:43.592510 21856 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:43.490199 27806 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.527645 27645 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:43.580366 21934 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.612900 21934 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:43.617142 21858 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:07:43.618472 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.629244 21854 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:43.568910 27838 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:43.631217 21858 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:43.600824 27840 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.528927 27914 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
W20260709 13:07:43.662091 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 197355us
I20260709 13:07:43.697652 21934 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:07:43.523465 27983 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.118s
I20260709 13:07:43.701141 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:43.702232 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:43.702402 21934 pending_rounds.cc:85] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 10
I20260709 13:07:43.703035 21934 pending_rounds.cc:107] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:07:43.703557 21934 raft_consensus.cc:2889] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:43.700541 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 180352us
W20260709 13:07:43.705938 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 64985.8us
I20260709 13:07:43.707283 21933 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:43.691046 21857 raft_consensus.cc:1217] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Deduplicated request from leader. Original: 14.8->[16.9-17.10]   Dedup: 16.9->[17.10-17.10]
I20260709 13:07:43.638201 27850 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:43.717113 21857 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: [16.9-17.10]
I20260709 13:07:43.731719 21972 leader_election.cc:400] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:07:43.741277 21972 leader_election.cc:403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:07:43.742661 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:43.717407 27957 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:07:43.757879 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:43.787554 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.772365 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 303296us
I20260709 13:07:43.790326 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:43.795323 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47131.9us
I20260709 13:07:43.654372 27985 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.745289 27686 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:07:43.719707 27917 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:07:43.719280 27949 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:07:43.657047 27770 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:43.716897 27977 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:07:43.783716 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 270012us
I20260709 13:07:43.755301 27861 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:43.793145 27779 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:43.764204 27964 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:43.718931 27952 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:07:43.802543 27852 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.802556 27845 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:07:43.878322 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47631.5us
I20260709 13:07:43.881649 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:43.888324 22585 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:43.762924 27915 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 13:07:43.813872 27920 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:43.898434 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:43.892424 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:43.837900 27939 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:43.874394 27957 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:43.940150 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:43.873311 27779 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:43.876132 27770 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:43.802389 27985 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:43.944446 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46103.1us
W20260709 13:07:43.945426 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 160270us
I20260709 13:07:43.880806 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:43.903817 27845 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:43.890066 27935 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:43.791042 27850 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.895094 27852 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.041599 21933 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:44.050035 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:44.036304 21931 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.005s
I20260709 13:07:44.046073 22590 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:07:43.993204 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 269752us
W20260709 13:07:44.075023 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.074668 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:43.969195 22006 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:43.967800 22585 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:07:44.100174 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.943121 27914 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:44.098654 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:44.069396 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 98301.5us
I20260709 13:07:44.000352 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.109308 22004 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:44.099962 22587 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:43.944550 27931 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.105324 21931 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
W20260709 13:07:44.051712 27896 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.688s	user 0.018s	sys 0.000s
W20260709 13:07:43.999949 27986 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.346s	user 0.000s	sys 0.000s
W20260709 13:07:44.064736 27996 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:07:43.956678 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.106921 27998 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.101454 27870 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:43.975885 27943 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.006s
I20260709 13:07:44.114555 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:44.115955 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 235270us
W20260709 13:07:44.000907 27987 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 13:07:44.063688 27995 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:07:44.062480 27869 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 13:07:44.123957 22587 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:44.115515 27806 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.114259 27914 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:44.107183 27952 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:07:43.950660 27938 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:44.073556 27917 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:43.951644 27920 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.132519 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:43.951892 27779 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:44.133059 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 176477us
I20260709 13:07:44.133450 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:43.965130 27939 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:44.097038 27860 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.166607 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:44.127364 27845 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:43.957336 27770 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:43.876451 27978 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:44.130689 27481 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:44.093909 27935 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.184612 27938 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:44.189153 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:44.194437 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 62001.5us
I20260709 13:07:44.198925 22585 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:44.174973 27948 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.614s	user 0.007s	sys 0.000s
I20260709 13:07:44.128865 27931 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.136325 28001 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:44.211331 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.214468 21899 leader_election.cc:400] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:07:44.229987 22589 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:44.232658 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.133968 27896 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.187805 27806 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:44.206512 27970 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.003s
I20260709 13:07:44.218693 28002 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:44.229440 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2814) took 1004 ms. Trace:
I20260709 13:07:44.247346 22000 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:44.247673 21851 rpcz_store.cc:276] 0709 13:07:43.224766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.225136 (+   370us) service_pool.cc:224] Handling call
0709 13:07:43.225476 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.391206 (+165730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.391237 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.395938 (+  4701us) log.cc:844] Serialized 26 byte log entry
0709 13:07:43.518565 (+122627us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:43.519560 (+   995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.519814 (+   254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:44.229204 (+709390us) raft_consensus.cc:1685] finished
0709 13:07:44.229242 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.229376 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22732,"raft.queue_time_us":105364,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":5487,"thread_start_us":264730,"threads_started":2,"wal-append.queue_time_us":132135}
I20260709 13:07:44.176905 27869 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.169082 27998 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.245486 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2812) took 1023 ms. Trace:
I20260709 13:07:44.219497 28006 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:44.186990 27999 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:07:44.236466 27939 raft_consensus.cc:2764] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:07:44.255918 21855 rpcz_store.cc:276] 0709 13:07:43.221788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.221957 (+   169us) service_pool.cc:224] Handling call
0709 13:07:43.237438 (+ 15481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.379973 (+142535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.380012 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.385037 (+  5025us) log.cc:844] Serialized 26 byte log entry
0709 13:07:43.610733 (+225696us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:43.610796 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.610925 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:44.245234 (+634309us) raft_consensus.cc:1685] finished
0709 13:07:44.245257 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.245421 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43200,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":10328,"thread_start_us":270118,"threads_started":2,"wal-append.queue_time_us":259134}
I20260709 13:07:44.271358 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:44.215271 27860 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.250033 27806 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.247354 28002 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:44.245417 27481 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:44.191190 27917 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:44.283795 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 117301us
I20260709 13:07:44.269961 27985 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.278090 27645 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:44.287658 27852 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.289494 27896 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.290522 27997 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:44.292488 27975 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.295737 27931 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.228799 27914 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.255851 27970 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.245656 28001 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:44.304826 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.305316 22591 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:44.203470 27920 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.385026 27920 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:44.282640 27938 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.331421 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.385706 27860 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.341917 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.387077 22585 raft_consensus.cc:2410] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:07:44.286057 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:44.306931 27973 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.000s
I20260709 13:07:44.281734 27952 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:44.365628 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:44.337227 28002 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:44.388990 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 103054us
I20260709 13:07:44.300565 27869 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:44.333791 27838 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.695s	user 0.007s	sys 0.006s
I20260709 13:07:44.302054 27861 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
W20260709 13:07:44.343959 27840 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.678s	user 0.013s	sys 0.000s
I20260709 13:07:44.341804 27481 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:44.380103 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.379863 27970 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.381601 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.381515 28001 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.363483 27645 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:44.391956 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.390368 22592 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:07:44.401273 28011 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:44.404137 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:44.375265 27961 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.808s	user 0.015s	sys 0.002s
I20260709 13:07:44.427455 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:44.342470 27998 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.428674 21856 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:44.436167 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:44.389576 27850 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.455220 28011 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:44.489786 28011 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.493929 27973 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:44.494767 27973 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.454082 28010 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.451006 27914 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:44.497866 28010 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.441974 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.443694 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.452490 22592 pending_rounds.cc:85] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 13
W20260709 13:07:44.452143 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:44.452881 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61031.2us
I20260709 13:07:44.504123 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.504987 21932 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:44.505240 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.506248 27806 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.507164 27935 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:44.390368 27997 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:44.389449 27770 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.455739 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.510941 22007 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:44.456853 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.512578 21860 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: [14.10-15.11]
I20260709 13:07:44.428097 27839 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.513484 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2819) took 1235 ms. Trace:
I20260709 13:07:44.513775 27839 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.456012 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.520087 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.524510 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.518334 21756 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 11, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 11 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:44.525677 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:44.526840 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.526336 21756 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 6, term changed from 6 to 13, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to <none>, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 13 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:44.527837 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.528249 27997 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:44.501260 22592 pending_rounds.cc:107] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:07:44.530608 22592 raft_consensus.cc:2889] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:44.465798 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.481346 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.532621 22001 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:44.461372 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.462941 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:44.484193 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:44.484964 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.486512 27999 raft_consensus.cc:2764] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:07:44.531432 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:44.556435 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 190929us
I20260709 13:07:44.396488 27896 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:07:44.514180 21860 rpcz_store.cc:276] 0709 13:07:43.278098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.511994 (+1233896us) service_pool.cc:224] Handling call
0709 13:07:44.512329 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:44.513031 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.513418 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:44.528827 21756 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 9, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:44.538951 21928 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:44.534837 21926 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:44.561337 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.453516 27852 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:44.389621 27952 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.531930 28011 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:44.554760 27838 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:44.557325 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.535s (SENT)
I20260709 13:07:44.586299 22000 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:44.531775 28010 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.559938 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:44.560717 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:44.561326 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 75660.9us
I20260709 13:07:44.551412 22584 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:44.562077 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.594385 22007 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:44.563400 22009 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:44.564924 22005 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:07:44.571115 27770 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:44.459259 27939 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:44.454857 27481 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:44.553476 27973 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.576301 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:44.579136 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.528975 27997 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:44.558902 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:44.601881 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.603962 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:44.612706 22588 raft_consensus.cc:1217] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Deduplicated request from leader. Original: 9.9->[11.10-15.12]   Dedup: 11.10->[13.11-15.12]
I20260709 13:07:44.619346 22588 pending_rounds.cc:85] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 10
I20260709 13:07:44.619663 22588 pending_rounds.cc:107] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:44.620003 22588 raft_consensus.cc:2889] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:44.624274 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.624809 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.627339 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.635871 22593 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:44.627410 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:44.639681 22556 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.641s (SENT)
I20260709 13:07:44.640509 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.611907 27770 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.642308 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:44.645409 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 85597.8us
I20260709 13:07:44.648456 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:44.579087 27852 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.650027 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.651854 27852 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.651978 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:44.594187 27838 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.649726 22006 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 12: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:44.615392 28010 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:44.616387 27997 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:44.667131 22584 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:44.664518 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:44.673286 21926 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:44.673940 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:44.668031 27972 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:44.668443 28012 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.672430 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:44.592168 27779 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.677517 27838 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.677695 27645 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.681648 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.701181 22000 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:44.682761 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.661185 27770 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:44.693311 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:44.668215 28010 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:44.707631 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:44.672915 28006 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:44.634001 27973 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:07:44.685499 27970 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:44.698547 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.013s	sys 0.014s
W20260709 13:07:44.710282 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.014s	sys 0.014s
I20260709 13:07:44.707412 22006 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:07:44.688410 27939 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.747661 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:44.715358 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.751019 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:44.754801 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48928.4us
I20260709 13:07:44.692174 27870 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.712009 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.756862 22006 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
W20260709 13:07:44.710835 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 580344us
I20260709 13:07:44.695604 27983 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:44.698661 27834 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:44.660709 27481 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.776878 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:44.709959 28010 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:44.710649 27914 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.710960 27896 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:44.756342 27972 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:44.803164 27954 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:44.804033 27889 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:44.804589 27889 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:44.817981 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2409) took 1443 ms (client timeout 1744 ms). Trace:
W20260709 13:07:44.818605 21933 rpcz_store.cc:269] 0709 13:07:43.374433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.377180 (+  2747us) service_pool.cc:224] Handling call
0709 13:07:44.817739 (+1440559us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54004295d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:44.817900 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626776832}
I20260709 13:07:44.751542 27887 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:44.709728 28006 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:44.751125 27970 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.810300 27645 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.785378 22006 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:44.783492 22003 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:44.795710 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:44.704032 28012 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.840857 28012 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.833148 28006 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:44.841594 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71791.4us
I20260709 13:07:44.841152 28001 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:44.842716 22000 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:44.761324 27939 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.853188 22003 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:44.824929 27889 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:44.795809 27834 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:44.767822 27870 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.830287 27887 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:44.844774 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93824.5us
I20260709 13:07:44.843676 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:44.861129 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65360.9us
I20260709 13:07:44.872609 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:44.847048 28019 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.168s	user 0.000s	sys 0.002s
I20260709 13:07:44.829396 27399 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:44.842242 27970 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:44.888059 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:44.889088 21852 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:44.892946 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.894760 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.895339 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.895874 22593 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:07:44.897693 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54146us
I20260709 13:07:44.828696 27999 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:07:44.896375 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:44.873167 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:44.876740 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:44.873533 22000 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:44.812919 27914 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.852334 27127 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.846408 27845 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:44.912452 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:44.904017 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:44.904641 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.913188 21856 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:44.905462 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:44.910651 28020 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:07:44.911211 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:44.916693 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:44.878542 27887 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:44.885835 28012 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:44.901378 27973 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.912695 27686 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:44.890414 27952 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.859140 28001 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.848950 27980 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:07:44.911513 27127 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.927237 27861 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:44.926934 27806 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.946409 27990 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:44.970781 27990 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.941202 28031 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:44.971756 27127 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:44.911576 27834 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.973526 27916 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:44.952338 27973 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.978464 27973 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:44.978866 27971 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:44.948594 27917 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:07:44.980489 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1155) took 1190 ms. Trace:
I20260709 13:07:44.948782 28012 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:44.981199 22591 rpcz_store.cc:276] 0709 13:07:43.789862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.790046 (+   184us) service_pool.cc:224] Handling call
0709 13:07:44.980264 (+1190218us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400684f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:44.980424 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181951744}
I20260709 13:07:44.982259 28012 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.984026 22587 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:44.922783 27925 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:07:44.907281 27399 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:44.988355 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:44.989454 28018 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:07:44.989609 27806 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:44.991055 28028 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:07:44.991992 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:44.992848 28017 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:07:44.916978 27840 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:07:44.929291 27845 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.962882 27954 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:44.821530 27997 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:44.994503 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2973) took 1023 ms. Trace:
I20260709 13:07:44.999755 22592 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:45.003902 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2969) took 1781 ms. Trace:
I20260709 13:07:44.995885 21860 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:45.007999 27983 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:45.033723 27983 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:45.026016 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.035476 27399 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.037724 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.039511 21930 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:45.029971 27481 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:44.998090 28026 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:07:45.008922 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 911) took 1755 ms (client timeout 1535 ms). Trace:
W20260709 13:07:45.041160 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.003929 28031 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:45.042776 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.008672 28035 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:07:45.045035 28022 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.206s	user 0.000s	sys 0.003s
I20260709 13:07:44.968572 27779 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:44.995304 28001 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.021996 22008 rpcz_store.cc:276] 0709 13:07:43.222745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.228975 (+  6230us) service_pool.cc:224] Handling call
0709 13:07:43.237839 (+  8864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.238173 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.238203 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:44.705814 (+1467611us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:44.706251 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:07:44.851006 (+144755us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:44.885584 (+ 34578us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.887934 (+  2350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.888004 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:45.003561 (+115557us) raft_consensus.cc:1685] finished
0709 13:07:45.003672 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.003818 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9387,"prepare.queue_time_us":15837,"prepare.run_wall_time_us":66,"raft.queue_time_us":3346,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":13680,"spinlock_wait_cycles":31556864,"thread_start_us":66614,"threads_started":1,"wal-append.queue_time_us":183100}
I20260709 13:07:45.017526 22002 rpcz_store.cc:276] 0709 13:07:43.971679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.971823 (+   144us) service_pool.cc:224] Handling call
0709 13:07:43.972270 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:44.246029 (+273759us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:44.432371 (+186342us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.686438 (+254067us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.686636 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:44.686677 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:44.686718 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.994296 (+307578us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b540038b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:44.994339 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.994450 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12508,"prepare.queue_time_us":279309,"prepare.run_cpu_time_us":432,"prepare.run_wall_time_us":426,"raft.queue_time_us":2290,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6320,"spinlock_wait_cycles":657838208,"thread_start_us":374459,"threads_started":2,"wal-append.queue_time_us":336744}
I20260709 13:07:45.030804 27896 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:44.972922 28002 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.008s
I20260709 13:07:45.049787 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:45.042351 21859 rpcz_store.cc:269] 0709 13:07:43.255528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.264717 (+  9189us) service_pool.cc:224] Handling call
0709 13:07:45.008713 (+1743996us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54002ba054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:45.008845 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425827712}
W20260709 13:07:45.025476 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 96712.9us
I20260709 13:07:45.053750 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.055902 21974 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:44.992929 27806 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.056682 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:45.045058 27995 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.860s	user 0.003s	sys 0.003s
I20260709 13:07:45.028573 27939 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.060912 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.061249 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.061631 21935 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:45.062307 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:45.062649 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.063370 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:45.065393 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 143235us
I20260709 13:07:45.057762 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:45.043064 28031 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:45.056977 27982 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:45.080343 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.085561 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.088307 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.065667 27806 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:45.092356 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.053821 27896 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.070086 27953 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.102542 22592 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 13: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:45.110617 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56935.3us
I20260709 13:07:45.112288 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.072968 27961 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:45.115491 27961 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.115928 27961 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:45.117271 27982 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.120072 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.103534 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.056318 28016 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:07:45.122603 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:45.101112 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.077548 27971 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:45.055423 28002 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.077011 27917 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.086790 27481 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:45.130090 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.132066 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:45.138219 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.124326 27971 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.138046 22592 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:45.135176 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.138114 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.163419 21859 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:45.138978 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:45.165781 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.166719 21853 raft_consensus.cc:2410] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:07:45.166671 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.139622 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.168471 21851 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:45.168915 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:45.169691 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:45.170404 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.140748 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.141840 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.171438 21858 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:45.172623 21858 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:45.142401 21935 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:45.174484 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.176381 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:45.144991 28034 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:45.182575 21851 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 10: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:45.131158 28001 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.172715 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.091619 27770 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:45.186561 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51446.7us
I20260709 13:07:45.186734 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.198544 21852 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:45.150909 27973 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:45.145717 27982 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.151557 27971 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:45.149981 27845 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:45.156947 28016 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:07:45.195674 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:45.156453 27481 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:45.136907 27861 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:07:45.235436 22587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
W20260709 13:07:45.380224 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50370.7us
W20260709 13:07:45.380856 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 185261us
I20260709 13:07:45.381794 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.382114 22587 raft_consensus.cc:606] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.308199 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1166) took 1311 ms. Trace:
W20260709 13:07:45.383348 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 237429us
I20260709 13:07:45.384418 22589 rpcz_store.cc:276] 0709 13:07:43.997181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.997330 (+   149us) service_pool.cc:224] Handling call
0709 13:07:45.307979 (+1310649us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54005325d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:45.308130 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361235200}
W20260709 13:07:45.248225 28045 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:07:45.247541 28030 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:07:45.413111 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:45.413192 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 130278us
I20260709 13:07:45.413837 21854 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:45.396641 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.197113 27934 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:45.310491 28024 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.472s	user 0.002s	sys 0.000s
I20260709 13:07:45.384522 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.411790 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.204107 27997 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:45.228408 27840 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:45.254981 28002 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.293020 28047 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.425567 21855 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:45.426646 22585 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:45.301743 27839 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.310593 27896 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.417820 21853 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:45.243408 28039 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:45.418989 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:45.419816 28041 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.412420 27982 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:45.421363 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.441705 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.410792 28036 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:45.421363 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.317216 28033 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 13:07:45.237910 28015 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.360s
I20260709 13:07:45.408075 27845 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:45.407845 27860 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:45.245678 27936 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.176s
I20260709 13:07:45.420467 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:45.241087 28006 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.303887 27889 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.243039 27961 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.423759 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:45.451381 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:45.447695 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 271313us
W20260709 13:07:45.450062 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 251465us
I20260709 13:07:45.428397 27997 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.463522 28012 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:07:45.501183 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49907.3us
I20260709 13:07:45.442293 27770 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:45.463840 27976 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.464416 28037 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.513384 21929 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:45.464666 22587 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:45.434188 27934 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.452052 27982 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:45.414149 28001 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:07:45.511833 27860 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:45.522014 28042 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.469491 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.443059 27127 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:07:45.480258 21854 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:45.483052 22588 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 10: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:45.432929 28039 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:45.421965 27917 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:07:45.454336 27806 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:45.503826 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.504067 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.504776 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.505487 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.507740 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.502063 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:45.458191 27870 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.511277 22586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:45.427423 27887 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.516732 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1077) took 1013 ms. Trace:
W20260709 13:07:45.571659 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 69184us
I20260709 13:07:45.518237 22590 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:45.518898 21930 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:07:45.520416 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.471134 28036 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.439529 27840 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.524859 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.501542 27997 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.439388 28047 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.532332 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.563791 22585 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:45.564841 21935 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:07:45.565580 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63613.6us
I20260709 13:07:45.582898 22586 raft_consensus.cc:606] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.582993 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:45.583353 22586 raft_consensus.cc:614] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:45.567039 21926 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:45.527148 27934 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.571023 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:45.551401 28010 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.739s	user 0.008s	sys 0.004s
I20260709 13:07:45.550940 27806 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:45.576804 21855 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:07:45.576721 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 56422.2us
I20260709 13:07:45.588783 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.590513 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.591679 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.566227 21928 raft_consensus.cc:2376] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19. Re-sending same reply.
I20260709 13:07:45.531203 28039 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:45.610901 21932 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 10.
I20260709 13:07:45.572115 21927 rpcz_store.cc:276] 0709 13:07:44.503105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.511187 (+  8082us) service_pool.cc:224] Handling call
0709 13:07:44.511534 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:44.511857 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:44.511919 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:44.513033 (+  1114us) log.cc:844] Serialized 26 byte log entry
0709 13:07:44.708211 (+195178us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:44.708254 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.708360 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:45.309882 (+601522us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:45.508386 (+198504us) raft_consensus.cc:1685] finished
0709 13:07:45.508412 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.516666 (+  8254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2797,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":18852224,"thread_start_us":193937,"threads_started":1,"wal-append.queue_time_us":324375}
I20260709 13:07:45.586799 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.612157 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.613317 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.485797 27481 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.523154 27980 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:07:45.593770 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.595183 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.595364 27399 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:45.596099 28010 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.634853 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.580554 27999 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:45.602640 28036 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.573400 27861 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:45.604204 27934 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:45.604332 28001 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:45.613473 27806 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:45.606112 27889 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.651789 21853 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:45.606313 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.607045 27870 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:45.633209 27840 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.628475 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.523327 28037 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.517315 27976 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.659060 27976 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.644145 27861 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.661366 27861 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:45.662678 21934 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:45.664565 21934 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:45.646354 27915 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:45.665257 27915 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:45.666553 27914 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.666124 27915 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.667061 28015 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.667557 27915 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:45.667616 28015 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.670455 28010 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:45.673022 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 251708us
I20260709 13:07:45.581104 28016 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:45.601778 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:45.638065 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.703318 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:45.736730 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 309108us
I20260709 13:07:45.737809 22557 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:45.638224 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.639304 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.738565 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:45.739523 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 101383us
I20260709 13:07:45.740583 21929 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:45.642328 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.592478 27845 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:45.644250 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.522532 28042 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:45.614447 27970 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.002s
W20260709 13:07:45.614882 28011 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.004s	user 0.006s	sys 0.010s
I20260709 13:07:45.645792 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.744525 21855 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:45.602686 28035 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:07:45.634069 27917 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:45.747768 27845 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:45.748579 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 146874us
I20260709 13:07:45.612100 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.654120 21854 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:45.673789 21756 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 13, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 13 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:45.701906 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.743978 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:45.708564 27961 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.710161 28047 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:45.654052 28053 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 13:07:45.720237 28031 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.000s
I20260709 13:07:45.701335 28064 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.647195 28001 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.742074 21934 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:45.769397 21926 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:45.648332 27982 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.614117 28052 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:45.775022 28052 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:45.775539 28052 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:45.672272 28057 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.221s
I20260709 13:07:45.781404 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.756078 21927 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:45.756116 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:07:45.749684 28056 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 13:07:45.786775 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.762666 27917 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:45.763384 27956 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:45.745769 27889 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:45.750118 28012 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:45.792939 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.768611 28035 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.768955 28031 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.761369 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:45.788942 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.801296 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.789250 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.802587 21935 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 10.
I20260709 13:07:45.802646 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.803813 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.803889 28031 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.804174 28049 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:45.804677 28049 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:45.805143 28049 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:45.805190 28031 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:45.771737 28001 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:45.805984 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44755.2us
I20260709 13:07:45.807503 28031 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:45.809278 22593 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:45.810669 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:45.804028 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.782896 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:45.768136 27961 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:45.811815 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.813699 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:45.813705 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:45.815014 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.816038 22005 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:45.790432 28068 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:45.818486 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51687.9us
I20260709 13:07:45.820592 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:45.796088 21934 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:45.755573 27970 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.822842 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:45.824378 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86002us
I20260709 13:07:45.824810 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.746649 28016 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.808528 28031 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.788079 28006 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.846740 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:45.841950 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 85936.4us
I20260709 13:07:45.827919 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:45.880945 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53141.1us
I20260709 13:07:45.885083 22585 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:07:45.887451 22584 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:45.887844 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 62386.3us
I20260709 13:07:45.847781 21928 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:45.839769 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:45.889137 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49549.2us
I20260709 13:07:45.890200 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.890944 22003 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:45.834378 27971 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.891829 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:45.892066 22589 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:45.892403 22003 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.893369 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.893966 27971 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:45.894995 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.907542 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:45.895991 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:45.908440 22004 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.896788 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.844614 27481 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.909245 22005 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:45.818239 27982 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:45.901120 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:45.902034 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:45.910586 22585 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:45.878063 27917 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:45.844146 27956 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:45.810912 28012 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:45.882256 28068 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:45.935511 28077 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:45.935971 28068 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.937995 28042 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.902082 28033 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:45.939913 28016 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:45.940379 28033 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.940886 28033 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:45.941347 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61188.9us
I20260709 13:07:45.875605 27887 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:45.914510 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:45.952028 22005 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:45.911552 27980 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:45.833798 27998 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:07:45.926359 22591 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:45.784487 28064 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.946353 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:45.948200 22003 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:45.951097 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.833848 27916 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:45.843883 27961 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.909439 27976 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:45.959962 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.960249 22588 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:45.965132 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.960655 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:45.961083 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 72483.2us
I20260709 13:07:45.966405 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:45.925940 27956 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.914003 27917 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:45.963708 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:45.843585 28057 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 13:07:45.961638 22585 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:45.974774 28079 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:45.833775 28035 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:45.986403 21935 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:45.936439 28077 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:45.979700 28066 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:45.988193 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.000028 22003 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:45.997583 22005 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:45.966025 27914 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:45.978817 27956 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:46.004534 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.979111 27917 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:46.007325 22005 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:46.008955 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:46.011565 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:46.012342 21934 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:46.014829 21855 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:46.015852 22004 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:46.003755 27914 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.000967 27916 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:46.019102 27961 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:46.019398 28069 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:07:46.011117 28015 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.020567 28073 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:46.021365 28073 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:46.022753 27770 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:46.023250 27770 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:46.022948 28073 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.024219 27770 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.025374 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:45.962817 28010 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:46.026397 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1173) took 1153 ms. Trace:
I20260709 13:07:46.029690 27481 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:46.044075 22584 rpcz_store.cc:276] 0709 13:07:44.872659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.896803 (+ 24144us) service_pool.cc:224] Handling call
0709 13:07:46.026188 (+1129385us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b5400668354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:46.026340 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239562112}
I20260709 13:07:46.047708 27770 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:46.049717 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:46.050381 21851 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:46.034934 27917 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.037518 27956 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.025964 27914 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:45.962951 27806 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.051251 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:46.062830 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:46.035033 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:45.994515 27980 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:07:46.065948 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:46.066978 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:45.986301 28039 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.071226 21851 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 10.
I20260709 13:07:46.072180 22005 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:46.072801 22588 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:07:46.073899 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.075168 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:46.099645 22007 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:46.103996 22009 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:46.104561 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:46.105228 21852 raft_consensus.cc:2376] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15. Re-sending same reply.
I20260709 13:07:46.071261 27980 consensus_queue.cc:1243] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:46.106359 21934 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:46.109503 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:46.110311 21852 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:46.113891 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.055393 27997 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.107072 28015 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:46.115976 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.120950 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.052373 28012 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.126444 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.123016 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:46.119462 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.116370 27481 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:46.119864 28015 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.158681 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:46.144816 27873 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:46.141772 27976 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:46.146627 27686 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:46.145061 27915 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:46.145586 28062 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.173400 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:07:46.174242 21934 raft_consensus.cc:606] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:46.174654 21934 raft_consensus.cc:614] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:46.154551 27481 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.168787 22586 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 9
I20260709 13:07:46.167424 28015 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:46.184257 27997 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.185629 28052 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.172390 27873 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.175824 27915 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.177712 22586 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.10
I20260709 13:07:46.223970 22586 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Aborting config change with OpId 14.10: Aborted: Op aborted by new leader
I20260709 13:07:46.179548 27481 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:46.224457 22586 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:46.179567 27686 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:46.226930 28082 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:07:46.231561 27956 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.236778 27914 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.237429 28077 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:46.238365 21851 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:46.238668 28064 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:46.239763 27861 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.004s
I20260709 13:07:46.240340 27914 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.221714 27873 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:46.243580 21857 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:46.222820 27915 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:46.251883 22589 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:46.180230 28062 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.230343 28035 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.240206 27861 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.245757 27806 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:46.260738 22004 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:46.267534 22003 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:46.247013 27976 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:46.258545 27917 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:46.313983 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:46.316337 27861 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:46.301072 28033 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:46.310935 28085 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:07:46.322440 21928 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:07:46.323938 21928 pending_rounds.cc:85] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 9
I20260709 13:07:46.324179 21928 pending_rounds.cc:107] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:07:46.324420 21928 raft_consensus.cc:2889] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:46.281898 27686 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.308080 27914 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:46.341919 27914 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.344141 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:46.327678 21928 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:07:46.329614 28092 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:07:46.331585 27938 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:46.333223 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:46.337854 28089 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:46.337446 21756 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:46.341395 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:07:46.350739 27481 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:46.351864 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.002s	sys 0.010s
W20260709 13:07:46.385426 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.003s	sys 0.010s
W20260709 13:07:46.385885 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 557840us
I20260709 13:07:46.389632 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:46.352037 28090 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:07:46.352444 28087 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:07:46.352793 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2424) took 1292 ms (client timeout 1574 ms). Trace:
I20260709 13:07:46.355316 28062 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:46.357132 28033 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.359582 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.415716 22005 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
W20260709 13:07:46.362412 28001 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.000s
I20260709 13:07:46.365871 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:46.367952 28093 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:46.368580 22591 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:46.378082 27914 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:46.347903 28052 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.379257 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:46.381693 28049 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 13:07:46.381551 27873 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.395175 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:46.434571 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 68794.3us
I20260709 13:07:46.396646 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:46.406559 21930 rpcz_store.cc:269] 0709 13:07:45.060241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.062428 (+  2187us) service_pool.cc:224] Handling call
0709 13:07:46.352574 (+1290146us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b54000983d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:46.352714 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047171456}
I20260709 13:07:46.436618 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:46.415113 28085 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:07:46.405961 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:46.405808 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:46.421512 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:46.427646 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:46.398123 27840 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.451668 21929 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:46.438426 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:46.405567 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:46.414320 27806 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:46.454222 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:46.441175 27916 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:46.441497 27990 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.442325 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:46.443586 27873 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.443672 28077 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:46.449740 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:46.437690 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:46.445915 27786 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:46.345115 27976 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.464394 21930 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:46.451481 27936 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:46.496948 21898 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.509s (SENT) [suppressed 15 similar messages]
I20260709 13:07:46.413686 27915 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.454668 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:46.499761 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45168.9us
I20260709 13:07:46.500809 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:46.468540 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:46.471201 21856 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:07:46.454031 27806 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.457312 27840 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.427771 28001 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.512329 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:46.513780 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.692s (SENT)
I20260709 13:07:46.523874 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:46.459781 27939 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:46.531060 27939 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:46.532121 27939 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.525345 21853 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:46.526858 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:46.527096 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:46.447414 28049 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.529704 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.537758 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.542725 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.547771 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:46.557562 22008 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:46.556790 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:46.560396 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:46.516567 27955 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.521898 27806 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:46.516845 28008 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:46.525873 28027 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.685s	user 0.006s	sys 0.000s
I20260709 13:07:46.523594 27961 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:46.578394 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:46.579421 22001 raft_consensus.cc:2368] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13.
I20260709 13:07:46.580786 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:46.581631 22000 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:46.539086 27914 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:46.539345 27887 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:46.593438 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:46.593992 21856 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:46.715366 28011 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:46.556229 27686 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:46.546059 27943 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:46.741173 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.575958 27978 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:46.576902 27961 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:46.774148 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1082) took 2255 ms. Trace:
I20260709 13:07:46.775437 27915 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.778599 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.814895 22007 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:46.780495 28037 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:46.780700 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1240) took 1366 ms. Trace:
I20260709 13:07:46.782307 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.782982 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.822758 22003 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:46.783598 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2950) took 1261 ms. Trace:
I20260709 13:07:46.823534 21926 rpcz_store.cc:276] 0709 13:07:45.522421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.606013 (+ 83592us) service_pool.cc:224] Handling call
0709 13:07:46.783435 (+1177422us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400429fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:46.783535 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269069568}
I20260709 13:07:46.786927 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.788074 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1372) took 1766 ms. Trace:
I20260709 13:07:46.812654 21931 rpcz_store.cc:276] 0709 13:07:44.518583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.518797 (+   214us) service_pool.cc:224] Handling call
0709 13:07:44.519158 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:44.519653 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:44.519679 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:44.520449 (+   770us) log.cc:844] Serialized 26 byte log entry
0709 13:07:44.757595 (+237146us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:44.757636 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.757703 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:46.432885 (+1675182us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:46.773859 (+340974us) raft_consensus.cc:1685] finished
0709 13:07:46.773879 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:46.774076 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1598,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":7399680,"thread_start_us":227980,"threads_started":1,"wal-append.queue_time_us":319122}
I20260709 13:07:46.788532 21851 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:07:46.788964 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1236) took 1368 ms. Trace:
I20260709 13:07:46.821255 22009 rpcz_store.cc:276] 0709 13:07:45.414692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.414878 (+   186us) service_pool.cc:224] Handling call
0709 13:07:45.415229 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.104442 (+689213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.780417 (+675975us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54001978d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:46.780623 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1583512320}
I20260709 13:07:46.828321 22009 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:46.790278 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2486) took 1907 ms (client timeout 1611 ms). Trace:
W20260709 13:07:46.832353 21860 rpcz_store.cc:269] 0709 13:07:44.883277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.898530 (+ 15253us) service_pool.cc:224] Handling call
0709 13:07:46.790098 (+1891568us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:46.790205 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184863104}
I20260709 13:07:46.824669 22593 rpcz_store.cc:276] 0709 13:07:45.021505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.021926 (+   421us) service_pool.cc:224] Handling call
0709 13:07:45.022223 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.809822 (+787599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.787788 (+977966us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b54005618d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:46.787994 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1809373952}
I20260709 13:07:46.792826 28010 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:46.794808 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:46.802179 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:46.802966 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:46.802532 21756 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 18, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:46.803504 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:46.805733 22008 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:07:46.841156 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:46.743770 28095 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:46.824184 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:46.809513 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 280987us
I20260709 13:07:46.817165 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:46.818524 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:46.746568 27982 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.832s	user 0.006s	sys 0.006s
I20260709 13:07:46.853029 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:46.854157 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:46.737754 27988 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:46.851133 21899 leader_election.cc:400] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13.
I20260709 13:07:46.851981 22589 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:46.852126 22588 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:46.826550 22587 rpcz_store.cc:276] 0709 13:07:45.420090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.421098 (+  1008us) service_pool.cc:224] Handling call
0709 13:07:46.788783 (+1367685us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b54006266d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:46.788905 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28264704}
W20260709 13:07:46.890519 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46323.1us
W20260709 13:07:46.892223 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:46.893992 28015 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.674s	user 0.006s	sys 0.005s
I20260709 13:07:46.577998 27955 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.719214 28035 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.256s
I20260709 13:07:46.819696 28008 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:46.895933 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44103.4us
I20260709 13:07:46.821700 27961 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:46.851850 28011 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.938462 21932 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:46.738950 28094 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.332s
I20260709 13:07:46.923529 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2826) took 1480 ms. Trace:
W20260709 13:07:46.924206 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73722.7us
I20260709 13:07:46.929036 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2434) took 1139 ms. Trace:
I20260709 13:07:46.847187 27942 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:46.946892 22000 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
W20260709 13:07:46.950795 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 357406us
I20260709 13:07:46.950832 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2837) took 1005 ms. Trace:
I20260709 13:07:46.952286 21853 rpcz_store.cc:276] 0709 13:07:45.945088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.480944 (+535856us) service_pool.cc:224] Handling call
0709 13:07:46.481397 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.535231 (+ 53834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.950613 (+415382us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b540010c554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:46.950767 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100634496}
I20260709 13:07:46.939606 22004 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:46.954479 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2859) took 1021 ms. Trace:
I20260709 13:07:46.954928 22004 rpcz_store.cc:276] 0709 13:07:45.933037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.920598 (+987561us) service_pool.cc:224] Handling call
0709 13:07:46.939438 (+ 18840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.954123 (+ 14685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.954403 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:46.899914 27978 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:46.835826 27860 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:46.918339 28037 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:07:46.943781 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.943440 21859 rpcz_store.cc:276] 0709 13:07:45.414879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.414992 (+   113us) service_pool.cc:224] Handling call
0709 13:07:45.415286 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.604748 (+189462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.604780 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:45.604825 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:45.604860 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.895385 (+1290525us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540028b754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:46.895428 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:46.895564 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435221888}
I20260709 13:07:46.944021 21935 rpcz_store.cc:276] 0709 13:07:45.789878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.803552 (+ 13674us) service_pool.cc:224] Handling call
0709 13:07:46.928825 (+1125273us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540003cd54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:46.928956 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418177792}
I20260709 13:07:46.847707 27953 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:46.919054 27481 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.850677 28010 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:46.744511 27943 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.958503 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2421) took 1922 ms. Trace:
I20260709 13:07:46.848451 28033 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:07:46.957798 27978 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.951388 27896 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.983572 21933 rpcz_store.cc:276] 0709 13:07:45.036140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.036269 (+   129us) service_pool.cc:224] Handling call
0709 13:07:45.036846 (+   577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.037336 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.037367 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.038286 (+   919us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.355885 (+1317599us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b18004020a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:46.445496 (+ 89611us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:46.445540 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.445615 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:46.946433 (+500818us) raft_consensus.cc:1685] finished
0709 13:07:46.946453 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:46.958404 (+ 11951us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1518,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":193,"spinlock_wait_cycles":36439936,"thread_start_us":89414,"threads_started":1,"wal-append.queue_time_us":1500789}
I20260709 13:07:46.969687 22004 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:07:47.058876 22000 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:07:46.972417 27860 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:46.982962 27889 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:07:46.983192 28037 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:46.996307 27943 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.986991 28049 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:46.998179 27955 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:46.998629 27840 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:47.001308 28033 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:47.008270 27645 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:47.071990 27645 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:47.010251 22002 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:47.072420 28037 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.073284 27860 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:47.012429 27938 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.000s
I20260709 13:07:47.014431 22007 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:46.957150 27786 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:07:46.947484 28008 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:07:47.075196 28008 consensus_queue.cc:1243] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:46.967324 27873 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:07:47.075783 27976 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:47.076218 27976 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:47.076690 27873 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:47.077217 27873 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:47.077041 27976 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.078682 27976 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:47.078297 27873 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.080173 27873 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:46.961010 27686 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:07:47.081224 27686 consensus_queue.cc:1243] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:46.945711 28011 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:46.929433 28015 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.014896 28001 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:47.086889 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:47.088768 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 229996us
I20260709 13:07:47.069065 28033 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:47.055154 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:47.096704 28088 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:47.090420 28039 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:47.109978 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 174739us
I20260709 13:07:47.056658 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.021903 27978 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:47.074476 21852 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:07:47.075367 27860 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:47.092160 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.020164 28010 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.047199 22584 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:47.099496 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1246) took 1273 ms. Trace:
I20260709 13:07:47.131433 22591 rpcz_store.cc:276] 0709 13:07:45.826227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.958413 (+132186us) service_pool.cc:224] Handling call
0709 13:07:45.958821 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.423447 (+464626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.099205 (+675758us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54005dcad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:47.099428 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941648768}
I20260709 13:07:47.105634 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.028905 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1366) took 2542 ms. Trace:
W20260709 13:07:47.063566 21972 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.000s	sys 0.038s
I20260709 13:07:47.066334 28111 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:07:47.136933 28033 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.139031 28033 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:47.124352 21852 pending_rounds.cc:85] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 8
I20260709 13:07:47.141230 21852 pending_rounds.cc:107] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:07:47.141589 21852 raft_consensus.cc:2889] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:47.097847 27806 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 13:07:47.142627 27806 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:47.143757 27936 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.645s	user 0.004s	sys 0.005s
I20260709 13:07:47.144191 27936 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:47.145398 22001 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:47.123756 28049 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:07:47.146934 27955 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:47.147292 27955 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:47.147866 28010 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:47.148078 27955 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.149320 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2501) took 1496 ms. Trace:
I20260709 13:07:47.149577 27955 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:47.149711 21855 rpcz_store.cc:276] 0709 13:07:45.652517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.340194 (+687677us) service_pool.cc:224] Handling call
0709 13:07:46.340553 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.407530 (+ 66977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.407563 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.420394 (+ 12831us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.740765 (+320371us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:46.740829 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.741354 (+   525us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.149074 (+407720us) raft_consensus.cc:1685] finished
0709 13:07:47.149095 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.149256 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66543,"raft.queue_time_us":69498,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":21216,"thread_start_us":276578,"threads_started":1,"wal-append.queue_time_us":631987}
W20260709 13:07:47.153096 27481 consensus_queue.cc:1175] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "821b3831b8a348c0826bd3a20a857f00" responder_term: 12 status { last_received { term: 10 index: 11 } last_committed_idx: 10 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:47.157275 27961 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.158571 27806 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:47.159804 27961 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:47.134426 21972 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.000s	sys 0.038s
I20260709 13:07:47.171139 28015 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:47.171532 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 666875us
I20260709 13:07:47.172425 27999 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.172955 27999 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:47.179085 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73530.4us
I20260709 13:07:47.100394 28088 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:47.064190 27896 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.127353 27860 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:47.134217 22590 rpcz_store.cc:276] 0709 13:07:44.486813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.501778 (+ 14965us) service_pool.cc:224] Handling call
0709 13:07:45.308367 (+806589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.572710 (+264343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.809507 (+1236797us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54005325d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:47.028594 (+219087us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54005325d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:47.028781 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334743680}
I20260709 13:07:47.180259 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:47.180233 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 88158.7us
W20260709 13:07:47.202054 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.127547 28001 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:47.225519 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:47.226303 22000 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:47.213567 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.228173 22005 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:47.215073 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:47.216401 22008 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:07:47.229935 22008 pending_rounds.cc:85] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 9
I20260709 13:07:47.231591 22008 pending_rounds.cc:107] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:07:47.231964 22008 raft_consensus.cc:2889] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:47.209298 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.236294 22003 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:47.200706 28088 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:47.238729 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 279936us
I20260709 13:07:47.239365 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:47.248968 22005 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:47.201822 28015 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:47.250851 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.254662 22002 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:47.251583 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 276376us
I20260709 13:07:47.219822 27943 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:47.246893 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.247191 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.203909 28111 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:47.220760 27970 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:47.263837 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.257714 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.266819 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.204313 28115 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:47.266952 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:47.267729 21898 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.000s	sys 0.019s
I20260709 13:07:47.269378 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:47.268131 21898 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.000s	sys 0.019s
W20260709 13:07:47.272126 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 775135us
I20260709 13:07:47.273077 22000 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:47.274724 21859 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:47.274235 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:47.235572 27870 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:07:47.243453 27988 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:07:47.274505 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 285729us
W20260709 13:07:47.283885 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 364954us
I20260709 13:07:47.284714 22000 pending_rounds.cc:85] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 9
I20260709 13:07:47.285148 22000 pending_rounds.cc:107] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.10
I20260709 13:07:47.286027 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.287127 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:47.288673 21853 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:47.290134 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.228542 28001 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.294173 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.295451 22000 raft_consensus.cc:2917] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Aborting config change with OpId 14.10: Aborted: Op aborted by new leader
I20260709 13:07:47.290072 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:47.268497 27933 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:47.325515 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78656.5us
I20260709 13:07:47.252297 28006 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:07:47.268042 28008 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.275337 27806 raft_consensus.cc:2764] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:47.274523 28051 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:47.326807 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.256951 28094 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:07:47.306151 22000 raft_consensus.cc:2889] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:47.279381 28015 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:47.282006 27943 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:47.290508 27889 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:47.331244 28116 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:47.263312 28065 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:07:47.327422 27873 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:47.278934 27938 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:07:47.291806 28016 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.257613 28050 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:07:47.306809 27839 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.261128 27949 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:07:47.293473 27953 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.291622 28079 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:47.294621 27860 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.294525 27970 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:47.307096 28052 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:47.373101 28052 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:47.336668 28008 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.325093 27982 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:07:47.332074 27933 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:47.263127 28095 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:07:47.383287 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.338312 28117 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:07:47.343827 28123 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:47.337585 27939 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:47.344811 28113 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:47.345383 28120 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:47.352670 28005 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:47.346805 27943 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.382472 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.342473 28015 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.328773 28049 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.374029 28052 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.384550 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 2983) took 1496 ms. Trace:
I20260709 13:07:47.384708 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.357160 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.386875 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.341889 27870 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:47.337626 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.399035 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:47.399869 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 115083us
I20260709 13:07:47.405112 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.398319 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:47.428463 21860 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:47.428680 22588 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:47.410286 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:47.411027 21934 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:07:47.415702 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 146402us
I20260709 13:07:47.342391 28094 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:47.399809 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:47.645978 22589 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:47.422732 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:47.486161 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.361408 28011 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:47.639406 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.370280 28050 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:47.361179 27896 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:47.369561 28006 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:47.364481 28001 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:47.452095 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.689049 21851 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:47.407069 28051 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.380591 27933 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:47.429464 21927 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:07:47.643587 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 286481us
I20260709 13:07:47.665629 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.666898 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.696521 21857 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:47.425675 22006 rpcz_store.cc:276] 0709 13:07:45.888219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.888336 (+   117us) service_pool.cc:224] Handling call
0709 13:07:45.888647 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.995423 (+106776us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:46.002160 (+  6737us) log.cc:844] Serialized 14 byte log entry
0709 13:07:46.114935 (+112775us) log.cc:844] Serialized 14 byte log entry
0709 13:07:46.115190 (+   255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:46.116522 (+  1332us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:46.116594 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.796850 (+680256us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b180002c380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:46.829105 (+ 32255us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b180002c380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419
0709 13:07:47.076035 (+246930us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b180002c380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:47.384315 (+308280us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400167954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:47.384365 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.384485 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7171,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":370,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":3198,"spinlock_wait_cycles":273806720,"thread_start_us":108656,"threads_started":1,"wal-append.queue_time_us":140820}
I20260709 13:07:47.698064 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:47.698988 22006 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:47.680790 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 406624us
I20260709 13:07:47.360308 27978 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.419201 27949 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:47.683867 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:47.422464 28119 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.664716 28050 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:47.694327 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.644090 27870 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:47.370397 27839 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.372014 27953 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.423991 28113 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:47.683651 21860 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
W20260709 13:07:47.741294 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 356612us
I20260709 13:07:47.742057 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.341518 28122 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.337s
I20260709 13:07:47.369863 28016 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.706660 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.649834 28012 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:47.777732 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:47.778313 21931 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:47.778404 21934 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
W20260709 13:07:47.709326 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 223242us
I20260709 13:07:47.712615 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:47.779520 21928 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 11: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:47.713753 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.676260 27845 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:47.727336 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:47.786337 21927 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.730960 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:47.738122 28033 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 13:07:47.372192 28079 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.640400 28049 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.414111 27939 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.749990 21857 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:47.715153 27933 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:47.801450 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:47.721582 28011 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:47.782718 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.424451 28120 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.787315 21851 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:47.744380 21854 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:47.693444 28006 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.780025 21853 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:47.805231 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.806341 21934 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:07:47.745716 27869 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.745214 28008 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:47.706435 28051 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.820914 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.777638 28115 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:47.806481 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 51782.8us
I20260709 13:07:47.821300 21860 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:47.809844 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.820982 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.828418 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.842324 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.843055 21857 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.786084 27845 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:47.878750 21928 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:07:47.880157 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49893.5us
I20260709 13:07:47.881793 21859 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:07:47.872025 22591 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:47.873958 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:47.895721 22006 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:07:47.898026 22006 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:47.874125 21934 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:07:47.808770 28124 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.425s	user 0.000s	sys 0.002s
I20260709 13:07:47.734522 27949 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:47.902709 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 101337us
I20260709 13:07:47.903999 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:47.738489 28119 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.892848 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.883069 27845 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:47.919462 21851 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:47.921097 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:47.874488 28123 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:47.913784 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 11.
I20260709 13:07:47.873508 28144 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:47.922214 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.923596 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:47.904911 21857 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:47.936712 21851 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
W20260709 13:07:47.930016 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 154299us
I20260709 13:07:47.940161 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.932369 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2839) took 1986 ms. Trace:
I20260709 13:07:47.941035 21856 rpcz_store.cc:276] 0709 13:07:45.945573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.502907 (+557334us) service_pool.cc:224] Handling call
0709 13:07:46.539996 (+ 37089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.594473 (+ 54477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.932129 (+1337656us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b54003d3e54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:47.932310 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123733888}
I20260709 13:07:47.889218 28005 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:47.930498 21928 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:47.830458 27869 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.911254 22006 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:47.951326 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:47.825920 27933 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:47.962558 21851 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:07:47.963128 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59194.7us
I20260709 13:07:47.913523 28094 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:47.969944 21851 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:07:47.893298 28033 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:07:47.973357 21853 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:07:47.976977 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.012s	sys 0.012s
W20260709 13:07:47.977486 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.013s	sys 0.012s
I20260709 13:07:47.977734 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:47.977120 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:47.978348 21928 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:47.927222 28123 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:47.978646 21934 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:07:47.977851 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 565081us
I20260709 13:07:47.940598 28132 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.888578 27870 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.987725 21856 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:47.987906 28015 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:47.990263 28136 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 13:07:47.944916 28005 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:47.918779 27988 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:07:47.809270 28050 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.947275 27887 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:07:47.990513 27978 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.096529 21927 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:47.945104 28144 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.097607 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:48.098328 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1375) took 2595 ms. Trace:
I20260709 13:07:48.100703 21854 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:07:48.108070 21928 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:48.109285 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:48.110467 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:48.092891 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.979998 27998 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:48.104539 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:48.105208 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 234681us
W20260709 13:07:48.035181 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 131113us
I20260709 13:07:48.051765 28146 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:47.906255 28113 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:47.976635 28094 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:48.004885 28033 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:48.174055 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 222776us
I20260709 13:07:48.174394 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:48.164611 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:48.180124 22587 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:07:48.093698 28152 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:48.176034 21855 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:48.102427 28120 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.182906 22589 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:48.143615 21854 raft_consensus.cc:2393] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:48.107993 22592 rpcz_store.cc:276] 0709 13:07:45.502779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.502913 (+   134us) service_pool.cc:224] Handling call
0709 13:07:46.788910 (+1285997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.789166 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:47.127374 (+338208us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.421540 (+294166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.421613 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:47.421707 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.421799 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.098145 (+676346us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54006266d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:48.098268 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":463345,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":215721,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":204,"spinlock_wait_cycles":2956710272,"thread_start_us":631518,"threads_started":2,"wal-append.queue_time_us":575082}
I20260709 13:07:48.172273 28016 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.197840 28052 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:07:48.175112 28128 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.393s	user 0.001s	sys 0.001s
I20260709 13:07:48.173962 28094 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:48.207401 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48176.3us
I20260709 13:07:48.178213 28033 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:48.107406 28015 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:48.191571 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:48.218844 21927 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:48.191160 28050 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:48.099391 28140 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.209s
I20260709 13:07:48.185820 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:48.234361 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48673.8us
I20260709 13:07:48.238710 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:48.208343 27873 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.851s	user 0.012s	sys 0.000s
I20260709 13:07:48.241164 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:48.242915 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:48.242736 27889 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:48.244235 27889 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:48.255941 22000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.012s	sys 0.012s
W20260709 13:07:48.221551 27934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.259s	user 0.005s	sys 0.012s
W20260709 13:07:48.256310 22000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.012s	sys 0.012s
I20260709 13:07:48.208916 28148 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:48.216418 28006 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.004354 28132 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.206211 27938 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:07:48.269452 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:48.223392 27939 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.213091 28001 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:07:48.178356 28146 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.248436 28137 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 13:07:48.171985 28123 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:48.255605 27961 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:07:48.673691 27840 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.675305 27949 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.556222 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:48.533922 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:48.273862 28015 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:48.674058 27933 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.535115 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:48.682330 21855 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:48.262648 27934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.005s	sys 0.012s
W20260709 13:07:48.530637 21851 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.003s	sys 0.013s
W20260709 13:07:48.692747 21851 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.008s	sys 0.018s
I20260709 13:07:48.675738 27840 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.695976 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2827) took 3277 ms. Trace:
I20260709 13:07:48.697641 28169 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.698565 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1102) took 1396 ms. Trace:
I20260709 13:07:48.263381 28140 consensus_queue.cc:1243] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:48.255200 27873 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:48.551934 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.387811 27645 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.720284 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2884) took 1344 ms. Trace:
I20260709 13:07:48.703584 28146 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.335083 28006 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:48.704396 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 434805us
W20260709 13:07:48.432879 28088 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.158s	user 0.009s	sys 0.006s
I20260709 13:07:48.263358 28135 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:48.693513 28049 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.431957 27889 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:48.706857 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2963) took 2392 ms. Trace:
I20260709 13:07:48.300727 28051 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.700150 28001 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:48.280857 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 100498us
I20260709 13:07:48.537109 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:48.265851 27852 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.003s	sys 0.020s
I20260709 13:07:48.563380 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3005) took 1350 ms. Trace:
I20260709 13:07:48.708971 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2445) took 1422 ms. Trace:
I20260709 13:07:48.694129 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:48.592464 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 349423us
I20260709 13:07:48.713930 21931 rpcz_store.cc:276] 0709 13:07:47.301824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.301975 (+   151us) service_pool.cc:224] Handling call
0709 13:07:47.302339 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.778740 (+476401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.697890 (+919150us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400229a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:48.698321 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093892992}
I20260709 13:07:48.726682 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:48.709575 21858 rpcz_store.cc:276] 0709 13:07:45.418711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.418838 (+   127us) service_pool.cc:224] Handling call
0709 13:07:46.793283 (+1374445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.941304 (+148021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.941335 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:46.941375 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:46.941418 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.406195 (+464777us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:47.406242 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.695384 (+1289142us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000fc650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:48.695695 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3500055680}
I20260709 13:07:48.680490 28015 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:48.774428 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1251) took 1027 ms. Trace:
I20260709 13:07:48.776223 28088 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.700860 28144 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.797422 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2856) took 1065 ms. Trace:
I20260709 13:07:48.806088 21860 rpcz_store.cc:276] 0709 13:07:47.732338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.741748 (+  9410us) service_pool.cc:224] Handling call
0709 13:07:48.692965 (+951217us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b5400058cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:48.797147 (+104182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179697536}
I20260709 13:07:48.362041 28153 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:48.728598 28169 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.797240 28006 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.267405 28148 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:48.754447 22009 rpcz_store.cc:276] 0709 13:07:47.376089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.377433 (+  1344us) service_pool.cc:224] Handling call
0709 13:07:47.377866 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:47.417201 (+ 39335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.417251 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:47.423800 (+  6549us) log.cc:844] Serialized 26 byte log entry
0709 13:07:47.881512 (+457712us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:47.881551 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.719503 (+837952us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b540040d654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:48.719577 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:48.719672 (+    95us) raft_consensus.cc:1685] finished
0709 13:07:48.719690 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.720072 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2315,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":6481,"spinlock_wait_cycles":87560576,"thread_start_us":187678,"threads_started":1,"wal-append.queue_time_us":385502}
I20260709 13:07:48.789532 22587 rpcz_store.cc:276] 0709 13:07:47.746669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.746828 (+   159us) service_pool.cc:224] Handling call
0709 13:07:47.748898 (+  2070us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:48.180568 (+431670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.773708 (+593140us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54006702d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:48.774157 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990872832}
I20260709 13:07:48.797173 28001 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:48.769930 21932 rpcz_store.cc:276] 0709 13:07:46.314490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.314744 (+   254us) service_pool.cc:224] Handling call
0709 13:07:46.928976 (+614232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.939053 (+ 10077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.706165 (+1767112us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b540003cd50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:48.706545 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411544448}
I20260709 13:07:48.779488 21933 rpcz_store.cc:276] 0709 13:07:47.286462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.286627 (+   165us) service_pool.cc:224] Handling call
0709 13:07:47.325395 (+ 38768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.821490 (+496095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.821532 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.821574 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:47.821616 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.708296 (+886680us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b540003b954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:48.708399 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.708748 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140437888}
I20260709 13:07:48.808699 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:48.827596 21860 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:48.828320 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:48.365482 27869 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.834043 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:48.825452 21935 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:48.828943 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:48.815187 21927 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:07:48.831277 21931 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:48.812969 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:48.808511 28123 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:48.813313 27852 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.006s	sys 0.046s
W20260709 13:07:48.852547 22006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.011s	sys 0.063s
I20260709 13:07:48.854246 22004 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:48.855141 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:48.857879 28001 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:48.810072 28153 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.858839 21853 raft_consensus.cc:2410] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:07:48.859766 28176 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:48.814651 28006 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:48.779150 22000 rpcz_store.cc:276] 0709 13:07:47.212838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.281917 (+ 69079us) service_pool.cc:224] Handling call
0709 13:07:47.284504 (+  2587us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:47.373636 (+ 89132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.373670 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:47.374332 (+   662us) log.cc:844] Serialized 52 byte log entry
0709 13:07:48.265615 (+891283us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:48.265658 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.265731 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:48.562660 (+296929us) raft_consensus.cc:1685] finished
0709 13:07:48.562711 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.563093 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":890840,"threads_started":1,"wal-append.queue_time_us":1016968}
I20260709 13:07:48.813459 28135 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.833213 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:48.820046 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:48.816864 28169 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:48.872151 22006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.012s	sys 0.063s
I20260709 13:07:48.833357 27861 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:48.876780 27861 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:48.875121 28135 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:48.875058 28088 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.903120 27942 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:48.903707 27942 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:48.904824 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 72337.8us
I20260709 13:07:48.909536 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:48.910598 22007 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:48.916047 27840 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:48.814024 27889 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.916742 28001 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:07:48.927934 27840 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:48.930555 28181 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:48.931687 28181 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.933347 28051 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.934172 28051 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.934844 27840 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.935428 28153 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:48.936543 28051 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.878682 27861 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:48.946220 28113 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:48.947595 28179 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:48.982921 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 166355us
I20260709 13:07:48.954432 22587 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:48.955943 27980 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:48.956439 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 136662us
I20260709 13:07:48.957540 27936 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:48.959147 28015 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:48.960494 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:48.964217 28148 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.963502 27983 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:48.968467 27999 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.972222 28181 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:48.997942 21858 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:07:48.995056 27980 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:48.981693 28183 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:48.954553 27889 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.026075 28065 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:49.026693 28065 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:49.026501 27942 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.028137 27942 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:49.024266 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.028270 28065 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.033569 28179 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.036301 27999 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.036754 28183 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:49.037122 27999 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.038169 28183 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.002364 27954 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:07:49.039566 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 130335us
I20260709 13:07:49.055693 21853 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:49.069712 21853 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:49.016578 28113 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.077143 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3011) took 1172 ms. Trace:
I20260709 13:07:49.079078 22009 rpcz_store.cc:276] 0709 13:07:47.900972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.835076 (+934104us) service_pool.cc:224] Handling call
0709 13:07:48.923202 (+ 88126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.069949 (+146747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.069976 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.073261 (+  3285us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:49.073317 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.073377 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.073484 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.051906 28051 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:49.083482 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.088801 28192 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.089604 28192 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.090564 28054 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:49.091452 28054 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.070367 28179 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:49.111478 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50155.8us
I20260709 13:07:49.127063 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:49.129240 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.113493 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.093092 28062 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:49.085307 28181 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:49.097177 28192 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.171835 22004 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:49.097084 27976 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.109268 28035 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.157675 28065 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:49.093127 28186 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.158155 28062 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:49.172508 27999 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.182998 21858 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:49.188578 28079 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:49.188868 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 164795us
I20260709 13:07:49.154992 28051 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:49.189834 27943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.014s	sys 0.030s
I20260709 13:07:49.093446 27954 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.189481 27936 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:49.101467 27980 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.198673 28054 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.228895 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3012) took 1229 ms. Trace:
I20260709 13:07:49.199141 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.199461 28191 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:49.200146 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 146933us
I20260709 13:07:49.200711 28189 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:49.201221 27976 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.202137 27840 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.209450 22003 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:49.213675 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.215444 28119 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.220276 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:49.190542 28062 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.226259 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.229382 22000 rpcz_store.cc:276] 0709 13:07:47.992680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.920549 (+927869us) service_pool.cc:224] Handling call
0709 13:07:48.951560 (+ 31011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.219894 (+268334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.219928 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.219976 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:49.220025 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.221675 (+  1650us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b540038aed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:49.221724 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.221851 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616225536}
W20260709 13:07:49.222666 27943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.014s	sys 0.031s
I20260709 13:07:49.234524 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.234926 28035 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.235657 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.236696 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:49.237565 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.238001 28149 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:49.238427 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3009) took 1533 ms. Trace:
I20260709 13:07:49.238634 28113 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.233656 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 955) took 1045 ms. Trace:
I20260709 13:07:49.259034 28139 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.222187 27999 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.265667 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.261888 22586 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:49.264333 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2523) took 1375 ms. Trace:
I20260709 13:07:49.272149 28051 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:49.272907 28191 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:49.273065 28037 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:49.243695 28079 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.277207 28191 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:49.266120 27999 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.288115 21858 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:07:49.298996 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 100040us
W20260709 13:07:49.285421 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48939.5us
I20260709 13:07:49.256749 22006 rpcz_store.cc:276] 0709 13:07:47.705524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.954583 (+249059us) service_pool.cc:224] Handling call
0709 13:07:47.955209 (+   626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:47.971710 (+ 16501us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:48.268036 (+296326us) log.cc:844] Serialized 14 byte log entry
0709 13:07:49.077040 (+809004us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:49.192593 (+115553us) log.cc:844] Serialized 26 byte log entry
0709 13:07:49.196027 (+  3434us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:49.196066 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.230925 (+ 34859us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.238147 (+  7222us) raft_consensus.cc:1685] finished
0709 13:07:49.238169 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.238355 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73153,"prepare.queue_time_us":498754,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":94719,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":23597,"thread_start_us":1005532,"threads_started":3,"wal-append.queue_time_us":868581}
W20260709 13:07:49.307431 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2864) took 2857 ms (client timeout 2165 ms). Trace:
I20260709 13:07:49.267464 27943 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:49.308305 27954 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.311182 27999 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.299512 28139 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.315302 22589 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:49.293678 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.317237 28033 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.318346 22586 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:49.319365 27980 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:49.319947 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.287565 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52062.7us
I20260709 13:07:49.317641 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.322794 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.325045 22009 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
W20260709 13:07:49.296893 27998 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.126s	user 0.013s	sys 0.013s
I20260709 13:07:49.318533 28033 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.270808 21851 rpcz_store.cc:276] 0709 13:07:47.885232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.985605 (+100373us) service_pool.cc:224] Handling call
0709 13:07:47.985981 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.986262 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:48.693571 (+707309us) log.cc:844] Serialized 14 byte log entry
0709 13:07:49.182485 (+488914us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.190591 (+  8106us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:49.190717 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.194574 (+  3857us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.260607 (+ 66033us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284436,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":23947,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"spinlock_wait_cycles":9984384,"thread_start_us":1178021,"threads_started":3,"wal-append.queue_time_us":487940}
W20260709 13:07:49.307978 22001 rpcz_store.cc:269] 0709 13:07:46.449582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.740784 (+291202us) service_pool.cc:224] Handling call
0709 13:07:49.269690 (+2528906us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54002db4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:49.307167 (+ 37477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926654336}
I20260709 13:07:49.269629 28113 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:49.301134 28005 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.130s	user 0.003s	sys 0.023s
I20260709 13:07:49.263577 21856 rpcz_store.cc:276] 0709 13:07:48.184298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.905598 (+721300us) service_pool.cc:224] Handling call
0709 13:07:48.993294 (+ 87696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.006690 (+ 13396us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:49.156678 (+149988us) log.cc:844] Serialized 14 byte log entry
0709 13:07:49.224868 (+ 68190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.225568 (+   700us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:49.225632 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.225707 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.229852 (+  4145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36771,"raft.queue_time_us":49181,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8442,"thread_start_us":123506,"threads_started":2}
I20260709 13:07:49.308965 27943 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.341972 27943 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.343715 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.344544 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.286998 28035 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.287922 28054 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:49.285835 28015 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:49.348196 28157 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:49.357997 22584 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:49.358837 28139 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.359979 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.372758 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:49.360266 28094 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.134s	user 0.008s	sys 0.015s
W20260709 13:07:49.362156 28193 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:07:49.362325 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.381487 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.384931 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:49.386119 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.386360 27896 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.386934 22005 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:49.387121 27896 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.387964 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.365504 28113 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.390256 21933 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:07:49.367077 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.367908 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2843) took 1939 ms. Trace:
I20260709 13:07:49.391422 28113 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.391552 21859 rpcz_store.cc:276] 0709 13:07:47.428489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.429651 (+  1162us) service_pool.cc:224] Handling call
0709 13:07:47.430018 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:47.891027 (+461009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.367298 (+1476271us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400017f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:49.367710 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038267648}
W20260709 13:07:49.394717 28194 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
W20260709 13:07:49.396049 22591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1194) took 1998 ms (client timeout 2192 ms). Trace:
I20260709 13:07:49.397962 28132 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.399596 22592 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:49.400597 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1255) took 1203 ms. Trace:
I20260709 13:07:49.367797 28119 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:49.368746 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:49.369056 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1103) took 2065 ms. Trace:
I20260709 13:07:49.368897 21756 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 9, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:49.370616 28050 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.136s	user 0.007s	sys 0.016s
I20260709 13:07:47.987212 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:49.289728 28135 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:49.378357 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.378793 27845 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.390s	user 0.012s	sys 0.013s
I20260709 13:07:49.417744 27845 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.418642 27953 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.420655 22007 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:49.388041 27976 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.391011 28149 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:49.423900 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55289.5us
W20260709 13:07:49.404767 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.400969 22592 rpcz_store.cc:276] 0709 13:07:48.197115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.197324 (+   209us) service_pool.cc:224] Handling call
0709 13:07:48.197724 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.399167 (+1201443us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b54005d32a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:07:49.400166 (+   999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.400522 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12555008}
I20260709 13:07:49.404217 22001 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:49.402043 27839 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.415748 27870 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:49.290531 28079 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.352458 28005 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.402621 27896 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.255374 28186 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:49.396736 22591 rpcz_store.cc:269] 0709 13:07:47.397218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.398810 (+  1592us) service_pool.cc:224] Handling call
0709 13:07:49.395115 (+1996305us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400531954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:49.395525 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848274304}
I20260709 13:07:49.365324 27998 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:49.426236 22006 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:07:49.427004 28181 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:49.441157 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.438189 28181 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.445530 28181 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:49.445130 28171 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.980s	user 0.000s	sys 0.001s
I20260709 13:07:49.449100 28001 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:49.449750 28001 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:49.450654 28079 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.427862 28149 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.451136 28001 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.435959 27976 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.458050 27976 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.462040 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:49.463611 28001 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.465086 28132 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:49.465538 28132 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.466217 28132 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:49.472169 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:49.473142 28006 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.005s
I20260709 13:07:49.473683 28006 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.407447 21926 rpcz_store.cc:276] 0709 13:07:47.303170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.303309 (+   139us) service_pool.cc:224] Handling call
0709 13:07:47.303568 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.664620 (+361052us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:47.931993 (+267373us) log.cc:844] Serialized 14 byte log entry
0709 13:07:48.209625 (+277632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:48.209692 (+    67us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:48.209757 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.368037 (+1158280us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b1800034d80. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419
0709 13:07:49.368617 (+   580us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b5400207954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:49.368664 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.368869 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11860,"prepare.queue_time_us":425175,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2843,"raft.queue_time_us":4602,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":11338,"spinlock_wait_cycles":837523456,"thread_start_us":503485,"threads_started":2,"wal-append.queue_time_us":433858,"wal-append.run_cpu_time_us":1487,"wal-append.run_wall_time_us":1026193}
I20260709 13:07:49.477016 28006 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.477849 27845 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.480661 27873 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.485917 28005 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:49.424358 28015 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:49.472309 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 95405.7us
I20260709 13:07:49.483119 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1253) took 1731 ms. Trace:
I20260709 13:07:49.488210 28052 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:49.487815 22585 rpcz_store.cc:276] 0709 13:07:47.751188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.751292 (+   104us) service_pool.cc:224] Handling call
0709 13:07:47.751603 (+   311us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:47.800151 (+ 48548us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:47.875025 (+ 74874us) log.cc:844] Serialized 14 byte log entry
0709 13:07:48.728060 (+853035us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b18001a99e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:48.928760 (+200700us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:48.930450 (+  1690us) log.cc:844] Serialized 78 byte log entry
0709 13:07:48.930744 (+   294us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:48.930784 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.930909 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.472876 (+541967us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b18001a99e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:49.482770 (+  9894us) raft_consensus.cc:1685] finished
0709 13:07:49.482793 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.482954 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4576,"prepare.queue_time_us":66418,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":3959,"raft.queue_time_us":64542,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":38849,"spinlock_wait_cycles":41582208,"thread_start_us":44138,"threads_started":1,"wal-append.queue_time_us":955226}
I20260709 13:07:49.491071 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.489477 28015 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.492727 28140 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:49.495347 28140 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:49.497325 27955 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.500619 28010 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.502440 28010 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.506783 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2495) took 3869 ms. Trace:
I20260709 13:07:49.500754 28111 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:07:49.507412 21852 rpcz_store.cc:276] 0709 13:07:45.636755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.190809 (+554054us) service_pool.cc:224] Handling call
0709 13:07:46.223904 (+ 33095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.142157 (+918253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.506292 (+2364135us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:49.506609 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1955291392}
I20260709 13:07:49.484536 21933 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:49.486096 27873 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.518162 27686 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.518445 28122 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.517956 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:49.480239 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2443) took 3074 ms. Trace:
I20260709 13:07:49.518988 27686 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.519023 27914 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:49.422258 28094 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:49.565567 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 221937us
W20260709 13:07:49.567427 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76503.6us
I20260709 13:07:49.568599 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.519117 28122 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.520320 21929 rpcz_store.cc:276] 0709 13:07:46.405234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.574999 (+169765us) service_pool.cc:224] Handling call
0709 13:07:46.728810 (+153811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.729169 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:46.826158 (+ 96989us) log.cc:844] Serialized 14 byte log entry
0709 13:07:49.241257 (+2415099us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b180036f700. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:49.353669 (+112412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.404907 (+ 51238us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:49.436539 (+ 31632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.479934 (+ 43395us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.480074 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19712,"prepare.queue_time_us":92125,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":3739,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":7259,"spinlock_wait_cycles":12479744,"thread_start_us":19934,"threads_started":1,"wal-append.queue_time_us":2555379}
I20260709 13:07:49.578959 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:49.582930 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.562204 28111 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.590931 22591 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:49.594477 21756 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:49.596583 21756 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 12, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 12 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:49.604398 22005 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:49.610708 21926 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:49.533054 27914 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.622572 21972 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:49.625617 22584 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:49.629262 27686 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.639425 28094 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.629912 21853 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.597247 28065 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:07:49.651901 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.653621 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.649472 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:49.653534 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 281732us
I20260709 13:07:49.618439 28010 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.661111 22009 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
W20260709 13:07:49.666462 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47703.6us
I20260709 13:07:49.603438 28111 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.666579 21853 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:49.679306 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.682797 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:49.685964 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48914us
I20260709 13:07:49.692078 22585 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:49.698324 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.700464 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3015) took 1439 ms. Trace:
W20260709 13:07:49.701867 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 102079us
I20260709 13:07:49.700891 22007 rpcz_store.cc:276] 0709 13:07:48.254957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.993215 (+738258us) service_pool.cc:224] Handling call
0709 13:07:49.036510 (+ 43295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.421215 (+384705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.693765 (+272550us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b540045b5d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:49.693910 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882901376}
I20260709 13:07:49.699355 28010 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:49.704943 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:49.688546 28050 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:49.708479 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.711488 21857 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:49.702572 27955 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:49.706107 28001 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:49.722221 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:49.725853 28199 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.121s	user 0.000s	sys 0.002s
I20260709 13:07:49.726467 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.727129 22005 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:49.728407 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.731467 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.732334 22592 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:49.733848 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.735019 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.710393 28186 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.736867 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.707199 28123 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:07:49.744913 28172 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:49.731535 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:49.736471 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.749028 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.709385 28050 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:49.724043 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.724666 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.713212 28010 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.760031 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.766573 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.763865 22008 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:49.771100 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:49.723446 27998 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:49.745801 28172 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:49.785367 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 216306us
I20260709 13:07:49.766039 27914 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.755090 28050 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:49.789592 28116 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:49.757301 28001 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:49.790853 22592 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:49.794474 28005 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:49.796342 27934 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:49.761651 28149 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.797219 28198 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:49.776680 27998 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:49.790176 27936 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.765494 27955 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:07:49.803431 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.804183 28010 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:49.806581 22008 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:49.807317 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.807206 27915 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:49.807945 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:49.809594 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 85709.1us
I20260709 13:07:49.812230 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.790354 28181 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.820199 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:49.824060 28111 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.791702 28122 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.824730 27976 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.827481 21900 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:49.836025 21860 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:49.837373 28116 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:49.831022 28198 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:49.828580 28005 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.844211 21853 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:07:49.831358 28062 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
I20260709 13:07:49.862401 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.862298 28062 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.863216 22591 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:49.863739 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.864394 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.852319 28149 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:49.857872 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 952) took 1744 ms. Trace:
I20260709 13:07:49.855986 28198 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:49.861116 28181 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.861441 28005 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:49.861326 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.861682 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:49.880255 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.864655 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 79304.4us
I20260709 13:07:49.866096 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:49.865039 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:49.872462 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.865725 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.784677 28094 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:07:49.876109 22006 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:49.834177 27998 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:49.849090 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:49.879191 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59128us
I20260709 13:07:49.877779 21853 rpcz_store.cc:276] 0709 13:07:48.113818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.869977 (+756159us) service_pool.cc:224] Handling call
0709 13:07:49.001712 (+131735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.593049 (+591337us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b4c0004ff40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6ea9ba20 00007f3b6eaaca29 00007f3b6ea72644 00007f3b6ea99f2d 00007f3b6ea99b93 00007f3b707ca18b 00007f3b707cacb5 00007f3b707c5f30 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416
0709 13:07:49.857490 (+264441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.857813 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81514752}
W20260709 13:07:49.863212 28033 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.008s
I20260709 13:07:49.898535 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.864781 28062 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:49.900429 22585 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.853260 28116 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:49.907713 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:49.908478 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60339.9us
I20260709 13:07:49.908701 22587 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:07:49.912657 21858 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:49.911382 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3016) took 1013 ms. Trace:
I20260709 13:07:49.902261 21856 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:49.910674 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.915670 22588 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:49.914814 21927 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:49.863802 27873 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:49.913769 22004 rpcz_store.cc:276] 0709 13:07:48.897940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.008140 (+110200us) service_pool.cc:224] Handling call
0709 13:07:49.025979 (+ 17839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.175135 (+149156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.910991 (+735856us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400167954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:49.911316 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334911872}
I20260709 13:07:49.838511 27936 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:49.866791 28052 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:49.905180 28181 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:49.923084 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.924167 28155 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:49.924929 21926 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
W20260709 13:07:49.946574 28175 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.131s	user 0.000s	sys 0.005s
I20260709 13:07:49.922292 28033 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.896862 28001 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.931267 21933 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:49.931211 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.944386 22557 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:49.952148 21927 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:07:49.952579 21859 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:49.952662 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:49.933907 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.935562 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.938086 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:49.899993 28015 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.915491 28149 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:49.947463 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:49.905761 28191 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:49.933117 28074 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:49.948530 22585 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:49.953385 21853 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:49.927707 27980 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.003s
I20260709 13:07:49.954823 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:49.968220 28112 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 13:07:49.962775 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56507.3us
I20260709 13:07:49.965445 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.965879 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.969199 21852 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:49.969445 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:49.970031 28149 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.970680 21852 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:07:49.968048 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:49.972050 28005 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.971936 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:49.960911 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:49.973104 22588 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:49.975720 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:49.976568 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:49.909821 28132 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.947604 28181 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.983052 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2886) took 2605 ms. Trace:
I20260709 13:07:49.983417 28191 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:49.985477 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:49.986128 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.991010 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.987116 28119 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:49.994664 21856 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:07:50.035706 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:50.042062 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:49.970754 28149 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:49.987493 28174 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.523s	user 0.001s	sys 0.004s
W20260709 13:07:50.058810 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 146551us
I20260709 13:07:50.059629 21927 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:50.064288 22585 raft_consensus.cc:2376] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16. Re-sending same reply.
W20260709 13:07:50.062216 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66539.5us
I20260709 13:07:49.973953 28074 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:49.994791 28119 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.070902 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:49.994426 28191 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:50.072468 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:49.990127 22002 rpcz_store.cc:276] 0709 13:07:47.377282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.878052 (+500770us) service_pool.cc:224] Handling call
0709 13:07:47.878555 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:48.266506 (+387951us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:48.266537 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:48.267020 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:07:48.585361 (+318341us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:48.585447 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.585593 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.945881 (+1360288us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:49.976258 (+ 30377us) raft_consensus.cc:1685] finished
0709 13:07:49.976279 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.982850 (+  6571us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95098,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4923,"spinlock_wait_cycles":36061952,"thread_start_us":630972,"threads_started":2,"wal-append.queue_time_us":523658}
I20260709 13:07:49.946856 28155 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:49.963900 28033 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.062137 28015 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:50.078516 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 93121.6us
I20260709 13:07:50.085219 28116 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.066231 28150 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:50.088236 28150 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:50.089679 28150 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:50.086072 28033 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.065944 28065 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:49.930051 28172 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.098043 21852 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:50.114627 21853 raft_consensus.cc:2410] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:50.072038 28119 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.079406 28191 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:50.125800 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 48012.8us
I20260709 13:07:50.129516 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.132086 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.133467 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:50.134223 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 162430us
I20260709 13:07:50.136968 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.097126 28155 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.110983 28065 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.144398 28074 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.148856 28074 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:50.149344 28212 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:50.153992 21927 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:50.155326 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:50.161659 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 209268us
I20260709 13:07:50.164319 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.074986 27955 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:07:50.120644 28033 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:50.095656 28050 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.171020 21858 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:50.179356 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1101) took 2878 ms. Trace:
I20260709 13:07:50.185104 21935 rpcz_store.cc:276] 0709 13:07:47.300550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.300939 (+   389us) service_pool.cc:224] Handling call
0709 13:07:47.301310 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:48.848600 (+1547290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.146720 (+1298120us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:50.179258 (+ 32538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3504074112}
I20260709 13:07:50.194314 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.162421 28065 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:50.200829 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 128463us
I20260709 13:07:50.203773 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:50.171222 28062 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:50.210580 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 103210us
I20260709 13:07:50.207690 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2975) took 2201 ms. Trace:
I20260709 13:07:50.173658 28033 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.222420 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.235850 21933 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:07:50.235695 28062 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.237872 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.238951 28062 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:50.239362 21900 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:50.211462 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 83250.2us
I20260709 13:07:50.223587 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:50.243597 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 71231.6us
I20260709 13:07:50.223964 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.248975 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.249576 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.226235 27955 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:50.233672 21928 rpcz_store.cc:276] 0709 13:07:47.932482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.179129 (+246647us) service_pool.cc:224] Handling call
0709 13:07:48.179473 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:48.196419 (+ 16946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:48.196459 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:48.205211 (+  8752us) log.cc:844] Serialized 26 byte log entry
0709 13:07:48.672423 (+467212us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:48.829770 (+157347us) log.cc:844] Serialized 14 byte log entry
0709 13:07:48.830152 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.830287 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.986606 (+1156319us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:50.097710 (+111104us) raft_consensus.cc:1685] finished
0709 13:07:50.121079 (+ 23369us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.133757 (+ 12678us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16420,"prepare.queue_time_us":32297,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":18970,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":407,"spinlock_wait_cycles":5862656,"thread_start_us":291488,"threads_started":1,"wal-append.queue_time_us":243650}
I20260709 13:07:50.233889 28033 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:50.255421 21927 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:50.266083 21931 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:50.266574 27934 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:50.269294 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.271276 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.330309 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:50.254940 27955 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:50.352233 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:50.317463 27852 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:07:50.265470 28132 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.318138 28181 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:50.389469 21928 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:50.394285 28213 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:07:50.333458 27998 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.396595 21859 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:50.332641 27934 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:07:50.401440 28132 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:50.401744 27934 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:50.402117 28132 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.405180 28132 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.405658 28050 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.369318 28217 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:50.407397 28217 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:50.407825 28217 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.359207 27955 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:50.359921 28219 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:50.317727 28198 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.389544 27852 consensus_queue.cc:1243] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:50.436976 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:50.453169 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.452869 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 194028us
I20260709 13:07:50.441614 28219 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:50.466742 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2882) took 3224 ms. Trace:
I20260709 13:07:50.467202 22003 rpcz_store.cc:276] 0709 13:07:47.242419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.297821 (+ 55402us) service_pool.cc:224] Handling call
0709 13:07:47.298091 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.239023 (+1940932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.465267 (+1226244us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54002152d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:50.466672 (+  1405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4357401856}
I20260709 13:07:50.463778 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:50.469017 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 223673us
I20260709 13:07:50.457288 27998 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.461654 27861 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:50.485827 21825 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:50.493144 21825 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:50.508249 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:50.521274 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 252355us
I20260709 13:07:50.506981 28133 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:50.526880 28115 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.514954 27852 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:50.543314 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:50.543910 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:50.517620 27998 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:50.543187 28225 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.552397 27998 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:50.552897 28225 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.554080 27998 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.555485 28150 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.557392 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:50.559754 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45214.7us
I20260709 13:07:50.561707 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:50.565150 21854 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:50.565912 21927 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:50.565408 28217 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.566653 28224 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.567224 28224 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.568490 28217 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.569425 28219 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.570214 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:50.572108 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:50.573292 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 135269us
I20260709 13:07:50.576165 28198 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.586042 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.587302 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.580955 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.594648 21928 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
W20260709 13:07:50.589402 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67203.9us
I20260709 13:07:50.596266 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:50.602017 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:50.602022 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.607260 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.609292 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.629401 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.635195 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.649946 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:50.599917 28191 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:50.660568 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 290551us
I20260709 13:07:50.628440 28225 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.665719 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:50.667964 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.685351 22002 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:50.677302 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 216225us
I20260709 13:07:50.635083 28198 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:07:50.688964 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.690308 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.672456 28135 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.649266 28133 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:07:50.707582 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:50.708312 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:50.717760 21854 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:50.631094 28115 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.680117 28225 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:50.675700 28224 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.729372 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.675092 28227 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.728093 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:50.673084 28132 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:07:50.739889 28074 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.591s	user 0.004s	sys 0.006s
I20260709 13:07:50.740188 28115 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.741041 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:50.748013 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1260) took 1551 ms. Trace:
I20260709 13:07:50.705821 28155 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:50.730063 22004 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
W20260709 13:07:50.733260 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67649.5us
I20260709 13:07:50.676023 27852 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:07:50.735982 27645 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:50.739615 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.731922 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 170384us
I20260709 13:07:50.767594 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:50.726284 28149 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.000s
I20260709 13:07:50.748461 22587 rpcz_store.cc:276] 0709 13:07:49.196175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.196317 (+   142us) service_pool.cc:224] Handling call
0709 13:07:50.747793 (+1551476us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b540060d6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:50.747950 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489455488}
I20260709 13:07:50.778090 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.778337 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:50.776135 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 174168us
I20260709 13:07:50.778729 28155 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.777768 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:50.726570 28135 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.782522 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.730201 28150 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.769658 27861 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:50.788192 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 79217.7us
I20260709 13:07:50.796329 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:50.797863 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:50.799227 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.748430 28224 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:50.802174 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.802659 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:50.804293 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:50.780012 28054 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.809612 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:50.790792 28074 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:50.787551 28155 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:50.834960 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.835786 22008 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:50.842299 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:50.840778 22004 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:50.840214 22005 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:50.844897 27860 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.830265 28225 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:50.852532 28015 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.736s	user 0.004s	sys 0.004s
I20260709 13:07:50.854116 28228 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:07:50.856812 28230 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:07:50.858803 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:50.862900 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.863265 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.783818 28149 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.827725 28227 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:50.871556 28015 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.913308 21854 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:50.892310 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:50.921917 21928 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:50.893574 27936 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.894378 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:50.842597 27999 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:50.931690 21855 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:50.828460 28054 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.920498 28227 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:50.934737 28227 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.935098 28054 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:50.935770 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.936154 28214 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.936458 21926 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:50.937614 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:50.919796 28229 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:07:50.956224 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.957161 21928 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:50.899559 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:50.902454 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:50.877296 28191 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.905893 27955 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.906315 28222 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:50.969292 21929 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:50.969472 27978 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.970271 28149 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:50.970070 27978 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:50.972255 28119 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.799s	user 0.006s	sys 0.006s
I20260709 13:07:50.972744 28119 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.975997 28119 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:50.921871 28226 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:07:50.977519 27961 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.977985 28226 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:50.978484 28226 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:50.911721 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:50.809689 27861 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.863992 22008 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:50.822114 28198 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 13:07:50.860965 28033 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:07:50.837030 28132 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:50.927949 22000 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:50.929414 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:50.836761 28135 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:50.898581 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:50.888200 28065 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.666s	user 0.004s	sys 0.008s
I20260709 13:07:50.914368 22002 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:07:50.937150 28214 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.966464 28248 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.968648 27955 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.040632 21926 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:07:50.978045 27961 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:50.834460 28074 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:50.986464 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 74849.5us
I20260709 13:07:50.987051 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:50.928151 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:51.047199 21854 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:51.026681 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 222483us
I20260709 13:07:51.027797 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:51.013984 28246 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.025672 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:51.032034 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49315.3us
I20260709 13:07:51.049495 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:50.986675 28149 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:51.052132 21853 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:51.026150 28191 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:51.048198 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.040455 22007 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:50.986649 28181 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:50.955428 28244 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:51.043817 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:50.979348 28226 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.053903 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.064821 22004 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:51.065927 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.067013 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:51.067432 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:51.054142 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2862) took 2864 ms (client timeout 2108 ms). Trace:
I20260709 13:07:51.050068 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:51.056315 22588 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:51.028606 28119 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:51.060660 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3019) took 1531 ms. Trace:
I20260709 13:07:51.077435 22009 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:51.098229 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.049700 28074 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.030630 27860 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:07:51.062590 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:51.062736 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:51.068104 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:51.146167 28228 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:51.147449 21856 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:51.158764 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 110650us
I20260709 13:07:51.148769 27870 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:51.152310 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 126730us
I20260709 13:07:51.150509 22006 rpcz_store.cc:276] 0709 13:07:49.529092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.533180 (+  4088us) service_pool.cc:224] Handling call
0709 13:07:49.572649 (+ 39469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.905960 (+333311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.060466 (+1154506us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400166f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:51.060614 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683302144}
I20260709 13:07:51.042222 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.155274 22592 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:51.050513 28217 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:51.151630 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.156314 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.185614 21860 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:51.187470 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:51.161149 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.165874 28119 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.169366 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2449) took 3758 ms. Trace:
I20260709 13:07:51.172026 28179 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:07:51.153242 21855 rpcz_store.cc:269] 0709 13:07:48.189289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.744253 (+554964us) service_pool.cc:224] Handling call
0709 13:07:51.053957 (+2309704us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b540018e554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:51.054053 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4748478208}
I20260709 13:07:51.037994 28065 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.054409 28248 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.033866 28033 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.148710 28214 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.166970 28251 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:51.198472 21854 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:51.204206 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:51.068727 27955 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:51.214668 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 85370.2us
I20260709 13:07:51.179057 27870 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:51.230551 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50717.5us
I20260709 13:07:51.059222 28149 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.068557 28244 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.229646 21855 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:51.061318 28245 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.169s
I20260709 13:07:51.219628 28227 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.233206 28135 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.237462 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:51.182981 28132 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:07:51.209882 28179 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:51.238660 28248 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:51.238539 28252 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:51.196036 21930 rpcz_store.cc:276] 0709 13:07:47.410488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.411470 (+   982us) service_pool.cc:224] Handling call
0709 13:07:47.411893 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.443101 (+2031208us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b180036ece0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707c7bfb 00007f3b707d264b
0709 13:07:49.443494 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.443518 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:49.444163 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 13:07:49.572064 (+127901us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:49.572115 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.168870 (+1596755us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b540042a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:51.168930 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:51.168969 (+    39us) raft_consensus.cc:1685] finished
0709 13:07:51.168982 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.169187 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14856,"raft.queue_time_us":247405,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8979,"spinlock_wait_cycles":932259840,"thread_start_us":404310,"threads_started":2,"wal-append.queue_time_us":242716,"wal-append.run_cpu_time_us":2279,"wal-append.run_wall_time_us":1037675}
I20260709 13:07:51.197144 28054 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:51.149835 28191 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.241230 21935 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:51.245136 21926 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:51.069049 28181 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.214679 28226 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:51.213696 28225 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:51.214514 28198 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:51.058859 28246 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.260866 21854 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:07:51.264530 22588 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:51.269304 28228 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:51.240736 28065 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.278086 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:51.288784 21926 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.290829 28228 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.293514 22008 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:51.239938 27870 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.299006 27870 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:51.309463 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:51.310412 21930 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:51.312108 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:51.318122 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:51.321394 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:51.322559 22588 raft_consensus.cc:2368] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21.
I20260709 13:07:51.322265 28179 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.323690 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:51.263391 28251 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:51.262790 28054 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:51.285439 22006 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:51.326349 28054 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.328064 28054 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.287648 22001 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:51.319505 27860 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:51.342243 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.323812 28179 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:51.279569 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 229634us
I20260709 13:07:51.250103 27955 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:51.337267 27983 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.337435 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.338594 21756 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:51.348901 22592 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:51.337152 21974 leader_election.cc:400] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21.
I20260709 13:07:51.352154 22588 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:51.357115 21974 leader_election.cc:403] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 13:07:51.352983 22009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 3026) took 1710 ms (client timeout 1687 ms). Trace:
I20260709 13:07:51.358386 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:51.358042 22009 rpcz_store.cc:269] 0709 13:07:49.639206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.683967 (+ 44761us) service_pool.cc:224] Handling call
0709 13:07:51.349772 (+1665805us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54004380d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:51.349905 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3158707072}
I20260709 13:07:51.351779 21926 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:51.340750 28119 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.364709 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.366807 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2847) took 3913 ms. Trace:
I20260709 13:07:51.367213 21857 rpcz_store.cc:276] 0709 13:07:47.453075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.948674 (+495599us) service_pool.cc:224] Handling call
0709 13:07:47.949107 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.745769 (+1796662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.362901 (+1617132us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54002ba554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:51.366744 (+  3843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3761130368}
I20260709 13:07:51.373646 22590 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:51.374811 28227 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.355655 27860 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.376711 28191 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:51.377724 28065 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:51.378917 28243 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:51.379330 28243 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:51.368482 28198 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:51.363442 28181 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.373677 27955 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.385301 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:51.385670 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 181719us
I20260709 13:07:51.378535 27860 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:51.391724 21860 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:07:51.392089 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:51.361555 27861 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:51.393878 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.393719 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59623.9us
I20260709 13:07:51.392503 28198 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:07:51.400492 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.403558 21930 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:51.403594 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:51.386749 28119 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:51.399284 27999 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.389401 28132 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:51.435725 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:51.438473 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:51.439216 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.433979 22003 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:51.446028 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:51.446548 21926 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:07:51.447283 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:51.447738 21926 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:51.449092 28149 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:51.449795 28008 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:51.449872 27645 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:51.450510 27645 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:51.450628 28008 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:51.451116 28008 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.451622 28008 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:51.453265 27999 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.450928 27645 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.396306 28248 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21.
W20260709 13:07:51.461444 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75301.5us
I20260709 13:07:51.462056 21930 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:07:51.462296 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:51.433955 28192 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.474025 21926 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [17.9-19.10]
I20260709 13:07:51.457216 27998 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:51.469858 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:51.480894 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 130245us
I20260709 13:07:51.467875 28065 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.485755 28065 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.487360 28244 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.501358 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:51.467662 27943 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:51.503685 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65299.2us
I20260709 13:07:51.483932 27998 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.510825 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:51.513041 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:51.516839 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47074us
I20260709 13:07:51.500303 28149 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:51.517722 27998 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:51.518234 28149 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.521382 27961 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:51.521724 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 2994) took 1389 ms. Trace:
I20260709 13:07:51.522298 21927 rpcz_store.cc:276] 0709 13:07:50.132210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.264650 (+132440us) service_pool.cc:224] Handling call
0709 13:07:51.521480 (+1256830us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400228654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:51.521637 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680200960}
I20260709 13:07:51.522987 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.523801 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:51.491784 28065 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.525473 27970 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:51.524220 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 963) took 1549 ms (client timeout 1717 ms). Trace:
I20260709 13:07:51.526103 27970 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:51.526664 21852 rpcz_store.cc:269] 0709 13:07:49.974993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.975362 (+   369us) service_pool.cc:224] Handling call
0709 13:07:51.524050 (+1548688us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b540010ca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:51.524153 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126872320}
I20260709 13:07:51.380668 28243 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.525702 28252 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:51.528098 22593 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:51.528306 28252 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:51.528748 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:51.529034 28243 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.514222 27943 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.530370 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1384) took 4580 ms. Trace:
I20260709 13:07:51.530768 22593 rpcz_store.cc:276] 0709 13:07:46.949498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.949821 (+   323us) service_pool.cc:224] Handling call
0709 13:07:46.950130 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.528441 (+4578311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.530066 (+  1625us) spinlock_profiling.cc:243] Waited 4.58 s on lock 0x7b54005000d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:51.530297 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10526502656}
I20260709 13:07:51.531617 27943 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.533425 28149 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:51.534610 28155 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.701s	user 0.013s	sys 0.002s
I20260709 13:07:51.488582 28005 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:51.536454 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:51.536435 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 100790us
I20260709 13:07:51.467382 28132 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.492062 28228 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:51.539515 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.542970 21927 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:51.542760 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.550995 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:51.545408 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50924.5us
W20260709 13:07:51.545811 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.557869 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:51.488969 28192 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.563292 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.563614 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:51.564649 22003 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:51.528760 28252 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:51.566586 21931 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:51.547606 28198 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:51.568622 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.555022 28135 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:51.603576 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:51.538434 28065 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:51.614748 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48045.5us
I20260709 13:07:51.545399 28155 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.627375 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:51.596323 27961 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:51.562409 28263 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:51.631546 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.552990 28005 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:51.561488 28228 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:51.636616 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:51.567279 28149 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:51.637480 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.652001 21855 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:51.653548 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.654398 21855 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:51.567586 27861 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:51.655821 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.647370 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:51.638633 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:51.657111 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.658525 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.717s (SENT) [suppressed 18 similar messages]
I20260709 13:07:51.672165 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.609632 28264 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:51.656872 21855 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:07:51.706909 28265 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:07:51.709275 28135 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:51.710151 28273 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:07:51.711196 28273 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:51.711585 28273 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:51.715474 28008 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:51.628186 27845 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:07:51.716977 21854 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:51.631577 27961 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:51.649307 28270 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:51.650712 28212 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.615187 28258 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.211s
I20260709 13:07:51.672576 28263 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:51.656749 28149 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:51.697821 28246 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:51.720048 27961 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.800931 28149 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:51.722704 28198 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:51.745743 27955 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:51.843787 28192 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:51.844146 28267 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:07:51.712927 27645 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:51.851279 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 214716us
W20260709 13:07:51.846472 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 219170us
W20260709 13:07:51.931519 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.939003 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:51.940908 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.712764 28228 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:51.940140 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 76728.7us
I20260709 13:07:51.954526 28270 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:51.955933 28008 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:51.747663 28212 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:51.950995 28192 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 13:07:51.984562 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.985761 28005 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:51.966135 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:51.969779 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 406499us
I20260709 13:07:51.971343 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:51.972224 28277 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.987139 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3042) took 1113 ms. Trace:
I20260709 13:07:51.991545 21933 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:07:51.992034 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3007) took 1302 ms (client timeout 1724 ms). Trace:
W20260709 13:07:51.992084 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.998600 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:51.991001 22009 rpcz_store.cc:276] 0709 13:07:50.873336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.364880 (+491544us) service_pool.cc:224] Handling call
0709 13:07:51.375805 (+ 10925us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:51.444616 (+ 68811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:51.444648 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:51.467360 (+ 22712us) log.cc:844] Serialized 26 byte log entry
0709 13:07:51.501913 (+ 34553us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:51.501986 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.502217 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:51.986923 (+484706us) raft_consensus.cc:1685] finished
0709 13:07:51.986943 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.987073 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59688,"raft.queue_time_us":73226,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":21243,"threads_started":1,"wal-append.queue_time_us":68142}
W20260709 13:07:51.992501 21933 rpcz_store.cc:269] 0709 13:07:50.689659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.965514 (+1275855us) service_pool.cc:224] Handling call
0709 13:07:51.991968 (+ 26454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.974151 28172 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.002153 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:51.980767 28033 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.025676 22009 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:07:52.056710 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.077680 21933 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:51.996848 28132 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.003695 27955 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:51.987519 27861 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:52.090838 28226 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.809s	user 0.013s	sys 0.000s
I20260709 13:07:52.094182 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1121) took 1188 ms. Trace:
I20260709 13:07:52.032042 28008 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:52.100034 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 110757us
I20260709 13:07:52.099320 21930 rpcz_store.cc:276] 0709 13:07:50.906050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.473180 (+567130us) service_pool.cc:224] Handling call
0709 13:07:51.496193 (+ 23013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:51.561263 (+ 65070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:51.561318 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:51.601267 (+ 39949us) log.cc:844] Serialized 26 byte log entry
0709 13:07:51.861441 (+260174us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:51.861549 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.861667 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:52.093923 (+232256us) raft_consensus.cc:1685] finished
0709 13:07:52.093949 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.094106 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24270,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8833,"thread_start_us":231705,"threads_started":2,"wal-append.queue_time_us":462854}
I20260709 13:07:52.100669 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:52.101150 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 962) took 2150 ms (client timeout 1576 ms). Trace:
I20260709 13:07:52.104063 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.117597 22001 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:52.117551 21859 rpcz_store.cc:269] 0709 13:07:49.950963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.965135 (+ 14172us) service_pool.cc:224] Handling call
0709 13:07:52.100904 (+2135769us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.101067 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982629632}
W20260709 13:07:52.118799 21927 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 13:07:52.119534 21927 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
W20260709 13:07:52.119673 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 153662us
I20260709 13:07:52.122902 21928 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:52.123805 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:52.126093 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.127272 27915 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:52.127602 27915 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:52.128587 21934 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:52.129331 28212 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.128234 27915 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.134776 27915 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:52.135656 28217 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.940s	user 0.009s	sys 0.001s
I20260709 13:07:52.136296 28217 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:52.120635 28155 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.138455 27999 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.138720 27978 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.140287 28052 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:52.140664 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.141417 22592 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:52.142849 22592 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:52.147233 27961 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.148449 28243 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.002s
I20260709 13:07:52.004671 28277 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.151418 28074 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.974s	user 0.012s	sys 0.000s
I20260709 13:07:52.151890 28074 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.154296 22009 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:52.094714 28198 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:52.156402 28179 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.797s	user 0.013s	sys 0.000s
I20260709 13:07:52.159808 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:52.098145 27998 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.001s
I20260709 13:07:52.163705 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:52.165781 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.167609 21930 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:52.113713 28054 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.785s	user 0.015s	sys 0.000s
I20260709 13:07:52.165896 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:52.170212 21933 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:52.162055 28270 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:52.178591 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:52.185902 28226 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:52.186550 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.188113 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.193845 27961 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.205500 27861 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:52.210048 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 83212.9us
I20260709 13:07:52.170182 28258 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:52.211618 28258 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.212102 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:52.213511 28278 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:52.214061 28278 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.214558 28278 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.179800 22590 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:52.218982 21933 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:52.223160 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:52.226996 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61188.2us
I20260709 13:07:52.241978 21930 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:07:52.241847 21927 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:52.242945 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:52.184944 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:07:52.244375 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59563us
I20260709 13:07:52.245035 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.248178 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:52.246827 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.250649 21851 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:52.179323 27645 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.214424 28258 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.254685 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:52.215422 28052 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:52.258088 28065 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:52.259462 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.260141 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:52.245882 28212 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:52.249811 28033 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:52.264268 28243 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:52.264530 28033 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.264804 28243 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.265184 28033 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:52.265352 28243 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.271340 27970 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:52.272364 28252 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.670s	user 0.007s	sys 0.000s
I20260709 13:07:52.173559 27998 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.278656 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:52.262734 28270 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:52.280589 21859 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:52.286206 21900 leader_election.cc:400] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:52.286655 21900 leader_election.cc:403] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:07:52.259096 27887 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.288312 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:52.292284 21933 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:52.244040 28005 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:07:52.296343 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.300136 21857 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:52.301667 21859 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:52.301700 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.302882 21856 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:52.279621 28052 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:52.272789 28252 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.291550 27873 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:52.289566 28006 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:52.310113 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.293596 28212 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:52.293418 27970 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:52.306107 28001 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:52.327807 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.335819 28251 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:52.336310 28251 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:52.338588 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.339727 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.322654 27970 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.343356 27970 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.344142 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:52.345019 28228 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:52.345438 28228 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:52.345834 28228 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:52.346196 22002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 1266) took 1864 ms (client timeout 2014 ms). Trace:
I20260709 13:07:52.316124 28050 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 13:07:52.346688 22002 rpcz_store.cc:269] 0709 13:07:50.481668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.487669 (+  6001us) service_pool.cc:224] Handling call
0709 13:07:52.346010 (+1858341us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400425ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.346138 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405008256}
I20260709 13:07:52.347846 28050 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:52.348199 28050 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:52.349117 28050 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.350634 28050 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:52.350472 28273 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.351872 28273 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.352658 28277 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.354360 28252 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:52.355559 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2882) took 1532 ms. Trace:
I20260709 13:07:52.355875 27998 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.355944 21858 rpcz_store.cc:276] 0709 13:07:50.823616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.860106 (+ 36490us) service_pool.cc:224] Handling call
0709 13:07:51.524422 (+664316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.524643 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:51.524665 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:51.524833 (+   168us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:51.524889 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.524955 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.355380 (+830425us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b540010ca50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.355497 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527005952}
I20260709 13:07:52.357846 28212 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.311156 28270 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 13:07:52.358940 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3001) took 2178 ms (client timeout 1575 ms). Trace:
W20260709 13:07:52.381901 21929 rpcz_store.cc:269] 0709 13:07:50.154877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.206761 (+ 51884us) service_pool.cc:224] Handling call
0709 13:07:52.333049 (+2126288us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b54002e86d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.333208 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1658735872}
I20260709 13:07:52.331555 28074 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.362784 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2462) took 1486 ms. Trace:
W20260709 13:07:52.362958 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 102024us
I20260709 13:07:52.384100 22001 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:07:52.365902 22557 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:52.384438 21931 rpcz_store.cc:276] 0709 13:07:50.876679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.901090 (+ 24411us) service_pool.cc:224] Handling call
0709 13:07:51.527005 (+625915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.596393 (+ 69388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.362034 (+765641us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b5400228654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:52.362583 (+   549us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b5400228650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.362724 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519094400}
I20260709 13:07:52.346400 22589 raft_consensus.cc:2410] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:07:52.344774 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.374909 22590 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:52.379309 21756 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:07:52.386510 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:52.396394 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:52.367702 28252 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:52.399569 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:52.400938 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.402546 21929 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:52.388531 27983 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.391497 28172 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:52.412374 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.413223 21931 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.414000 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.398917 27970 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:52.419660 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.426440 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:52.406602 27998 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:52.430879 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:52.434067 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 122026us
I20260709 13:07:52.435266 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:52.441684 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70079.2us
I20260709 13:07:52.442303 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:52.445708 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.446568 22590 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:52.445724 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.448396 21929 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:52.408111 28252 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.425845 28172 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:52.435694 28001 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.418478 27983 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.434031 27998 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.458230 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:52.431730 27970 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:52.451704 28149 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.002s
I20260709 13:07:52.424089 28033 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.453218 27845 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:52.452677 28005 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:52.467530 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:52.466467 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 2987) took 2336 ms. Trace:
I20260709 13:07:52.468271 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1268) took 2617 ms. Trace:
I20260709 13:07:52.454188 28243 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.468861 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:52.418744 28052 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.472699 28282 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:52.473135 28282 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.473416 28243 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.473527 28282 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.445864 28006 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:07:52.483700 27970 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.474617 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.470851 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.468156 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.463491 27998 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.456341 28065 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:07:52.500423 21931 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:52.465646 28277 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:52.463951 27955 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.470979 22591 rpcz_store.cc:276] 0709 13:07:49.847107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.901458 (+ 54351us) service_pool.cc:224] Handling call
0709 13:07:49.928090 (+ 26632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.058078 (+129988us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:50.245727 (+187649us) log.cc:844] Serialized 14 byte log entry
0709 13:07:50.684708 (+438981us) log.cc:844] Serialized 14 byte log entry
0709 13:07:50.905964 (+221256us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b1800462fa0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:52.330271 (+1424307us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b1800462fa0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:52.464496 (+134225us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b1800462fa0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287
0709 13:07:52.464597 (+   101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:52.464645 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:52.464696 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.464765 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.464891 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":210912,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":4492,"raft.queue_time_us":27437,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":7299,"spinlock_wait_cycles":39497856,"thread_start_us":539025,"threads_started":3,"wal-append.queue_time_us":496528,"wal-append.run_cpu_time_us":2874,"wal-append.run_wall_time_us":1194690}
I20260709 13:07:52.481616 28283 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.483500 28278 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.501219 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.468631 21932 rpcz_store.cc:276] 0709 13:07:50.129978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.130831 (+   853us) service_pool.cc:224] Handling call
0709 13:07:50.131184 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:50.209532 (+ 78348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:50.209610 (+    78us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:50.220514 (+ 10904us) log.cc:844] Serialized 26 byte log entry
0709 13:07:52.277850 (+2057336us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b180041b0c0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1
0709 13:07:52.278105 (+   255us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:52.426685 (+148580us) log.cc:844] Serialized 14 byte log entry
0709 13:07:52.463756 (+ 37071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.463895 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:52.463936 (+    41us) raft_consensus.cc:1685] finished
0709 13:07:52.466287 (+  2351us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.466410 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53524,"prepare.queue_time_us":45034,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":23,"raft.queue_time_us":120043,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":16640,"spinlock_wait_cycles":76222208,"thread_start_us":303087,"threads_started":4,"wal-append.queue_time_us":266601,"wal-append.run_cpu_time_us":2342,"wal-append.run_wall_time_us":1037951}
I20260709 13:07:52.478926 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:52.474350 28258 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:52.515196 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.459056 27861 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:07:52.519554 27861 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:52.519966 27861 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:52.520896 27861 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.522065 27861 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:52.522621 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:52.508778 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.523154 27887 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:52.493423 28292 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:07:52.479995 28231 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.506681 21927 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:52.523996 27978 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.464282 28257 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:07:52.466797 28149 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.524737 27978 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.524804 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.527719 21927 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.529834 27978 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.533808 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:52.534991 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 56080.7us
I20260709 13:07:52.552439 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.553745 21931 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:52.468120 28287 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
W20260709 13:07:52.558590 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.560312 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.547755 28005 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:52.561133 22587 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:52.507915 28277 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.568317 28277 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.566461 28231 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.568832 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.570091 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.570453 28231 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:52.560542 28293 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:07:52.570679 22590 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:52.571632 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.572141 22590 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.573740 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.574321 21931 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.576246 28277 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.515434 28258 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:52.594502 28258 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.515933 28283 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.598621 28278 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.599196 28283 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:52.600561 28033 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.556102 28252 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:52.600982 28228 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.603217 28228 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.602887 28282 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.604250 21932 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:52.605067 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:52.605639 22003 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:52.608520 28282 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.580343 28277 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.576920 21929 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
W20260709 13:07:52.577529 28291 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:07:52.582240 27645 raft_consensus.cc:1064] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:07:52.623690 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.624557 21929 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:52.579942 21927 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:52.625845 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.626680 21927 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:52.588738 21931 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:52.512262 27955 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.591830 28251 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.505784 28172 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.633504 22007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2905) took 1900 ms (client timeout 1938 ms). Trace:
I20260709 13:07:52.634639 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.623354 21756 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 17, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 17 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.635991 21929 raft_consensus.cc:2410] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:52.636469 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:52.637528 21931 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:52.634375 22007 rpcz_store.cc:269] 0709 13:07:50.733474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.733613 (+   139us) service_pool.cc:224] Handling call
0709 13:07:52.633225 (+1899612us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540040d154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.633424 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597035264}
I20260709 13:07:52.639966 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2479) took 1021 ms. Trace:
I20260709 13:07:52.640439 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2482) took 1020 ms. Trace:
I20260709 13:07:52.640471 21931 rpcz_store.cc:276] 0709 13:07:51.619090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.636066 (+1016976us) service_pool.cc:224] Handling call
0709 13:07:52.639885 (+  3819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.641443 21929 rpcz_store.cc:276] 0709 13:07:51.620229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.634215 (+1013986us) service_pool.cc:224] Handling call
0709 13:07:52.640336 (+  6121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.643867 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.644315 28263 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:52.645124 28263 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:52.646817 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.647918 22592 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:52.656491 27955 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:52.656652 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2465) took 1719 ms (client timeout 2201 ms). Trace:
W20260709 13:07:52.665055 21928 rpcz_store.cc:269] 0709 13:07:50.937364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.986767 (+ 49403us) service_pool.cc:224] Handling call
0709 13:07:52.656421 (+1669654us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54004290d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.656572 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482029824}
I20260709 13:07:52.648777 27998 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.671144 21930 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 13:07:52.679353 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2467) took 1715 ms. Trace:
I20260709 13:07:52.679953 21930 rpcz_store.cc:276] 0709 13:07:50.964044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.670112 (+1706068us) service_pool.cc:224] Handling call
0709 13:07:52.670585 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:52.671865 (+  1280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.679194 (+  7329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.682498 21928 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:07:52.683075 21928 pending_rounds.cc:85] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:07:52.684160 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.676757 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:52.687448 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:52.662941 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.689121 22592 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:52.662211 22003 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:52.644757 21927 raft_consensus.cc:2410] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:07:52.662338 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.698396 21931 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:52.700918 21931 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:52.701870 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:52.703193 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2466) took 1748 ms. Trace:
I20260709 13:07:52.704469 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.707160 21930 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:52.715447 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.716187 21855 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:52.683385 21928 pending_rounds.cc:107] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:07:52.717547 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.712922 21929 rpcz_store.cc:276] 0709 13:07:50.955138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.665055 (+1709917us) service_pool.cc:224] Handling call
0709 13:07:52.665484 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.685332 (+ 19848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.685369 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:52.702790 (+ 17421us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:52.702902 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.702984 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.703123 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.709560 21858 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:52.708712 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.708034 27887 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:52.725967 27887 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:52.726481 28251 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.727058 27887 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.664307 27961 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:07:52.711300 21851 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:52.718156 21928 raft_consensus.cc:2889] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:52.709101 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2477) took 1736 ms. Trace:
I20260709 13:07:52.744572 21931 rpcz_store.cc:276] 0709 13:07:50.967492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.699351 (+1731859us) service_pool.cc:224] Handling call
0709 13:07:52.699773 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.703131 (+  3358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.703394 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:52.745302 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.714211 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2468) took 1749 ms. Trace:
I20260709 13:07:52.711300 21825 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:52.721709 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.721992 28149 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:52.723977 21932 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:52.700978 28138 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:52.729460 28263 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.754668 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:52.747370 21927 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:52.756069 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1124) took 1696 ms. Trace:
W20260709 13:07:52.751854 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3004) took 2595 ms (client timeout 1739 ms). Trace:
W20260709 13:07:52.756783 21935 rpcz_store.cc:269] 0709 13:07:50.156213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.209829 (+ 53616us) service_pool.cc:224] Handling call
0709 13:07:52.751641 (+2541812us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54002a8354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.751783 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285425024}
I20260709 13:07:52.704463 28033 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:52.747609 21930 rpcz_store.cc:276] 0709 13:07:50.964568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.681385 (+1716817us) service_pool.cc:224] Handling call
0709 13:07:52.706802 (+ 25417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:52.713912 (+  7110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.714152 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.757843 21927 rpcz_store.cc:276] 0709 13:07:51.059377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.719400 (+1660023us) service_pool.cc:224] Handling call
0709 13:07:52.747135 (+ 27735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.755818 (+  8683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.756015 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.755976 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1125) took 1258 ms. Trace:
I20260709 13:07:52.761646 21932 rpcz_store.cc:276] 0709 13:07:51.497910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.723466 (+1225556us) service_pool.cc:224] Handling call
0709 13:07:52.723794 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.755719 (+ 31925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.755922 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.752388 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2473) took 1786 ms. Trace:
I20260709 13:07:52.706362 28212 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:52.755975 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.759444 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.763406 21928 rpcz_store.cc:276] 0709 13:07:50.966269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.681491 (+1715222us) service_pool.cc:224] Handling call
0709 13:07:52.681986 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.751975 (+ 69989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.752247 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.756043 21931 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:52.766472 21859 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:52.777329 21930 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:52.784411 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1128) took 1243 ms. Trace:
I20260709 13:07:52.777221 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.785398 21931 rpcz_store.cc:276] 0709 13:07:51.541393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.745922 (+1204529us) service_pool.cc:224] Handling call
0709 13:07:52.746347 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.784099 (+ 37752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.784350 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.785655 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2484) took 1157 ms. Trace:
I20260709 13:07:52.777446 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.786684 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.786973 21855 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.759238 28251 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:52.785656 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2485) took 1157 ms. Trace:
I20260709 13:07:52.787683 21858 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:52.789733 21928 rpcz_store.cc:276] 0709 13:07:51.628691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.766909 (+1138218us) service_pool.cc:224] Handling call
0709 13:07:52.785145 (+ 18236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.785359 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.785382 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:52.785421 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:52.785456 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.785502 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.785603 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.791574 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.786226 21930 rpcz_store.cc:276] 0709 13:07:51.628178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.765716 (+1137538us) service_pool.cc:224] Handling call
0709 13:07:52.777144 (+ 11428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.785393 (+  8249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.785604 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:52.794514 21928 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 13:07:52.790287 21935 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:07:52.790928 21931 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [14.9-16.10]
I20260709 13:07:52.756043 28138 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.777652 28065 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.758508 28228 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:52.780172 28263 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.796260 28251 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.795241 28294 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:52.767262 27955 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:52.786369 28052 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:52.765983 28212 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:52.841907 21856 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:52.805199 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:52.805656 21935 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [15.9-17.10]
I20260709 13:07:52.777503 28050 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:07:52.837916 27998 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:52.845206 21859 raft_consensus.cc:2368] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19.
I20260709 13:07:52.834944 28138 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.767649 28033 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.847769 21935 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:07:52.857967 21899 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 2.165s (SENT)
I20260709 13:07:52.855965 22590 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:52.836144 28251 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.838835 27955 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.838932 28149 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.868501 21972 leader_election.cc:400] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19.
I20260709 13:07:52.843984 28212 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.871686 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2542) took 1697 ms. Trace:
I20260709 13:07:52.868294 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.872102 21860 rpcz_store.cc:276] 0709 13:07:51.174801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.187105 (+ 12304us) service_pool.cc:224] Handling call
0709 13:07:52.871511 (+1684406us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400259754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.871631 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374790528}
I20260709 13:07:52.872610 22590 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.868861 21972 leader_election.cc:403] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:07:52.870942 21853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 2879) took 2172 ms (client timeout 1829 ms). Trace:
I20260709 13:07:52.871059 21856 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:52.874086 21853 rpcz_store.cc:269] 0709 13:07:50.698785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.722786 (+ 24001us) service_pool.cc:224] Handling call
0709 13:07:52.870682 (+2147896us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400017f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.870863 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729991040}
I20260709 13:07:52.875360 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:52.839007 28065 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.879212 21860 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:52.839455 28228 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.881393 22589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1265) took 3029 ms (client timeout 1665 ms). Trace:
I20260709 13:07:52.867511 27955 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.892432 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:07:52.905275 21853 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:52.843609 28052 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:52.902099 22589 rpcz_store.cc:269] 0709 13:07:49.846154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.846353 (+   199us) service_pool.cc:224] Handling call
0709 13:07:52.871971 (+3025618us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b54007300d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:52.875480 (+  3509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5627369728}
I20260709 13:07:52.888906 28294 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:07:52.923944 28303 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 13:07:52.924988 28252 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:07:52.925869 28252 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:52.926208 28252 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:52.927091 28252 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.921639 28278 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.877990 28149 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.929260 28278 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:52.930291 28308 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:07:52.881469 28001 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:52.929579 28309 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:07:52.867686 27961 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:07:52.928256 28252 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:52.936100 28033 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.937356 28033 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:52.937862 28033 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.938230 21858 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:52.939896 28033 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:52.952636 28258 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.953493 28252 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:52.954160 28252 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:52.954623 22002 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:52.956146 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:52.957278 22585 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.966090 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:52.968760 22592 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:52.974858 21854 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:52.976647 28303 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.977253 28303 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:52.979604 28303 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:52.996588 21935 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [17.9-19.10]
W20260709 13:07:53.003482 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.006232 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 161682us
I20260709 13:07:53.011272 22585 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:53.008975 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:53.023133 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:52.999529 28050 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.043334 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:53.045152 21935 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
W20260709 13:07:53.059331 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 57557.6us
I20260709 13:07:53.061437 21857 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:07:53.084671 28307 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
W20260709 13:07:53.092006 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.037588 28050 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.107856 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.088315 27934 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:53.129433 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:53.130272 21856 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:53.134765 28252 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:53.136535 22585 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:07:53.100549 28001 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19.
W20260709 13:07:53.144296 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 126208us
W20260709 13:07:53.147480 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.148097 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.155418 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:53.163560 21928 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:53.168195 21930 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:53.127084 28149 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:53.173540 28172 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:53.124075 27934 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:53.168496 28256 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:53.199587 21855 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:07:53.203297 28052 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:53.205387 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:53.177246 27998 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:53.181516 28172 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.186050 28116 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:53.213644 28312 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:07:53.196728 28256 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:53.220814 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:53.232919 21851 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:53.231429 22589 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:53.222013 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.234740 21860 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 11.
I20260709 13:07:53.235387 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:53.236941 22588 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:53.240284 21860 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:07:53.240710 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1219) took 1242 ms. Trace:
I20260709 13:07:53.241281 22588 rpcz_store.cc:276] 0709 13:07:51.998189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.998344 (+   155us) service_pool.cc:224] Handling call
0709 13:07:53.234917 (+1236573us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:53.240647 (+  5730us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5552384}
I20260709 13:07:53.219228 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.247377 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.220431 28172 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:53.254622 21860 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:53.262164 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.264484 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.266305 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1228) took 1007 ms. Trace:
I20260709 13:07:53.268465 22004 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:53.268838 22008 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:53.273168 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.275205 21860 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:53.220216 27998 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:53.275734 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.275964 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.253238 28112 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.279489 22000 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:53.282307 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.284518 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.267710 28001 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:53.286391 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.287230 22004 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:53.294901 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.295488 22004 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:53.288784 22008 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:53.297509 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.298882 21856 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:07:53.282159 28321 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:07:53.286247 28112 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:53.307585 28286 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.009s	user 0.000s	sys 0.005s
I20260709 13:07:53.307889 28112 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:53.309371 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.310825 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.311432 22000 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:07:53.312281 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.311892 28256 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:53.313181 28256 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:53.313392 28320 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:07:53.315464 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1270) took 1881 ms. Trace:
W20260709 13:07:53.316210 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.576s (SENT)
I20260709 13:07:53.316656 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:53.315482 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3020) took 1009 ms. Trace:
I20260709 13:07:53.316017 22009 rpcz_store.cc:276] 0709 13:07:51.434125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.078792 (+644667us) service_pool.cc:224] Handling call
0709 13:07:52.079132 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.155406 (+ 76274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.315222 (+1159816us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b540045bad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:53.315406 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172205952}
I20260709 13:07:53.316015 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.319906 22001 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 11: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:53.319934 21852 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:53.317251 21927 rpcz_store.cc:276] 0709 13:07:52.306134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.846372 (+540238us) service_pool.cc:224] Handling call
0709 13:07:52.863020 (+ 16648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:52.863501 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.863542 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:52.878904 (+ 15362us) log.cc:844] Serialized 26 byte log entry
0709 13:07:53.239600 (+360696us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:53.239637 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.239699 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:53.315286 (+ 75587us) raft_consensus.cc:1685] finished
0709 13:07:53.315303 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.315428 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1244,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2382,"thread_start_us":271162,"threads_started":1,"wal-append.queue_time_us":402505}
I20260709 13:07:53.321709 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.312870 22000 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:53.323246 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.323712 28323 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:53.322861 22003 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:53.324244 28251 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.325182 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.326484 28251 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:53.326773 22001 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:07:53.327154 22001 pending_rounds.cc:85] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 9
I20260709 13:07:53.327351 22001 pending_rounds.cc:107] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:07:53.327586 22001 raft_consensus.cc:2889] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:53.325330 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.331573 22003 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:53.335052 22004 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:53.335223 22005 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:53.336023 28191 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.337628 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.338343 22009 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:53.339075 21855 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:53.342518 22008 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:53.343029 22008 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:53.344059 22008 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:53.345149 22008 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.235822 27934 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.346470 27934 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:53.295558 28138 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.353237 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.336767 28191 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.361591 21854 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:53.362661 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.374078 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.377084 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.378515 22003 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:53.380939 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.382081 22004 raft_consensus.cc:2368] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20.
I20260709 13:07:53.300843 27955 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:53.385573 28138 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:53.386811 27645 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.353591 28050 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:53.388932 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.392504 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:53.389338 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 113631us
I20260709 13:07:53.390440 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.393095 22008 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:07:53.393157 22004 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:53.393908 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.393479 22008 pending_rounds.cc:85] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 10
I20260709 13:07:53.394649 22008 pending_rounds.cc:107] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:07:53.395030 22008 raft_consensus.cc:2889] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:53.402216 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.402927 22008 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:53.403101 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.406453 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.357652 28001 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.410646 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.408197 21854 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:53.411499 22004 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:53.404881 28263 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.413115 28263 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.418617 28278 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:53.421643 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.422760 22008 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:53.404837 27983 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:53.423280 21851 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:07:53.426848 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.415786 28263 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:53.434019 21851 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:07:53.437211 22004 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:53.794952 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.890074 22000 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.567s	user 0.050s	sys 0.022s
I20260709 13:07:53.890697 22000 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:53.438498 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:53.475208 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.891616 22008 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:53.891862 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:53.477952 22005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 3035) took 2649 ms (client timeout 1701 ms). Trace:
I20260709 13:07:53.779227 22001 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:53.460161 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:53.684340 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.717s (SENT)
W20260709 13:07:53.447654 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.896199 22004 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:53.896661 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.509443 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3011) took 2817 ms (client timeout 1752 ms). Trace:
I20260709 13:07:53.894366 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:53.266816 22592 rpcz_store.cc:276] 0709 13:07:52.258667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.999239 (+740572us) service_pool.cc:224] Handling call
0709 13:07:52.999573 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.999770 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:53.094257 (+ 94487us) log.cc:844] Serialized 14 byte log entry
0709 13:07:53.265940 (+171683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:53.265993 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:53.266048 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.266113 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.266251 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196296,"prepare.run_cpu_time_us":1569,"prepare.run_wall_time_us":9148,"raft.queue_time_us":106,"thread_start_us":253171,"threads_started":2}
I20260709 13:07:53.899431 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:53.894068 22003 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
W20260709 13:07:53.898157 21934 rpcz_store.cc:269] 0709 13:07:50.691496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.724577 (+ 33081us) service_pool.cc:224] Handling call
0709 13:07:53.509211 (+2784634us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400206a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:53.509357 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3107413888}
W20260709 13:07:53.892664 22005 rpcz_store.cc:269] 0709 13:07:50.828977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.874768 (+ 45791us) service_pool.cc:224] Handling call
0709 13:07:53.477729 (+2602961us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400168854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:53.477866 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5465729792}
W20260709 13:07:53.905652 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 221356us
W20260709 13:07:53.600162 28212 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.711s	user 0.007s	sys 0.005s
W20260709 13:07:53.779184 28172 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.002s
W20260709 13:07:53.488897 28316 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.251s	user 0.000s	sys 0.000s
I20260709 13:07:53.913246 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:53.912127 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.945815 22004 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:53.953383 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3033) took 3125 ms. Trace:
I20260709 13:07:53.466852 28258 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:53.910267 28336 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:07:53.957088 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.464967 28033 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:53.950700 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.730746 27998 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:53.841324 28001 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:53.636668 28323 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:53.891203 27955 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:53.904913 28263 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.953815 22006 rpcz_store.cc:276] 0709 13:07:50.828391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.326437 (+498046us) service_pool.cc:224] Handling call
0709 13:07:52.635329 (+1308892us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:52.654071 (+ 18742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.654103 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:52.654672 (+   569us) log.cc:844] Serialized 52 byte log entry
0709 13:07:52.863893 (+209221us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:52.863948 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.864009 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:53.340862 (+476853us) raft_consensus.cc:1685] finished
0709 13:07:53.340894 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.951103 (+610209us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540040d150. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:53.953314 (+  2211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":541,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":3009142144,"thread_start_us":208681,"threads_started":1,"wal-append.queue_time_us":371509}
I20260709 13:07:53.904685 27915 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:53.954206 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.959807 28033 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:53.947350 22003 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:53.910986 28052 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:53.947965 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.912806 27983 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:07:53.961851 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1283) took 1344 ms. Trace:
I20260709 13:07:53.963148 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2556) took 1272 ms. Trace:
W20260709 13:07:53.974118 27934 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.000s
I20260709 13:07:53.964380 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2569) took 1023 ms. Trace:
I20260709 13:07:53.965535 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1123) took 3055 ms. Trace:
I20260709 13:07:53.970263 22005 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:53.985191 21857 rpcz_store.cc:276] 0709 13:07:52.940846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.940970 (+   124us) service_pool.cc:224] Handling call
0709 13:07:52.941292 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.063543 (+122251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.961164 (+897621us) spinlock_profiling.cc:243] Waited 98.5 ms on lock 0x7b540025ab54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:53.964308 (+  3144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226513280}
W20260709 13:07:53.981209 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.953425 28172 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.960142 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.982692 21858 rpcz_store.cc:276] 0709 13:07:52.691251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.839707 (+148456us) service_pool.cc:224] Handling call
0709 13:07:52.937980 (+ 98273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:52.938698 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.962973 (+1024275us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b54003d3950. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:53.963093 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182312320}
I20260709 13:07:53.982120 22002 rpcz_store.cc:276] 0709 13:07:52.617474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.617653 (+   179us) service_pool.cc:224] Handling call
0709 13:07:52.617990 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.955151 (+337161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.961646 (+1006495us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b540042bdd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:53.961793 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773701504}
I20260709 13:07:53.990823 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:53.984372 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:53.951613 28212 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:53.971915 22009 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.002s
I20260709 13:07:54.001732 22009 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
W20260709 13:07:53.986589 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.004746 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.985896 21926 rpcz_store.cc:276] 0709 13:07:50.910595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.480525 (+569930us) service_pool.cc:224] Handling call
0709 13:07:53.509644 (+2029119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.509941 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:53.509970 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:53.510014 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:53.510055 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.510114 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.965377 (+455263us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400206a50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:53.965481 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4665470336}
I20260709 13:07:53.986177 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.006296 22006 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:07:53.991497 21853 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:53.970831 27998 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:54.017491 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2486) took 2387 ms (client timeout 2115 ms). Trace:
I20260709 13:07:53.975965 28323 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:53.978777 28324 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:53.985702 22008 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:53.994987 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.023357 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:54.019055 21933 rpcz_store.cc:269] 0709 13:07:51.629217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.247635 (+618418us) service_pool.cc:224] Handling call
0709 13:07:54.016355 (+1768720us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:54.016654 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73375616}
I20260709 13:07:54.020721 22000 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:07:54.019805 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:54.024776 22009 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:54.026166 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:54.027120 22009 raft_consensus.cc:2368] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:54.024168 22006 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 11.
I20260709 13:07:54.023912 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.028144 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.028857 22593 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:07:54.031164 22584 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:54.033497 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.034163 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.034534 22004 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:53.980824 28263 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.036202 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:54.038729 28116 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.039391 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:54.040684 21851 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:54.041157 22006 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:54.034790 28228 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.042157 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:53.989288 27915 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.044633 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.047683 28228 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:54.051221 21859 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:54.043478 28033 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.051632 21859 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:54.052107 28033 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:54.052510 21859 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.053857 21859 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:54.025909 27998 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.070923 21859 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:07:54.078696 22003 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:54.078388 28341 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:07:54.072789 21823 connection.cc:256] RPC call timeout handler was delayed by -1.7979159550000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3187s for a response
W20260709 13:07:54.081369 21823 leader_election.cc:336] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.724s (SENT)
I20260709 13:07:54.081842 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:54.083956 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.085057 21853 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:54.085265 28172 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:54.081362 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.093628 21855 raft_consensus.cc:2410] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:07:54.093971 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.042332 28116 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.096526 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.102807 28263 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.107317 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2481) took 2487 ms. Trace:
I20260709 13:07:54.108136 27873 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.092581 27983 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:54.109297 28224 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.121172 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:54.107863 21929 rpcz_store.cc:276] 0709 13:07:51.619796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.767704 (+1147908us) service_pool.cc:224] Handling call
0709 13:07:52.768123 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:52.790418 (+ 22295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.790451 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:52.804045 (+ 13594us) log.cc:844] Serialized 52 byte log entry
0709 13:07:52.977475 (+173430us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:53.024421 (+ 46946us) log.cc:844] Serialized 14 byte log entry
0709 13:07:53.487565 (+463144us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:54.106680 (+619115us) spinlock_profiling.cc:243] Waited 8.9 ms on lock 0x7b18002d1d80. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:07:54.106893 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.107006 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:54.107053 (+    47us) raft_consensus.cc:1685] finished
0709 13:07:54.107069 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:54.107244 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30551,"prepare.queue_time_us":27510,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":1108848,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":201,"spinlock_wait_cycles":37383296,"thread_start_us":157526,"threads_started":2,"wal-append.queue_time_us":332796}
I20260709 13:07:54.110154 28006 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.108872 27873 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.131608 28006 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.134009 27873 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.137705 21934 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:07:54.160233 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 165317us
I20260709 13:07:54.135861 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:54.161863 21899 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 1.511s (SENT)
I20260709 13:07:54.162161 21926 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:54.136260 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:54.094929 27934 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:54.165956 27934 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:54.166287 28006 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:54.166581 28256 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.853s	user 0.014s	sys 0.000s
I20260709 13:07:54.135488 27873 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.166906 27934 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.167618 27873 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.168280 27998 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.168668 27915 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.168844 27998 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:54.169265 27915 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:54.169373 27998 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.166919 28006 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:54.171096 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:54.171608 28116 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.109926 28224 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.171854 21928 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:54.172362 28212 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.173542 28224 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.173970 28323 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.174852 27983 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:54.175302 27983 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.176249 21929 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:54.079711 28228 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:54.177233 21931 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:54.178742 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:54.182662 21929 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:54.180047 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:54.146155 21932 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:54.184247 21935 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:07:54.189478 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.190924 21928 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:54.193250 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:54.257258 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:54.203114 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 310913us
I20260709 13:07:54.169855 27873 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:54.220100 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 174338us
I20260709 13:07:54.175668 27983 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:54.258615 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.259202 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.259732 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:54.260676 28116 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:54.131623 28324 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:54.261216 28116 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:54.261590 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:54.262591 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.262343 28116 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:54.263602 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 350462us
I20260709 13:07:54.264088 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.265182 27873 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:54.265650 22006 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:54.266244 22006 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:54.266084 27873 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:54.267030 22006 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.267417 28346 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:54.268066 22006 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:54.269754 28001 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:54.270380 28001 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:54.270633 28191 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.271592 22008 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:54.171624 27998 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.275318 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.123590 28172 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:07:54.288194 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1279) took 2290 ms. Trace:
I20260709 13:07:54.288693 22593 rpcz_store.cc:276] 0709 13:07:51.997458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.997598 (+   140us) service_pool.cc:224] Handling call
0709 13:07:53.219907 (+1222309us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aae9c 00007f3b6d6aaa5b 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:53.220079 (+   172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:54.029307 (+809228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.286675 (+257368us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54006266d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:54.288122 (+  1447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877315968}
I20260709 13:07:54.289707 21756 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:54.276178 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 194829us
I20260709 13:07:54.197671 28228 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:54.275629 28352 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:54.277820 28340 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:54.272099 28050 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:54.268085 28346 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:54.277977 28001 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.278549 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.279181 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:54.305784 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.306479 21854 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:54.307741 22004 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:54.279641 27983 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.311199 27983 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.281414 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.282460 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.282714 22590 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:54.283017 28353 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:53.410053 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:54.284488 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:54.290560 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:54.318059 21931 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:54.295001 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.289482 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:54.197693 27955 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:07:54.300083 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:54.325320 22588 raft_consensus.cc:2410] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
W20260709 13:07:54.276010 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.301708 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.294795 21756 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 13, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 13 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:54.276512 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 116591us
I20260709 13:07:54.328088 21858 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:07:54.267625 27873 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:54.334095 21926 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
W20260709 13:07:54.339208 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49827.7us
W20260709 13:07:54.314131 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 310665us
I20260709 13:07:54.365023 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:54.369156 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:54.370034 21926 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:54.301787 28349 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.372164 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:54.374334 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1414) took 2009 ms. Trace:
I20260709 13:07:54.372802 21931 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:54.370059 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:54.298864 27998 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:54.302713 28050 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.378000 21756 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:54.367412 27970 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:54.413764 27970 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:54.413553 21756 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.415328 21756 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.415376 27970 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.420562 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:54.421311 22004 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:54.392781 21926 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:54.378821 28344 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:54.378809 28116 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:54.441449 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 159040us
I20260709 13:07:54.449129 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.441474 21931 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
W20260709 13:07:54.441841 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 141875us
I20260709 13:07:54.451269 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:54.433706 21756 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.417541 28050 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:54.423513 27873 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.433230 27970 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:54.371747 27983 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:07:54.477970 21756 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.478988 22004 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:07:54.471205 28350 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:07:54.494570 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 113648us
I20260709 13:07:54.411808 28349 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.423815 27998 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.501142 21756 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.476301 28352 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:54.525979 21756 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:54.545311 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.530652 21753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 283) took 77377 ms (client timeout 15000 ms). Trace:
I20260709 13:07:54.483103 27873 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:54.577831 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.489678 28348 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:54.506170 28323 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:54.589208 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 224687us
I20260709 13:07:54.626921 21824 leader_election.cc:400] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20.
W20260709 13:07:54.649745 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 188430us
I20260709 13:07:54.651053 21824 leader_election.cc:403] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:54.651752 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.580032 28052 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.524654 27955 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:54.655982 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 156666us
I20260709 13:07:54.653532 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:54.652328 28323 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.581123 28353 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:54.662817 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.682215 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55454.7us
W20260709 13:07:54.683753 21934 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.003s	sys 0.006s
I20260709 13:07:54.528312 21756 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 13, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 13 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:07:54.491863 28346 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.653643 28052 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.625032 28362 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:54.591675 28367 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:54.687209 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:54.625845 28352 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
W20260709 13:07:54.691120 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.701370 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 250185us
W20260709 13:07:54.723673 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70243.3us
I20260709 13:07:54.693758 21933 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:54.723716 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.724378 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:54.695678 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2913) took 1436 ms. Trace:
W20260709 13:07:54.708541 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 427171us
I20260709 13:07:54.698381 28369 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20.
W20260709 13:07:54.687673 21934 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.003s	sys 0.006s
I20260709 13:07:54.717343 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3056) took 1365 ms. Trace:
I20260709 13:07:54.717767 21756 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 6 to 10, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 10 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:07:54.720652 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.729687 22001 rpcz_store.cc:276] 0709 13:07:53.351753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.907680 (+555927us) service_pool.cc:224] Handling call
0709 13:07:53.954772 (+ 47092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.955002 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:54.299290 (+344288us) log.cc:844] Serialized 14 byte log entry
0709 13:07:54.689725 (+390435us) log.cc:844] Serialized 14 byte log entry
0709 13:07:54.716806 (+ 27081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:54.716868 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:54.716919 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.716993 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:54.717179 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65257,"prepare.queue_time_us":30382,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":184878,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":594486,"threads_started":3}
I20260709 13:07:54.691270 28033 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:54.724965 22590 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:07:54.724581 21899 leader_election.cc:400] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:07:54.725685 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:54.684206 28370 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:54.727032 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:54.741408 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.741758 22587 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:54.742128 22002 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:07:54.744010 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.745062 22004 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
W20260709 13:07:54.720924 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.694759 27998 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.739058 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.727691 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.750135 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.750921 22591 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:07:54.750705 28353 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.752399 28376 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:54.753357 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.725188 22007 rpcz_store.cc:276] 0709 13:07:53.259217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.271379 (+ 12162us) service_pool.cc:224] Handling call
0709 13:07:53.284538 (+ 13159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.424641 (+140103us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:53.636951 (+212310us) log.cc:844] Serialized 14 byte log entry
0709 13:07:53.744165 (+107214us) log.cc:844] Serialized 14 byte log entry
0709 13:07:53.744613 (+   448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:53.744679 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:53.744759 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.047058 (+302299us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b18004b86e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:07:54.694294 (+647236us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b18004b86e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419
0709 13:07:54.695431 (+  1137us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003890d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:54.695480 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:54.695615 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":848935,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":6854,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":15628,"spinlock_wait_cycles":691785216,"thread_start_us":291129,"threads_started":2,"wal-append.queue_time_us":339241}
W20260709 13:07:54.750339 28358 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:07:54.754195 22588 raft_consensus.cc:2368] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:54.769670 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.771587 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.772940 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.695179 27873 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.757437 22005 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:54.691460 28386 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
W20260709 13:07:54.759156 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72006.9us
I20260709 13:07:54.784206 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:54.759910 27998 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:54.760443 28052 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.628063 28348 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.799587 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.736374 28033 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:54.761302 27645 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:54.761968 28372 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:54.765184 28369 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:54.765645 28359 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:07:54.765836 28362 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.766309 28368 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:54.766654 28349 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:54.766883 28383 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:54.767041 28006 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:54.833554 28006 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:54.585845 21753 rpcz_store.cc:269] 0709 13:06:36.934529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.966960 (+ 32431us) service_pool.cc:224] Handling call
0709 13:06:37.586706 (+619746us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:54.286360 (+76699654us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:37.854044 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:37.854798 (+   754us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:37.854855 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:07:37.854872 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:37.855661 (+   789us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:37.855748 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:37.855784 (+    36us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:07:37.856381 (+   597us) tablet.cc:701] Row locks acquired
0709 13:07:37.856439 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:07:37.856620 (+   181us) write_op.cc:270] Start()
0709 13:07:37.856756 (+   136us) write_op.cc:276] Timestamp: P: 1783602457856541 usec, L: 0
0709 13:07:37.856801 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:07:38.038684 (+181883us) log.cc:844] Serialized 2147 byte log entry
0709 13:07:38.224039 (+185355us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cae28 00007f3b74205581 00007f3b74204d58 00007f3b742099f1 00007f3b742099a9 00007f3b74209939 00007f3b74209901 00007f3b74208b2d 000000000038f354 000000000038f199
0709 13:07:39.296331 (+1072292us) op_driver.cc:464] REPLICATION: finished
0709 13:07:39.481134 (+184803us) write_op.cc:301] APPLY: starting
0709 13:07:39.481247 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:39.605272 (+124025us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:39.605321 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:39.608711 (+  3390us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:39.608990 (+   279us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:07:39.621224 (+ 12234us) write_op.cc:312] APPLY: finished
0709 13:07:39.621593 (+   369us) log.cc:844] Serialized 57 byte log entry
0709 13:07:39.622445 (+   852us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:39.622609 (+   164us) write_op.cc:454] Released schema lock
0709 13:07:39.622673 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:39.803161 (+180488us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b54003ed6a8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b76b8b9e1 00007f3b76b8a64c 00007f3b7421b3fd 00007f3b74207389 00007f3b74206b53 00007f3b7420ad61 00007f3b7420ad19 00007f3b7420aca9 00007f3b7420ac71 00007f3b74209e9d 000000000038f354
0709 13:07:39.803762 (+   601us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:39.803793 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3033946,"reported_tablets":48,"spinlock_wait_cycles":224268544,"child_traces":[["op",{"apply.queue_time_us":184391,"apply.run_cpu_time_us":15257,"apply.run_wall_time_us":322661,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25057,"num_ops":5,"prepare.queue_time_us":351734,"prepare.run_cpu_time_us":36443,"prepare.run_wall_time_us":370286,"raft.queue_time_us":948026,"raft.run_cpu_time_us":11952,"raft.run_wall_time_us":119247,"replication_time_us":1439341,"spinlock_wait_cycles":386058752,"thread_start_us":1525350,"threads_started":4,"wal-append.queue_time_us":124682,"wal-append.run_cpu_time_us":19973,"wal-append.run_wall_time_us":2469179}]]}
I20260709 13:07:54.804386 27998 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:54.697194 28374 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:54.836314 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.836603 27998 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.813874 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.841143 22588 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:07:54.780082 22006 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.000s
I20260709 13:07:54.782625 22001 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:54.766882 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.793953 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:54.843250 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49360.5us
I20260709 13:07:54.761197 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:54.799546 22591 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:54.736187 28367 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:54.808416 28372 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:07:54.810364 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.782137 28352 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.792330 27915 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:54.813128 28065 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:54.824651 28378 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:54.828836 28323 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.795083 28363 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:54.848958 22587 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:54.850657 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.794965 27983 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.864758 28054 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:54.844173 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:54.860354 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.870187 28374 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.863261 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:54.893150 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49196.9us
W20260709 13:07:54.864880 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 94622us
W20260709 13:07:54.893877 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.888615 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162967us
I20260709 13:07:54.804342 27873 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.892715 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.876572 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:54.877471 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.821920 27645 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:54.878145 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:54.921697 22005 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:54.859083 28349 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:54.883999 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.922511 21852 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:07:54.923393 28343 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:54.923846 28343 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.924187 28343 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:54.836395 28006 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.860354 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.925597 21855 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
W20260709 13:07:54.854573 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93467.1us
I20260709 13:07:54.865682 22006 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:54.926829 21853 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:54.894694 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:54.832026 28348 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:54.894852 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:54.894583 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:54.918292 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:54.918416 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:54.822422 28369 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.921514 22591 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:54.927620 22000 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:54.864964 28377 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:54.374845 22585 rpcz_store.cc:276] 0709 13:07:52.364767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.137680 (+772913us) service_pool.cc:224] Handling call
0709 13:07:53.138076 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.138375 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:53.304817 (+166442us) log.cc:844] Serialized 14 byte log entry
0709 13:07:54.027336 (+722519us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18003ef3e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:54.373679 (+346343us) log.cc:844] Serialized 14 byte log entry
0709 13:07:54.374001 (+   322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:54.374047 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:54.374099 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.374166 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:54.374280 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27773,"prepare.queue_time_us":112592,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":2020,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":12629,"spinlock_wait_cycles":26302336,"thread_start_us":503865,"threads_started":3,"wal-append.queue_time_us":963812}
I20260709 13:07:54.951493 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:54.889575 28054 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:54.889644 28365 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.893095 27955 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.892560 28389 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.975826 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:54.987167 21855 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:54.989012 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:54.973102 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.858196 28383 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.856971 28362 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.873221 28352 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:54.994390 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 99816.7us
I20260709 13:07:55.006579 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:55.007694 28052 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.011109 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:54.995163 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 100667us
W20260709 13:07:54.998052 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 79717.9us
I20260709 13:07:54.974314 28388 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:55.019691 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.004689 21851 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:54.864543 28368 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.000530 28348 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:54.926369 28376 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:55.013036 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:54.931599 27873 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:54.892506 27983 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.885196 22588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a91310e68464d64aa8324092182518f"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:55.009747 28394 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:55.030580 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:55.015570 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:55.008180 28385 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:54.972267 27645 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.040362 22588 raft_consensus.cc:606] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Received request to transfer leadership to TS 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:55.052405 22588 raft_consensus.cc:614] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:54.889899 28033 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.024183 28258 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:55.008746 28191 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:55.009857 28380 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:54.973593 28363 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:07:55.019590 28388 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.949275 28374 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:54.999923 27955 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.047370 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:55.075248 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 87722.5us
I20260709 13:07:55.042434 28360 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:55.046953 28394 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.015313 28052 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.000914 28389 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.057039 28006 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:55.071722 28324 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.794s	user 0.010s	sys 0.003s
I20260709 13:07:54.994192 28365 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.059303 28390 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:55.088485 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.126134 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 137227us
I20260709 13:07:55.045617 27970 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:07:55.059530 28385 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:55.087692 28258 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.057574 28224 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:55.123355 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 111544us
I20260709 13:07:55.108453 28065 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:55.076833 28388 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:55.081349 28360 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.060271 28323 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:07:55.078506 28191 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:55.129974 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82705.1us
W20260709 13:07:55.116348 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.134723 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.118264 28369 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.036859 28367 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.129084 28390 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:55.140813 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.045269 28054 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.088986 28380 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:55.129168 28324 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:55.140208 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:55.039376 28372 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.156702 28191 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:55.082444 28397 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:07:55.130875 28394 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:55.140313 28345 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 13:07:55.151521 28360 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:55.154347 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 72701.9us
I20260709 13:07:55.133694 28377 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:55.175292 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.144292 28224 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.161777 28380 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:55.139606 28258 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:55.219147 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 84687.9us
W20260709 13:07:55.220498 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69251.4us
I20260709 13:07:55.157830 28390 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:55.210125 28371 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
W20260709 13:07:55.226953 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 86844.5us
I20260709 13:07:55.224648 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:55.168501 28403 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:55.167569 28054 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.159457 28065 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.241065 22588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f588420255042dda396ba6419abf6bc"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:55.245065 22588 raft_consensus.cc:606] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:55.245507 22588 raft_consensus.cc:614] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:55.218273 28324 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:55.229130 28369 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:55.253242 28369 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:55.170856 28370 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:55.257231 27983 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.251179 28324 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:55.260456 28050 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.778s	user 0.005s	sys 0.005s
I20260709 13:07:55.262130 28065 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.262693 28065 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.262856 28224 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.263161 28065 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:55.221225 28399 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:55.252471 22592 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:55.235894 22557 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:55.266820 28365 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.267506 27873 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.165593 28400 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:07:55.269234 28409 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:55.269608 28258 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:55.270043 28409 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:55.233664 28401 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:07:55.220245 28402 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:07:55.271106 28401 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:55.274677 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.275990 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.265815 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.277282 22009 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:07:55.277194 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.277807 22000 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 13.
I20260709 13:07:55.234987 28380 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:55.281234 21974 leader_election.cc:400] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:55.284364 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:55.279611 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.242298 28403 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.290999 21974 leader_election.cc:403] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:55.234437 28006 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.301682 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.293152 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:55.295027 22590 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:07:55.296950 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 56833.9us
I20260709 13:07:55.244232 28383 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.248871 28352 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:55.309386 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 84831.2us
I20260709 13:07:55.312805 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:55.260520 28324 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:55.315553 22008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.003s	sys 0.018s
I20260709 13:07:55.310370 22588 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEARNER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:55.316807 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.171322 28340 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.125s
W20260709 13:07:55.383229 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.314877 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.009s	sys 0.013s
I20260709 13:07:55.165688 28376 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.290138 28401 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:55.408833 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89425.3us
W20260709 13:07:55.395201 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.390916 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 56898.8us
W20260709 13:07:55.388001 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.009s	sys 0.016s
I20260709 13:07:55.260905 28050 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:55.390877 22008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.009s	sys 0.023s
W20260709 13:07:55.389186 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 158729us
W20260709 13:07:55.439999 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.440393 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.440511 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.271163 28409 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.442044 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:55.442600 21854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.023s	sys 0.078s
I20260709 13:07:55.300000 28403 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.270154 28408 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.118s
W20260709 13:07:55.443819 21854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.023s	sys 0.078s
I20260709 13:07:55.289779 27970 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.102s
W20260709 13:07:55.311327 28345 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:07:55.274353 28406 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.145s
I20260709 13:07:55.386521 28383 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:55.242318 28385 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.307567 28368 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.458745 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:55.308219 28362 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:55.452082 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 64370.7us
W20260709 13:07:55.441268 28349 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.005s
I20260709 13:07:55.453559 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:55.437608 28401 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:55.488605 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.490612 21932 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:55.490924 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.490613 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:55.488883 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:55.492421 21926 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:55.493342 21931 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:55.494390 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.324613 28405 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.161s
I20260709 13:07:55.449105 27955 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:55.505131 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51699.8us
I20260709 13:07:55.510926 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:55.501562 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58235.6us
I20260709 13:07:55.508049 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.512647 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.504552 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.385690 28377 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.507297 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.517999 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.519160 21931 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:55.463699 28374 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.473230 28349 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:55.311126 28006 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.518088 21932 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:55.517978 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.527257 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1416) took 3086 ms. Trace:
I20260709 13:07:55.513473 21933 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:55.448482 28413 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17.
I20260709 13:07:55.542215 21859 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:55.525883 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.527429 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:55.530884 21972 connection.cc:256] RPC call timeout handler was delayed by -1.1819140530000141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 13:07:55.533349 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.545780 22588 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:55.535246 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.505755 27873 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:55.564083 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.537945 22589 rpcz_store.cc:276] 0709 13:07:52.440651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.238705 (+798054us) service_pool.cc:224] Handling call
0709 13:07:53.239066 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:53.239445 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:53.239474 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:53.468625 (+229151us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b2c001465b8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:53.469263 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 13:07:54.273397 (+804134us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:54.526733 (+253336us) log.cc:844] Serialized 14 byte log entry
0709 13:07:54.528606 (+  1873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.528682 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:55.515459 (+986777us) raft_consensus.cc:1685] finished
0709 13:07:55.527045 (+ 11586us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:55.527192 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":105026,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":673,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":9883,"spinlock_wait_cycles":7942016,"thread_start_us":557375,"threads_started":2,"wal-append.queue_time_us":915174}
I20260709 13:07:55.544421 22590 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:55.502430 28389 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.566448 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.566210 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.549577 21859 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:55.567063 22589 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:55.465907 28365 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 13:07:55.571846 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.572479 22587 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:07:55.577090 21856 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:07:55.578049 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67264us
I20260709 13:07:55.540635 28397 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.579824 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.581629 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.582686 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:55.583227 21860 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.587080 21859 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:55.592830 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:55.596169 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:55.598903 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1220) took 3600 ms. Trace:
I20260709 13:07:55.597064 22590 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:55.602917 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:55.603971 21856 raft_consensus.cc:2376] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21. Re-sending same reply.
I20260709 13:07:55.599423 22584 rpcz_store.cc:276] 0709 13:07:51.998858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.999051 (+   193us) service_pool.cc:224] Handling call
0709 13:07:53.469638 (+1470587us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b58000c6398. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:53.469935 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:54.036448 (+566513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.598601 (+1562153us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b5400668fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:55.598835 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289677952}
I20260709 13:07:55.571542 28383 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:55.589184 28393 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.256s	user 0.002s	sys 0.000s
I20260709 13:07:55.580273 28065 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.579533 28421 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:55.589130 27970 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:55.638846 28052 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.595487 28397 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.644407 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.645396 21856 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:55.645596 28419 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:55.646525 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:55.647300 21856 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:55.647506 28410 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:07:55.647811 28397 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:55.649176 28415 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:55.649915 28415 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:55.656352 21856 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:55.631641 28407 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:55.646611 28418 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:07:55.674454 21860 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:55.677469 21856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10"
dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:52006
W20260709 13:07:55.677291 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 382663us
I20260709 13:07:55.692082 21856 raft_consensus.cc:606] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:55.661707 28050 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.657986 28415 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:55.693727 21856 raft_consensus.cc:614] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:55.689337 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 93340us
I20260709 13:07:55.700235 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:55.691350 28388 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.001s
I20260709 13:07:55.649832 28383 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.704325 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:55.659638 27970 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.693377 28407 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:55.711333 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.705247 28224 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:55.713987 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.714712 21932 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:07:55.721174 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.722715 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.723070 28340 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.723611 28340 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.723956 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.724681 21933 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:55.730731 28340 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:55.733493 21934 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:55.733806 21852 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:07:55.709360 28434 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.711076 28389 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:55.737572 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.739024 21854 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:55.745146 22588 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:55.745498 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.695295 28413 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:07:55.718551 28224 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.762884 28224 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:55.719972 28420 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:07:55.766501 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2903) took 1469 ms. Trace:
I20260709 13:07:55.767079 28413 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:55.767529 28413 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:55.706660 28388 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.767097 21857 rpcz_store.cc:276] 0709 13:07:54.297225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.297465 (+   240us) service_pool.cc:224] Handling call
0709 13:07:54.439052 (+141587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:54.439234 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.439258 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:54.439300 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:54.439337 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.439398 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:55.766244 (+1326846us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540010cf50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:55.766415 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324859776}
I20260709 13:07:55.756121 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:55.762363 28424 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:07:55.777182 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 52655.8us
I20260709 13:07:55.759348 28381 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 13:07:55.766582 27970 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:55.781315 28344 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:55.795426 21855 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:55.796307 21972 connection.cc:256] RPC call timeout handler was delayed by -1.0042377469999337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 13:07:55.796968 22590 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:07:55.795023 28344 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:55.794509 28397 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:07:55.790132 28407 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.772631 28403 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.802126 27915 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:55.794757 28421 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.803911 28421 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.784870 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:55.768369 28413 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.807399 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:55.807926 22590 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.814106 22590 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:55.806159 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:55.804782 28403 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.817788 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:55.772625 28324 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.759812 28054 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:07:55.805092 28116 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.754266 28434 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:55.816907 28191 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.003s
I20260709 13:07:55.824659 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:55.815441 28344 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.820020 28372 raft_consensus.cc:2764] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:55.830899 22585 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:55.820827 28397 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:55.824218 28278 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:55.835336 28402 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:55.835172 28427 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:07:55.838413 28434 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.821363 27970 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 13:07:55.823570 28258 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:07:55.851163 28421 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.832682 28413 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:55.823747 27983 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:07:55.829818 28191 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.845736 28408 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.833740 28278 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:55.857683 28278 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:55.857451 28408 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.859957 28408 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:55.827880 28116 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.842669 28397 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:55.873456 28116 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:55.878894 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.879315 28360 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 13:07:55.879742 28360 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.881016 21928 raft_consensus.cc:2368] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22.
W20260709 13:07:55.882397 28390 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 13:07:55.882901 28390 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:07:55.865562 28394 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.002s
I20260709 13:07:55.885339 28394 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:55.892992 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 866096us
I20260709 13:07:55.893762 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:55.894320 28397 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.895671 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:55.896539 28395 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:55.897071 28395 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.846411 28381 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
W20260709 13:07:55.866017 28343 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.941s	user 0.003s	sys 0.012s
I20260709 13:07:55.897537 28395 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.906306 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:55.888036 28394 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.910063 28343 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.930819 28343 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:55.907752 28380 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.004s
I20260709 13:07:55.931871 28191 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.931701 28380 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.932909 22006 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:55.921506 28390 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:55.934634 28380 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.936658 28278 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.942687 28278 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:55.926982 28395 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.947909 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.948345 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.927692 28415 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:55.905846 28376 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:55.923844 28050 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:55.926594 28369 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:55.949646 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.950325 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.953191 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.953840 21935 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:55.952085 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.952708 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.958256 28324 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.959733 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.960419 21933 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:55.960608 21930 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:07:55.961128 21934 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:55.950539 28412 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:55.962203 28412 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:55.963701 28415 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.964442 28412 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:55.964473 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.965622 21926 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:55.966090 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.961462 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3032) took 1782 ms. Trace:
I20260709 13:07:55.967903 21930 rpcz_store.cc:276] 0709 13:07:54.179126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.179258 (+   132us) service_pool.cc:224] Handling call
0709 13:07:55.959928 (+1780670us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757a8fc0 00007f3b75793929 00007f3b757933d2 00007f3b6ed0abd4
0709 13:07:55.960171 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.961135 (+   964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.961396 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40685440}
I20260709 13:07:55.970939 21933 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:55.977382 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:55.978341 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 162482us
I20260709 13:07:55.979192 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.978279 21930 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:55.980373 21932 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:07:55.980991 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 155538us
I20260709 13:07:55.989255 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:55.990073 21932 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:07:55.995924 28050 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:56.002837 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:55.996899 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.003846 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:55.997535 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.001283 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:56.002985 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 178392us
I20260709 13:07:55.996920 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.010195 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.010998 21851 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:07:55.994695 28278 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:07:56.011953 28376 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:56.014597 28224 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.016871 28224 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:55.999027 28348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.007s	sys 0.001s
I20260709 13:07:56.024343 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:56.026779 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.026595 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.027446 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:56.028072 21857 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:56.023168 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:56.025545 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.029134 28172 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:56.030166 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:56.030778 21851 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 15: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:56.029320 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:56.031780 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.032466 21935 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:56.035000 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.039229 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:56.039391 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:56.039641 21926 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [15.9-17.10]
W20260709 13:07:56.032723 28348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.007s	sys 0.001s
W20260709 13:07:56.040397 28439 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:07:56.034734 28172 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:56.036048 28065 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.051481 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.041155 28324 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:56.057363 28172 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.058300 28360 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.053902 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:56.059336 22592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1292) took 1776 ms (client timeout 1964 ms). Trace:
I20260709 13:07:56.059636 21851 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:56.059638 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.045202 28394 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.045409 28224 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.041584 28348 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:56.059706 22592 rpcz_store.cc:269] 0709 13:07:54.283324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.284229 (+   905us) service_pool.cc:224] Handling call
0709 13:07:56.059157 (+1774928us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54005839d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.059272 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966304000}
I20260709 13:07:56.080211 21857 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:07:56.064173 21899 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:56.064755 28324 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:56.053027 28065 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.066679 28344 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.069198 28394 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.053519 28453 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:56.083374 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:56.087859 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:56.053102 28365 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.134721 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.087836 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.101677 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.136180 21857 raft_consensus.cc:2410] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:07:56.044924 28397 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
W20260709 13:07:56.079672 28448 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.136s	user 0.003s	sys 0.000s
I20260709 13:07:56.111092 28394 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.112617 28446 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:07:56.105365 28453 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.114156 28324 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.116374 28344 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:56.117699 28442 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:07:56.118945 28445 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:07:56.121711 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.121364 28348 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.125370 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.124372 21932 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:07:56.131441 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 43686.9us
I20260709 13:07:56.175540 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.137938 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.175467 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:56.080283 28412 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:56.137848 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.177958 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.178195 22001 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:56.101854 28376 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:56.156667 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.164892 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:56.188959 21851 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:56.165891 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.189767 22007 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:07:56.101825 28001 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:07:56.147378 27934 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:56.080030 28224 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.136520 28374 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:56.172502 21860 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:56.166697 28453 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.087352 28050 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.152388 28415 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:56.176911 21857 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 12: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:56.148669 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60914.4us
I20260709 13:07:56.204385 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.206959 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:56.158021 28256 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:56.182559 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.211046 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.212774 21857 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 13:07:56.213649 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.215267 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.211683 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.217017 21932 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:07:56.183561 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:56.178608 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.191864 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.193096 22002 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:56.195638 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.199774 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:56.154260 28365 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.180446 28395 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.187706 28376 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.190209 28413 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:56.210987 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.226547 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.228139 21929 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:56.225637 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:56.301575 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 79808.7us
I20260709 13:07:56.229871 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:56.221863 27998 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:56.227921 28212 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:56.244135 21860 raft_consensus.cc:2376] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18. Re-sending same reply.
W20260709 13:07:56.247419 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 64618.8us
W20260709 13:07:56.293828 22558 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.20.66.190:46167, user_credentials={real_user=slave}} blocked reactor thread for 47597.9us
W20260709 13:07:56.305065 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 75291.4us
I20260709 13:07:56.305500 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:56.305779 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.300665 21932 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:07:56.294107 28278 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:56.208280 28374 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.307898 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:56.309434 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 99547us
I20260709 13:07:56.313632 28344 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:56.313817 28346 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:56.314327 28346 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.315737 28346 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:56.317452 28446 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:07:56.318509 28459 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:07:56.323367 28460 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:07:56.324005 21935 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:07:56.325310 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.326610 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.327217 21935 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:56.298977 28413 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:56.381098 28413 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:56.385044 28224 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.297079 28376 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:56.385939 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80235.4us
W20260709 13:07:56.386497 28461 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:07:56.388252 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:56.296548 28410 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:07:56.389803 28365 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.314257 28344 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.390947 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:56.343407 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:56.313283 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:56.335999 28001 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:56.390384 28387 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:56.413403 28387 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.413785 28387 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:56.414079 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.382694 28449 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:56.417605 28449 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.418061 28449 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.426220 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.392935 21927 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:56.239192 28454 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:56.406026 28278 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:56.442938 21929 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:07:56.390024 28456 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:07:56.410156 28403 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:56.435357 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3031) took 2257 ms (client timeout 1750 ms). Trace:
I20260709 13:07:56.437820 21935 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:56.308785 28397 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:56.442260 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51388.2us
I20260709 13:07:56.469944 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:56.408980 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 95746us
I20260709 13:07:56.471596 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:56.441502 22008 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:56.308856 28172 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:56.464591 21927 rpcz_store.cc:269] 0709 13:07:54.178259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.204141 (+ 25882us) service_pool.cc:224] Handling call
0709 13:07:56.125190 (+1921049us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:56.435250 (+310060us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42081664}
I20260709 13:07:56.406908 28343 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:56.473721 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:56.437870 28191 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:56.437273 28389 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.439431 28454 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:56.446148 28390 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:56.462251 28001 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.418998 28405 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:07:56.533996 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:56.480119 28172 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.467585 28403 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:56.536038 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62412.3us
I20260709 13:07:56.561049 21853 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:56.522085 28348 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:56.524485 28191 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.562163 28480 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:56.562301 28341 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:56.595520 28413 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.533231 28454 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:56.566788 28348 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:56.596666 28348 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:56.597134 28387 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.467597 28278 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.533541 28054 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:56.595301 28480 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:56.522342 28414 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:07:56.570211 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.560304 28403 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.558548 28172 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:56.533074 27998 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:56.491303 28397 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:56.625555 21974 leader_election.cc:336] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.909s (SENT)
I20260709 13:07:56.601495 28191 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.534925 28389 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.557178 28390 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.615491 21851 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:56.559003 28001 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.608990 28256 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.698712 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.698385 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.652954 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.703938 22002 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:56.697961 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.661818 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.705632 22004 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:56.699786 22009 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:56.652402 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.607570 28341 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:56.708807 21853 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:56.709854 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 11.
I20260709 13:07:56.712898 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.714740 21931 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:56.714815 22000 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:56.612051 28480 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:56.741788 28278 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:56.688555 28360 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:56.614403 28054 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.688829 28365 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.750748 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.751629 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.691949 28224 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.747459 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.755353 22009 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:56.742749 28360 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.760361 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.760605 22002 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:07:56.747524 28407 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.749553 28479 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:56.750650 28414 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:56.789319 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:56.776664 27934 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.004s
I20260709 13:07:56.737668 28256 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.629755 28228 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.808538 22008 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:56.762069 28365 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.766110 28224 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.819090 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1424) took 1271 ms. Trace:
I20260709 13:07:56.822675 22588 rpcz_store.cc:276] 0709 13:07:55.547940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.578024 (+ 30084us) service_pool.cc:224] Handling call
0709 13:07:56.818890 (+1240866us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54004cca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.819026 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378752768}
I20260709 13:07:56.787840 28387 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:56.814425 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.810990 28344 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:56.833786 28453 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.006s
I20260709 13:07:56.802572 28479 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.868232 28453 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.868523 28370 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:56.869125 28370 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:56.869962 28479 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:56.870390 28453 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.870223 28370 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.833568 28413 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:56.874821 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:56.876488 28323 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:56.878113 21926 raft_consensus.cc:2393] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:56.878296 27970 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:56.878672 27970 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.879029 27970 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:56.880330 28116 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:56.885643 22006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2950) took 1611 ms (client timeout 1699 ms). Trace:
W20260709 13:07:56.886124 22006 rpcz_store.cc:269] 0709 13:07:55.274500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.279332 (+  4832us) service_pool.cc:224] Handling call
0709 13:07:56.885443 (+1606111us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54002710d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.885571 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474967296}
I20260709 13:07:56.887612 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.889395 27970 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.892270 27970 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:56.893591 28440 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.014s	user 0.002s	sys 0.000s
I20260709 13:07:56.901678 28033 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:56.906440 28374 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.912070 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.916702 21854 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:07:56.919744 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2500) took 1429 ms (client timeout 1631 ms). Trace:
W20260709 13:07:56.920178 21928 rpcz_store.cc:269] 0709 13:07:55.490147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.490287 (+   140us) service_pool.cc:224] Handling call
0709 13:07:56.919548 (+1429261us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54003075d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.919677 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894339840}
I20260709 13:07:56.834988 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.923246 22002 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:56.924166 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:56.924707 22002 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:56.835299 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.928357 28388 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.930815 28401 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.932183 28401 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.934376 27645 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.936482 27998 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:56.936961 27998 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.829213 28228 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:56.939683 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52006 (request call id 21) took 1352 ms. Trace:
I20260709 13:07:56.940150 21856 rpcz_store.cc:276] 0709 13:07:55.593159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.658202 (+ 65043us) service_pool.cc:224] Handling call
0709 13:07:56.939434 (+1281232us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b540028cb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707c9853 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691 00007f3b6ed1d60c 00007f3b6ed1c8c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.939568 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31128832}
I20260709 13:07:56.809289 28414 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.944574 28369 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.946302 28414 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.835991 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.841885 28343 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:56.842471 28365 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.843698 28006 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.844961 28389 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.846671 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:56.990191 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.847323 28469 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 13:07:56.850641 28256 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:56.795187 28407 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.856055 28054 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:56.860316 28441 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.981s	user 0.002s	sys 0.000s
I20260709 13:07:56.864427 28224 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.866008 28001 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:56.843832 28360 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.880854 28116 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:56.820150 28341 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:56.834801 28397 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:56.866683 28065 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:56.947789 28465 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:07:56.948045 22008 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:56.949693 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 997) took 1228 ms. Trace:
I20260709 13:07:57.062770 21855 rpcz_store.cc:276] 0709 13:07:55.721723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.722473 (+   750us) service_pool.cc:224] Handling call
0709 13:07:56.949278 (+1226805us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b540010c054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.949607 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133823360}
W20260709 13:07:56.950631 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3034) took 2770 ms (client timeout 1964 ms). Trace:
W20260709 13:07:57.064229 21929 rpcz_store.cc:269] 0709 13:07:54.179938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.204780 (+ 24842us) service_pool.cc:224] Handling call
0709 13:07:56.051301 (+1846521us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b5800063f98. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6aaa22 00007f3b765c3c09 00007f3b7571dee0 00007f3b757e70ab 00007f3b757e7131 00007f3b757950a0 00007f3b75794984 00007f3b6ed0f6d4 00007f3b6ed0f662
0709 13:07:56.950416 (+899115us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400228b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.950564 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445529728}
I20260709 13:07:56.950757 27998 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:56.953588 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:56.957423 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 998) took 1231 ms. Trace:
I20260709 13:07:57.065701 21852 rpcz_store.cc:276] 0709 13:07:55.726060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.737044 (+ 10984us) service_pool.cc:224] Handling call
0709 13:07:56.957162 (+1220118us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:56.957346 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653136256}
I20260709 13:07:56.958443 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:07:57.067035 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 108727us
W20260709 13:07:56.957289 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.967617 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55648.1us
I20260709 13:07:57.069082 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:56.966233 28404 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:56.971151 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.888909 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.075448 22006 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:56.979972 22009 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:07:57.076005 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:56.985931 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.077090 22000 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.918561 28454 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:56.997088 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 150484us
I20260709 13:07:56.997706 22002 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:07:57.019043 28376 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.001s
I20260709 13:07:56.928808 28191 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.015151 27861 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.097597 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:07:57.016510 28389 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:57.065831 22007 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:57.070968 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:56.970255 21825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.964s (SENT) [suppressed 37 similar messages]
I20260709 13:07:57.073946 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.078322 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:57.101361 21932 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:57.079538 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:57.102820 21932 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:57.098407 21934 raft_consensus.cc:606] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:57.103504 21934 raft_consensus.cc:614] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:57.083369 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.105373 22587 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.084110 21933 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:57.083542 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.095207 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:57.094916 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:57.097659 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.097666 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:57.107735 22592 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.108058 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.109469 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:57.110759 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.112287 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.112982 21930 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:57.114642 21931 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:57.115098 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44224.2us
I20260709 13:07:57.116843 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.080713 21929 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:57.137702 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 261674us
I20260709 13:07:57.142583 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.196795 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.146320 22584 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:57.141901 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:57.083684 28360 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:57.202081 22001 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:57.196811 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.094233 28349 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:57.220039 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.201982 21852 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:57.116242 28392 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:07:57.203399 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61614.4us
I20260709 13:07:57.258420 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.259248 21857 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:07:57.135949 28390 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.096529 28362 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.130s
I20260709 13:07:57.202215 28360 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.097550 28446 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:57.275683 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1303) took 1145 ms. Trace:
I20260709 13:07:57.213748 28278 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:57.228190 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 118803us
I20260709 13:07:57.230445 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.277508 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.074014 28397 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.128s
I20260709 13:07:57.235256 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:57.057639 28480 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.237594 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.137811 27861 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.254796 28006 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:07:57.293504 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.298286 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:57.300081 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.150027 22592 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:57.135759 28381 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:57.144833 28406 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:57.302845 28406 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:57.208897 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.221602 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:57.265703 21851 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:57.226039 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:57.304898 21853 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.277108 28360 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.272225 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.202515 28404 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:57.235812 28486 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:07:57.276162 22003 rpcz_store.cc:276] 0709 13:07:56.130730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.130886 (+   156us) service_pool.cc:224] Handling call
0709 13:07:56.135760 (+  4874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:56.161127 (+ 25367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:56.161178 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:56.172631 (+ 11453us) log.cc:844] Serialized 26 byte log entry
0709 13:07:56.471735 (+299104us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:56.471774 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:56.471848 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:56.947154 (+475306us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:07:57.271377 (+324223us) raft_consensus.cc:1685] finished
0709 13:07:57.271400 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.275613 (+  4213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19032,"raft.queue_time_us":18631,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9901,"spinlock_wait_cycles":10495360,"thread_start_us":250351,"threads_started":1,"wal-append.queue_time_us":500024}
I20260709 13:07:57.233385 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.324105 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.325388 21860 raft_consensus.cc:2410] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
W20260709 13:07:57.257296 28403 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.002s
I20260709 13:07:57.195521 28116 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.108124 28376 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.279563 22586 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.282246 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:57.339279 21852 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.340459 28480 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.342334 28407 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.344622 21859 raft_consensus.cc:2410] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:07:57.345443 28405 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.345952 28405 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.347105 22593 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:57.348022 28191 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:57.336933 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.201582 28368 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.308768 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.303301 28406 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:57.364061 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55378.3us
I20260709 13:07:57.320134 28387 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:57.312794 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.326593 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.373299 22007 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:57.373389 21856 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
W20260709 13:07:57.305855 28348 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.000s
I20260709 13:07:57.329989 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:57.314270 28172 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.004s
W20260709 13:07:57.335548 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63418us
I20260709 13:07:57.354452 21852 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:57.303922 28414 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:07:57.253549 22009 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:57.288645 28278 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:57.361282 21930 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:07:57.356981 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2912) took 2465 ms. Trace:
I20260709 13:07:57.394411 21858 rpcz_store.cc:276] 0709 13:07:54.891169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.891336 (+   167us) service_pool.cc:224] Handling call
0709 13:07:54.891789 (+   453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:54.949931 (+ 58142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.949966 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:56.969480 (+2019514us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b2c000e8e68. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:07:56.970091 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 13:07:57.196151 (+226060us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:57.196192 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.196751 (+   559us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:57.353605 (+156854us) raft_consensus.cc:1685] finished
0709 13:07:57.353639 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.356885 (+  3246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2951,"raft.queue_time_us":4212,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":8650,"spinlock_wait_cycles":10853120,"thread_start_us":225660,"threads_started":1,"wal-append.queue_time_us":376873}
I20260709 13:07:57.301178 22008 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:57.365772 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.310840 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.284080 28446 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.386238 21926 raft_consensus.cc:2368] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15.
I20260709 13:07:57.399789 22000 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: []
W20260709 13:07:57.358211 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 265290us
I20260709 13:07:57.405525 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.406364 21929 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:57.390645 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:57.415575 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.417505 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.418416 21928 raft_consensus.cc:2376] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14. Re-sending same reply.
I20260709 13:07:57.421931 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.422796 21932 raft_consensus.cc:2376] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19. Re-sending same reply.
I20260709 13:07:57.424090 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.426002 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:57.427875 21932 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:07:57.432590 21933 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:57.353709 27983 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.436729 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:57.441572 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.445637 21927 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [17.9-20.10]
I20260709 13:07:57.372768 28387 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.452363 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:57.453352 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.444245 21933 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:57.447031 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.447175 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:57.371960 28323 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:57.385387 28001 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:57.457233 28001 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:57.457775 28001 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.462452 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.444257 21929 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:07:57.463634 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.465740 21934 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.336551 28403 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.393673 28454 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.327031 28404 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.468923 21931 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:57.454418 27983 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.470484 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:57.453167 28224 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:57.439164 28191 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:57.466275 28387 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.451625 28413 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:07:57.394595 28348 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.467813 28258 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:07:57.489547 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 519273us
I20260709 13:07:57.510047 22007 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:57.449530 28006 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:07:57.517751 22007 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:57.519675 22007 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:57.525990 28397 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:57.529016 27998 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.483142 28375 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:57.533500 22005 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:57.565168 22586 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:57.565973 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:57.533630 28477 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.727s	user 0.003s	sys 0.000s
I20260709 13:07:57.568540 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.571254 22005 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:57.532612 28001 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:57.534983 28346 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.218s	user 0.006s	sys 0.008s
I20260709 13:07:57.535530 28348 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.536008 21855 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:57.409802 28172 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:57.536741 28491 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 13:07:57.537567 21824 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:57.538196 28404 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.540485 28278 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:57.512372 28224 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:07:57.547377 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:57.508991 28449 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.048s	user 0.013s	sys 0.001s
I20260709 13:07:57.511445 28390 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:57.575492 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:57.580109 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.017s	sys 0.013s
I20260709 13:07:57.583262 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.545787 28006 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:07:57.567965 28228 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.630s	user 0.014s	sys 0.004s
W20260709 13:07:57.584307 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.017s	sys 0.013s
I20260709 13:07:57.579223 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.564365 28414 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.607477 28403 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.548843 28397 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:57.608402 28397 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:57.597144 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59641.5us
I20260709 13:07:57.609468 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:57.611142 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:57.611913 27998 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:57.612305 27998 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:57.612660 27998 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.564893 28454 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.613350 28454 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.615403 28454 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.567940 28401 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.616014 28401 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.617264 22008 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:57.618098 28401 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.618877 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.619592 21858 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:07:57.595870 28449 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.579986 28493 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:57.622303 27998 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.604501 28404 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:57.583319 28390 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:57.605999 28495 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:07:57.606695 28376 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.618220 28001 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:57.621120 28006 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.577230 28492 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:57.604394 22009 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
W20260709 13:07:57.629053 28380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.000s	sys 0.022s
I20260709 13:07:57.601261 27861 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:57.583161 28346 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.607540 28414 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.668061 21852 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
W20260709 13:07:57.599277 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 584912us
I20260709 13:07:57.628325 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.628644 21934 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 12.
I20260709 13:07:57.669919 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:07:57.604005 28172 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:57.620716 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:57.637948 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.629961 28348 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.655228 22009 pending_rounds.cc:85] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 12
I20260709 13:07:57.623308 22584 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:57.645475 28446 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.630044 28407 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:07:57.716424 22589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:57.717760 22589 raft_consensus.cc:606] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:57.718271 22589 raft_consensus.cc:614] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:57.639349 28404 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:57.671820 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.676533 22009 pending_rounds.cc:107] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 16.13
W20260709 13:07:57.664938 28380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.001s	sys 0.022s
I20260709 13:07:57.676647 21858 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.672739 21856 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:57.714061 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44596.3us
I20260709 13:07:57.645763 28387 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.635964 28369 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:07:57.730676 22009 raft_consensus.cc:2889] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:57.682082 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.677286 28389 raft_consensus.cc:2764] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:07:57.647426 28390 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:57.644228 28454 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:57.757297 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:57.773408 21858 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:57.719162 28449 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.749173 28324 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:57.756521 28407 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:07:57.750527 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:57.782097 22001 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:57.780220 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.783792 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.784788 22590 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:57.788189 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.793036 22590 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:57.794167 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.778867 27861 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:57.805766 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:57.807193 22592 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:07:57.807032 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:57.791607 28414 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.799750 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.767817 28489 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.811839 22585 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:57.815266 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:57.830804 22588 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.838279 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:57.840834 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 425300us
I20260709 13:07:57.845146 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.814723 27861 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:57.816679 28324 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:57.859293 22591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:07:57.859997 22591 raft_consensus.cc:606] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:57.851230 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.851668 21858 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:07:57.864151 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.865757 21858 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:07:57.852419 21852 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:57.843003 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3074) took 1088 ms. Trace:
I20260709 13:07:57.874825 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.875578 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:57.857690 28341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.015s	sys 0.049s
I20260709 13:07:57.843739 28489 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.874712 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.861575 28346 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.867295 22004 rpcz_store.cc:276] 0709 13:07:56.754812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.756281 (+  1469us) service_pool.cc:224] Handling call
0709 13:07:56.756552 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:56.756751 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:57.076826 (+320075us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.478608 (+401782us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.842627 (+364019us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:57.842686 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:57.842743 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.842807 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.842922 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11630,"prepare.queue_time_us":201387,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":7345,"raft.queue_time_us":63499,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":3393,"thread_start_us":1026184,"threads_started":3,"wal-append.queue_time_us":528287}
I20260709 13:07:57.880882 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.895038 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.895951 22004 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:57.885783 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.895720 22556 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.886778 21858 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.879558 27998 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:57.904246 22005 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.914557 22004 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:57.901829 21852 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:57.918689 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:57.908138 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:57.909821 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 990) took 2328 ms (client timeout 1658 ms). Trace:
I20260709 13:07:57.910925 21857 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:07:57.921867 21857 pending_rounds.cc:85] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 11
I20260709 13:07:57.922164 21857 pending_rounds.cc:107] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:07:57.922303 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2527) took 1353 ms. Trace:
I20260709 13:07:57.911295 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:57.922418 21857 raft_consensus.cc:2889] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:57.923934 21934 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:07:57.906013 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:57.897910 28341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.015s	sys 0.050s
I20260709 13:07:57.918474 22590 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:57.915784 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.883046 28502 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:57.935727 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:57.937238 21933 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:57.936596 22590 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:57.940369 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:57.907090 28481 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.917295 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.869333 28404 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.929458 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.907985 28052 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:57.933434 22004 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:07:57.888253 27861 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:57.911160 28406 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.008s
W20260709 13:07:57.919760 21859 rpcz_store.cc:269] 0709 13:07:55.581674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.592570 (+ 10896us) service_pool.cc:224] Handling call
0709 13:07:57.909603 (+2317033us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54002ba554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:57.909750 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4025743104}
I20260709 13:07:57.943044 21929 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:57.902127 28369 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:57.902364 28503 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:57.919942 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.913389 28006 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:57.944917 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:57.946564 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:57.927994 28360 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.005s
I20260709 13:07:57.950558 21858 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:07:57.950986 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 457976us
W20260709 13:07:57.951375 21860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.005s	sys 0.034s
I20260709 13:07:57.959713 28481 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.992376 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.995944 22004 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:57.996718 28427 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:57.946723 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 494386us
I20260709 13:07:57.997201 28427 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:57.997707 28427 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.998152 21852 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:57.999233 21858 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:58.000906 22002 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.001261 28278 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:58.002293 28278 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:58.002781 28515 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:58.003886 28515 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:57.930968 27998 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:58.004715 27998 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:58.005237 21860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.006s	sys 0.034s
I20260709 13:07:58.005280 27998 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.006539 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:58.006603 28427 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.008000 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:57.981848 21933 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:07:58.009243 28360 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.961220 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:57.981158 28381 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:57.954030 28510 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:58.010802 28381 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.011684 28510 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:07:57.983912 28501 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:07:57.986621 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.025821 21929 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.978387 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:57.906988 28449 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:07:57.934815 28346 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:57.934882 28387 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:57.956092 22588 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:58.004309 28515 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:58.036752 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.003851 28278 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:58.037387 22588 raft_consensus.cc:2376] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19. Re-sending same reply.
I20260709 13:07:58.013929 21934 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:07:58.009783 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 63361.5us
I20260709 13:07:58.013242 28510 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.039104 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.016891 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.040585 22005 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:07:58.040796 21860 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:58.014581 27861 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:58.023301 28381 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.022940 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.046355 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:58.047407 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.047924 22590 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:57.920565 28505 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:58.030306 28369 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:57.980783 28406 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.052816 28369 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:58.053416 28406 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:58.054334 21854 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:58.055817 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:58.055850 27998 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.056399 21860 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:07:58.056552 21854 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:07:58.057019 28414 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:58.057250 21858 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:58.058274 27998 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.059393 21929 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:07:58.059439 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.060727 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.061934 22006 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:58.062139 21853 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:58.032872 28481 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.066277 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:07:58.028777 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.028956 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.029541 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.029735 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.015295 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:58.031050 22002 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:07:57.921413 28324 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.036180 28346 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:57.982518 28513 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:07:58.090494 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60820.1us
I20260709 13:07:58.021907 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.075037 28383 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:58.096179 28383 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.044138 28278 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:58.097462 28383 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.044557 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.045626 28397 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.099179 22588 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:58.099547 22587 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:58.067404 28033 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:58.100466 28033 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:58.101425 28397 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:58.050074 28407 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:58.102033 28033 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:58.084874 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69754.6us
I20260709 13:07:58.088083 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.103493 28033 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:58.097707 28278 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.104240 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:58.104565 28278 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.105283 28520 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:58.105748 28520 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:58.106106 28520 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.108374 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:58.075350 28360 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:58.109233 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.109553 28360 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:58.103833 21854 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:58.109635 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:58.110417 28360 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.090590 28389 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:58.111778 28360 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.042232 22005 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:58.118719 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:58.119199 21854 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:58.119705 22002 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:58.112764 28389 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.114053 28517 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 13:07:58.121421 28370 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.118741 28376 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:07:58.122663 28346 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.122884 28376 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:58.123358 28376 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.123670 28390 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.081719 28427 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.112782 28006 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:58.125777 28006 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:07:58.126166 28006 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.124270 28376 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.127301 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3093) took 1286 ms. Trace:
I20260709 13:07:58.127426 28389 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.127831 22009 rpcz_store.cc:276] 0709 13:07:56.840975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.456479 (+615504us) service_pool.cc:224] Handling call
0709 13:07:57.456813 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.763661 (+306848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.127064 (+363403us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001978d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:58.127242 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338521472}
I20260709 13:07:58.121189 28518 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:58.132210 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.134490 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:58.135207 22002 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:58.138411 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.143110 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:58.133131 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.163146 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.143822 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3089) took 1306 ms. Trace:
I20260709 13:07:58.171614 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.175885 21929 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:07:58.144642 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 115178us
I20260709 13:07:58.181489 28346 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:58.127975 28376 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.182073 28346 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.182328 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:07:58.179530 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46657.7us
W20260709 13:07:58.164584 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45398.5us
I20260709 13:07:58.183945 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.187485 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.188145 22589 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.190097 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.193518 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.200491 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.201776 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.203502 22002 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:07:58.194340 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.203078 21929 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:58.191398 22000 rpcz_store.cc:276] 0709 13:07:56.837274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.400999 (+563725us) service_pool.cc:224] Handling call
0709 13:07:57.410746 (+  9747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.411034 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:57.444610 (+ 33576us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.625722 (+181112us) spinlock_profiling.cc:243] Waited 9.71 ms on lock 0x7b18000581e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:58.131035 (+505313us) log.cc:844] Serialized 14 byte log entry
0709 13:07:58.143483 (+ 12448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:58.143537 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:58.143590 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.143655 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.143769 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22105,"prepare.queue_time_us":45489,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":10928,"raft.queue_time_us":22930,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5770,"spinlock_wait_cycles":74038784,"thread_start_us":494658,"threads_started":1}
I20260709 13:07:58.206328 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.223562 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.143160 28505 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:58.166955 28001 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:58.188169 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.231521 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.232138 21934 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.232383 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.163937 28506 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:58.212841 22585 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:58.203527 22590 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:58.226526 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.075722 28515 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.230427 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.238611 21932 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:58.239204 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2531) took 1163 ms. Trace:
I20260709 13:07:58.233641 21930 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:58.239792 21926 rpcz_store.cc:276] 0709 13:07:57.076130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.076245 (+   115us) service_pool.cc:224] Handling call
0709 13:07:58.238746 (+1162501us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54004290d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:58.239130 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641103232}
I20260709 13:07:58.240795 21927 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:58.131073 28323 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
W20260709 13:07:58.247705 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 220128us
I20260709 13:07:58.249476 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.225379 21929 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:58.206631 28349 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:58.203934 28369 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:58.144804 27861 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
W20260709 13:07:58.262832 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74966.1us
I20260709 13:07:58.221653 28407 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:58.272929 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.274106 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.268556 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.270756 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.283988 22588 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.192903 28397 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:07:58.231186 28505 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:58.231652 28001 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:58.252967 28481 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.313859 27970 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:58.314410 27970 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:58.256000 28405 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:58.317622 28524 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:07:58.318670 28523 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:07:58.266290 28172 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:58.211194 28052 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:07:58.311874 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.254766 28414 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:58.267812 28349 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:58.324347 28006 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.271389 28407 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.381685 28407 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.273571 28344 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 13:07:58.344089 22588 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:58.331537 28346 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.332250 22589 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:07:58.389371 22590 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:58.308061 21934 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:07:58.252274 28427 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:07:58.301613 28515 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.373349 21930 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:58.312230 28505 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:58.329979 28001 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.381183 28006 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:58.368780 28376 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:58.401667 21859 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:58.310702 28503 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 13:07:58.389132 28349 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:58.462826 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.401193 28390 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:58.465269 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:07:58.454695 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 291587us
I20260709 13:07:58.470041 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:58.468933 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 219782us
W20260709 13:07:58.471262 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 204493us
I20260709 13:07:58.471714 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:58.472421 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 266196us
I20260709 13:07:58.473865 21930 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
W20260709 13:07:58.452724 28348 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.002s
W20260709 13:07:58.478201 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.480195 21974 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.482853 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.484498 21974 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.486811 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.475378 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:58.476904 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 289475us
I20260709 13:07:58.477080 22585 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:58.490197 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.493021 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:58.484555 28390 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:58.482636 28348 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:58.500530 28369 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:07:58.506697 28407 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.458575 28376 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.508175 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3095) took 1665 ms. Trace:
I20260709 13:07:58.510581 22001 rpcz_store.cc:276] 0709 13:07:56.842948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.471741 (+628793us) service_pool.cc:224] Handling call
0709 13:07:57.472121 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.782767 (+310646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.507903 (+725136us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400197dd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:58.508108 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711996288}
I20260709 13:07:58.513278 28536 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:58.516253 28376 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.518996 28407 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.521872 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.524050 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.525209 28407 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:58.525156 28532 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:58.494652 28534 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:58.482803 28256 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:07:58.526711 28407 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.483672 28397 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:07:58.527981 28397 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:58.527818 28407 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.528419 28397 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:58.529430 28397 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.530700 28397 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:58.525554 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.533492 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.534843 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.500136 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.493247 28502 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:58.499750 28405 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:58.508296 28404 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:58.557700 28404 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.558701 28404 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.542948 28390 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:58.561837 28390 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.563046 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:58.563666 22005 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:07:58.564472 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:58.564790 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:07:58.564900 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.565251 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:58.566090 22590 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:58.566315 22007 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:07:58.543602 28534 raft_consensus.cc:2764] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:07:58.539886 28533 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:58.567166 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:58.567545 22007 raft_consensus.cc:1275] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:58.567615 28533 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.568171 28533 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.568583 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:58.568866 22006 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:58.539489 28228 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.569648 28228 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.571754 28228 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:58.544431 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 60236.7us
I20260709 13:07:58.543641 28369 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:58.574559 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.494974 28406 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:07:58.556592 28481 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:58.562065 28001 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.543179 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.566331 22005 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.574572 21858 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:07:58.580905 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 105591us
I20260709 13:07:58.582772 28534 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:58.583057 21854 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:58.536746 28518 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:58.585235 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.591379 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.593071 28065 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:58.593503 28065 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.594395 28065 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.595527 28065 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:58.596441 28521 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:58.597550 28521 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.598834 28375 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:58.601635 22593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1242) took 3838 ms (client timeout 1909 ms). Trace:
I20260709 13:07:58.598618 28521 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:58.602506 22593 rpcz_store.cc:269] 0709 13:07:54.763520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.766551 (+  3031us) service_pool.cc:224] Handling call
0709 13:07:58.601414 (+3834863us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54005019d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:58.601558 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5759471360}
I20260709 13:07:58.605739 22592 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:58.606752 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.607668 28389 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:58.608928 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.609772 28389 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.612061 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2600) took 1493 ms. Trace:
I20260709 13:07:58.614705 28065 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:58.615552 28065 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:58.617165 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.618106 28389 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:58.619160 21860 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:58.619472 28065 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:58.620416 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:07:58.621460 28065 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:58.621868 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.627645 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.628404 28375 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:58.628813 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.629057 28375 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:58.622135 28065 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.630048 28065 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.631029 28532 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.626181 28387 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:58.631800 28387 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:58.632725 28387 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.633935 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.634173 28387 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.641973 28389 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:58.643886 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:58.647395 28527 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 13:07:58.612630 21855 rpcz_store.cc:276] 0709 13:07:57.118548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.118922 (+   374us) service_pool.cc:224] Handling call
0709 13:07:57.119377 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.580683 (+461306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.611711 (+1031028us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:58.611977 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957720960}
I20260709 13:07:57.922852 21935 rpcz_store.cc:276] 0709 13:07:56.569066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.743772 (+174706us) service_pool.cc:224] Handling call
0709 13:07:56.762279 (+ 18507us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:56.781666 (+ 19387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:56.784471 (+  2805us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:56.803075 (+ 18604us) log.cc:844] Serialized 78 byte log entry
0709 13:07:57.096765 (+293690us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:57.546283 (+449518us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.917741 (+371458us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.921548 (+  3807us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.921775 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.921848 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:57.921887 (+    39us) raft_consensus.cc:1685] finished
0709 13:07:57.921901 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.922228 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2608,"prepare.queue_time_us":786219,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":1848,"raft.queue_time_us":59448,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":11924,"spinlock_wait_cycles":4977920,"thread_start_us":945912,"threads_started":3,"wal-append.queue_time_us":451286}
I20260709 13:07:58.653430 28389 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:58.654433 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.656574 28006 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:58.657652 28006 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:58.658259 28224 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:58.659061 28006 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:58.657716 28375 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:58.659474 28006 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.659883 28375 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.661401 28541 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:58.661208 28375 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.662075 28541 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:58.662685 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:58.665426 28522 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.264s	user 0.003s	sys 0.000s
I20260709 13:07:58.665318 28006 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.665984 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.663318 28541 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:58.667739 28529 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:07:58.668124 28006 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.671080 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.671602 28375 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.681694 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:07:58.672103 28530 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:07:58.683616 28065 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.686331 28065 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.674453 27970 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.677434 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:07:58.668722 27983 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:58.661006 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:07:58.680517 28528 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:07:58.686898 21930 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:58.690397 21933 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:58.672690 28494 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.123s	user 0.001s	sys 0.015s
I20260709 13:07:58.664402 21928 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:07:58.693284 28517 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:58.654376 28389 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:58.696730 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.587430 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:07:58.701593 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.754331 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.754998 21856 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.701696 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:58.701712 22006 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:58.702330 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1013) took 1440 ms. Trace:
I20260709 13:07:58.760102 21857 rpcz_store.cc:276] 0709 13:07:57.262043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.262210 (+   167us) service_pool.cc:224] Handling call
0709 13:07:57.910206 (+647996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.925217 (+ 15011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.702146 (+776929us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54002ba550. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:58.702276 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489435520}
I20260709 13:07:58.761495 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:58.703212 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:58.703826 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.707463 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:58.762413 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:07:58.765164 21928 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.722438 21927 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:07:58.730858 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:58.766469 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.719173 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:58.726655 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2922) took 1154 ms. Trace:
I20260709 13:07:58.773432 21859 rpcz_store.cc:276] 0709 13:07:57.587808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.961575 (+373767us) service_pool.cc:224] Handling call
0709 13:07:57.961904 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:58.448214 (+486310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.726376 (+278162us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:58.726583 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983478656}
I20260709 13:07:58.774870 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.770714 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:58.772605 22587 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:58.768045 21860 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:58.779539 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.780910 21854 raft_consensus.cc:2393] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:58.781355 21857 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:07:58.776809 21929 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:07:58.782028 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:58.782756 21860 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.783249 21935 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:07:58.710793 27970 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:58.729239 28403 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:58.787027 28403 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.787901 28403 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.730243 28517 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:58.789160 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.789853 28517 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.790817 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:07:58.731010 28389 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.792716 28172 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:58.793241 27983 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:07:58.793767 27983 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.747489 28116 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:58.749195 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.762465 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.585460 28502 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:58.588392 28405 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:58.759241 28520 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.001s
I20260709 13:07:58.802225 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:58.803431 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54339.8us
I20260709 13:07:58.813593 21929 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:58.813920 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.815778 21823 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.817047 28368 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:58.817766 28368 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.818866 28368 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.824234 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:58.825323 28343 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:58.827440 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.807211 21930 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:07:58.798609 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:58.811595 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:58.806254 21928 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:58.832376 28343 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:58.833096 28343 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.834231 28343 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.837400 21860 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:07:58.720517 28383 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:58.843514 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 256270us
I20260709 13:07:58.712698 28006 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.870616 21856 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:07:58.867784 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:58.841046 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:07:58.699059 28532 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:58.848779 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:07:58.842473 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74007us
W20260709 13:07:58.843510 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.853080 28483 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:58.852965 27934 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:58.839927 28263 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:58.750136 28518 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.731510 28489 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:58.793289 28172 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:58.866585 28547 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:58.839934 28375 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:58.867014 28370 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:58.788870 28403 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.840996 28405 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.770910 28539 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:58.879791 28502 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.867908 28492 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:07:58.850307 28427 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:58.748806 28369 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 13:07:58.907836 22007 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:07:58.932539 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1164) took 1021 ms. Trace:
W20260709 13:07:58.949592 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.845086 28065 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:07:58.999742 21931 rpcz_store.cc:276] 0709 13:07:57.910883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.913043 (+  2160us) service_pool.cc:224] Handling call
0709 13:07:57.922809 (+  9766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.923196 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:57.945154 (+ 21958us) log.cc:844] Serialized 14 byte log entry
0709 13:07:58.572927 (+627773us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b180041c380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:07:58.932163 (+359236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:58.932222 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:58.932271 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.932337 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.932479 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7845,"prepare.queue_time_us":21839,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":14103,"spinlock_wait_cycles":18931584,"thread_start_us":356571,"threads_started":1}
I20260709 13:07:58.899802 28348 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:58.849318 28520 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:58.921273 28518 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.980759 28541 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:58.988479 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.888446 28006 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:58.841002 28481 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
W20260709 13:07:59.009907 28515 consensus_meta.cc:287] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.000s
I20260709 13:07:58.910215 28212 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:07:58.908776 28536 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:07:58.959914 28483 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:59.022810 22005 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:07:59.026141 28348 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:07:58.920857 28510 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:07:59.049782 21927 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: [17.16-17.16]
W20260709 13:07:58.988920 28278 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.000s	sys 0.020s
W20260709 13:07:59.011662 28346 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.001s	sys 0.010s
I20260709 13:07:58.993552 28427 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.970319 28540 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:07:59.008783 28539 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:59.058056 21934 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.065567 22587 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:59.070749 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:35104 (request call id 30) took 1208 ms. Trace:
I20260709 13:07:59.082860 22591 rpcz_store.cc:276] 0709 13:07:57.849127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.849527 (+   400us) service_pool.cc:224] Handling call
0709 13:07:58.562784 (+713257us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b54006a0ca0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b7074c520 00007f3b707c9652 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691 00007f3b6ed1d60c 00007f3b6ed1c8c2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.056904 (+494120us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4c0039f480. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6ea9ba20 00007f3b6eaaca29 00007f3b6ea72644 00007f3b6ea99f2d 00007f3b6ea99b93 00007f3b707c9c92 00007f3b707c984a 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691
0709 13:07:59.057088 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53837056}
W20260709 13:07:59.055044 28278 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.000s	sys 0.021s
W20260709 13:07:59.066027 28346 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.001s	sys 0.010s
I20260709 13:07:59.109697 27983 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:59.110854 28548 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:07:59.082084 28407 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.009s
I20260709 13:07:59.073757 28427 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.079070 28536 consensus_queue.cc:1243] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:59.078425 28052 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:59.081698 28493 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:07:59.129140 28407 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.010s	sys 0.009s
I20260709 13:07:59.124433 28001 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:59.138110 28052 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:07:59.142028 28554 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:59.168721 28554 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:59.168848 28052 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.170058 28052 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.170161 28554 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.120800 28483 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:59.164904 28370 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:59.167625 28551 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:07:59.058769 28172 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.127106 28518 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:59.194562 28549 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:59.152030 28493 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:59.201781 28505 consensus_meta.cc:287] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.740s	user 0.005s	sys 0.006s
W20260709 13:07:59.205405 28349 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.000s
I20260709 13:07:59.184613 28370 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:59.206736 28533 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.638s	user 0.006s	sys 0.007s
I20260709 13:07:59.185730 28348 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:59.269806 28557 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:07:59.248818 28533 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.298409 28533 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.290506 28518 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.283535 28520 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.240047 28505 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.301918 28505 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.342381 21856 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:59.357775 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.001s	sys 0.013s
W20260709 13:07:59.496977 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.001s	sys 0.013s
W20260709 13:07:59.497685 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 668051us
W20260709 13:07:59.359633 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:59.374176 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.372848 21934 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:07:59.378656 22000 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:07:59.500164 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:07:59.380085 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 2978) took 1248 ms. Trace:
I20260709 13:07:59.503523 22009 rpcz_store.cc:276] 0709 13:07:58.131174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.138446 (+  7272us) service_pool.cc:224] Handling call
0709 13:07:58.138960 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:58.435155 (+296195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:58.435206 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:58.468824 (+ 33618us) log.cc:844] Serialized 26 byte log entry
0709 13:07:58.579432 (+110608us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:58.579469 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.579544 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:59.379349 (+799805us) raft_consensus.cc:1685] finished
0709 13:07:59.379392 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.379753 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27997,"raft.queue_time_us":54036,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":17007,"thread_start_us":349150,"threads_started":2,"wal-append.queue_time_us":481151}
I20260709 13:07:59.388927 22585 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:59.506232 28520 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:59.388374 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:59.508101 28520 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.393121 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:59.191535 28001 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.227s
I20260709 13:07:59.509155 28520 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.207676 28006 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.511683 28006 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:59.435786 28116 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.625s	user 0.007s	sys 0.006s
I20260709 13:07:59.513211 22003 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:07:59.229718 28360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.020s	sys 0.064s
W20260709 13:07:59.514292 28360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.020s	sys 0.065s
I20260709 13:07:59.515636 28360 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:59.516152 28360 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:59.516582 28520 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.264498 28493 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:59.386483 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:59.518857 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.240003 28349 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.520774 21854 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:59.521816 28349 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.282951 28547 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:59.523665 28360 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.524297 28547 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.525972 28360 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.504988 21900 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:59.529899 28541 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:59.530550 28541 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:07:59.532061 28541 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.552541 27861 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.541859 28405 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.596038 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2933) took 1932 ms. Trace:
W20260709 13:07:59.596776 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.000s	sys 0.004s
W20260709 13:07:59.597292 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.001s	sys 0.004s
W20260709 13:07:59.598241 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 749507us
I20260709 13:07:59.601644 27861 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:59.457285 28370 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.596725 21852 rpcz_store.cc:276] 0709 13:07:57.663762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.000279 (+336517us) service_pool.cc:224] Handling call
0709 13:07:58.000605 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:58.096836 (+ 96231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:58.096868 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:58.097406 (+   538us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:58.097463 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.595144 (+1497681us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b540010cf54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.595269 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.595609 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220816512}
I20260709 13:07:59.619431 28533 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.620420 28533 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:07:59.624330 28533 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:07:59.625113 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50076.8us
W20260709 13:07:59.625838 21926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.007s	sys 0.023s
W20260709 13:07:59.633646 28389 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.841s	user 0.014s	sys 0.009s
I20260709 13:07:59.633858 28370 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.634639 28389 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:59.640023 28389 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:07:59.645067 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 140267us
I20260709 13:07:59.646833 28389 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:59.647209 28584 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:07:59.647980 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.647821 28389 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.648222 28584 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:59.696580 28375 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.005s	sys 0.038s
W20260709 13:07:59.714564 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:59.711313 21899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.003s	sys 0.010s
W20260709 13:07:59.721738 21825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.008s	sys 0.007s
W20260709 13:07:59.722396 21825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.008s	sys 0.008s
I20260709 13:07:59.730757 28212 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.731597 28212 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:59.733474 22003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2970) took 2646 ms (client timeout 3159 ms). Trace:
W20260709 13:07:59.733959 22003 rpcz_store.cc:269] 0709 13:07:57.087374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.327292 (+239918us) service_pool.cc:224] Handling call
0709 13:07:59.733190 (+2405898us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b540042b8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:59.733362 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4893149568}
I20260709 13:07:59.735332 28584 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.718777 28518 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
W20260709 13:07:59.738519 21899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.003s	sys 0.010s
W20260709 13:07:59.743764 28375 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.005s	sys 0.039s
W20260709 13:07:59.749441 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.009s	sys 0.006s
I20260709 13:07:59.765265 28387 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.766160 22584 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:59.767015 28446 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:59.767529 28446 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:07:59.767599 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:07:59.777204 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.776338 28375 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:07:59.782778 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.010s	sys 0.006s
W20260709 13:07:59.758682 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 872687us
W20260709 13:07:59.792689 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 925584us
I20260709 13:07:59.779475 28375 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.802691 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:59.802913 28116 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:59.807555 28116 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.807894 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:59.808284 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1175) took 1035 ms. Trace:
I20260709 13:07:59.810564 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 2986) took 1108 ms. Trace:
I20260709 13:07:59.810933 28567 raft_consensus.cc:2764] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:59.811172 22007 rpcz_store.cc:276] 0709 13:07:58.702152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.702719 (+   567us) service_pool.cc:224] Handling call
0709 13:07:59.810105 (+1107386us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001973d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:59.810334 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332465024}
I20260709 13:07:59.811619 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:07:59.812183 28567 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:59.808722 21927 rpcz_store.cc:276] 0709 13:07:58.772479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.774232 (+  1753us) service_pool.cc:224] Handling call
0709 13:07:58.774617 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.050296 (+275679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.807732 (+757436us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54003070d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.808119 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632199040}
I20260709 13:07:59.819667 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:07:59.824232 28454 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:59.820233 27998 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:07:59.824818 28454 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:07:59.831982 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.766232 28387 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:59.834815 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:59.839350 28341 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.842060 28341 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.847074 28116 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.850200 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:07:59.853935 28427 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.009s
I20260709 13:07:59.854559 28427 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:59.856602 21860 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:07:59.858594 28427 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.858800 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:07:59.859436 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.861507 21852 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:59.865746 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2624) took 1280 ms. Trace:
I20260709 13:07:59.866700 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2623) took 1281 ms. Trace:
I20260709 13:07:59.867405 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2547) took 1179 ms. Trace:
I20260709 13:07:59.867966 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:59.868820 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3109) took 1306 ms. Trace:
I20260709 13:07:59.870137 22000 rpcz_store.cc:276] 0709 13:07:58.562134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.565692 (+  3558us) service_pool.cc:224] Handling call
0709 13:07:58.566001 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.503176 (+937175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.868491 (+365315us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400168d54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.868765 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867093248}
I20260709 13:07:59.867794 21934 rpcz_store.cc:276] 0709 13:07:58.688352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.703598 (+ 15246us) service_pool.cc:224] Handling call
0709 13:07:59.867164 (+1163566us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b540017f054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:07:59.867315 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674820864}
I20260709 13:07:59.868808 21859 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:59.875602 22590 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:59.867125 21860 rpcz_store.cc:276] 0709 13:07:58.585266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.858734 (+273468us) service_pool.cc:224] Handling call
0709 13:07:58.885707 (+ 26973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.865521 (+979814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.866345 (+   824us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b5400059bd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.866621 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2231492480}
I20260709 13:07:59.816769 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2549) took 1099 ms. Trace:
I20260709 13:07:59.822647 28348 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:59.880589 28553 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.347s	user 0.001s	sys 0.003s
I20260709 13:07:59.880887 28454 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:07:59.843901 28397 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.313s	user 0.015s	sys 0.009s
I20260709 13:07:59.824698 28554 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:07:59.866325 21856 rpcz_store.cc:276] 0709 13:07:58.585667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.871937 (+286270us) service_pool.cc:224] Handling call
0709 13:07:58.872264 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.496155 (+623891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.865314 (+369159us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b540028cb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.865619 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079328128}
I20260709 13:07:59.904939 21855 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:59.906347 28052 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.716s	user 0.015s	sys 0.012s
I20260709 13:07:59.906903 21855 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:07:59.907434 22589 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:07:59.908429 21855 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:59.909961 28454 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:59.910974 28349 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.911535 28349 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.886819 28348 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:59.913045 28348 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:59.914140 28404 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.914846 28404 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.917379 27998 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:07:59.917732 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1264) took 1155 ms. Trace:
I20260709 13:07:59.876647 28446 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.919538 28446 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:59.918210 22592 rpcz_store.cc:276] 0709 13:07:58.761687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.763695 (+  2008us) service_pool.cc:224] Handling call
0709 13:07:58.784042 (+ 20347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:58.793853 (+  9811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:58.793887 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:58.794485 (+   598us) log.cc:844] Serialized 52 byte log entry
0709 13:07:59.055104 (+260619us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:59.711186 (+656082us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.826899 (+115713us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.887108 (+ 60209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.917070 (+ 29962us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:59.917223 (+   153us) raft_consensus.cc:1685] finished
0709 13:07:59.917238 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.917392 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19254,"prepare.queue_time_us":263751,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":77,"raft.queue_time_us":23298,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":47287,"thread_start_us":757572,"threads_started":2,"wal-append.queue_time_us":364545}
I20260709 13:07:59.919852 28404 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.918344 28349 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:59.924254 28403 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.951s	user 0.018s	sys 0.004s
I20260709 13:07:59.927883 28212 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.927981 28349 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:59.918201 27998 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.931178 28389 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.932188 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3107) took 1370 ms. Trace:
I20260709 13:07:59.932922 28539 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.933861 28539 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.936586 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:07:59.937659 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:59.937673 22009 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:07:59.938767 21856 raft_consensus.cc:2410] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:07:59.900069 28554 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.940007 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:59.941867 22005 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:07:59.945878 28341 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.946676 28387 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:59.948014 22005 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:59.948205 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:07:59.948640 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.949291 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:07:59.949697 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47944.8us
I20260709 13:07:59.950568 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:07:59.912544 27861 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:07:59.951152 27998 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.953543 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2622) took 1371 ms. Trace:
I20260709 13:07:59.953938 21854 rpcz_store.cc:276] 0709 13:07:58.582012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.807459 (+225447us) service_pool.cc:224] Handling call
0709 13:07:58.859062 (+ 51603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.521307 (+662245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.953273 (+431966us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54000fcb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.953488 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520682752}
I20260709 13:07:59.954558 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:07:59.954833 22592 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:59.955166 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:59.956331 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:59.958506 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:59.959234 22584 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:07:59.932641 22001 rpcz_store.cc:276] 0709 13:07:58.561570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.564098 (+  2528us) service_pool.cc:224] Handling call
0709 13:07:58.564405 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:58.572598 (+  8193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:58.572630 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:58.574841 (+  2211us) log.cc:844] Serialized 26 byte log entry
0709 13:07:59.008772 (+433931us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:59.008814 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.931521 (+922707us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b54000ca8d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:07:59.931591 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:59.931668 (+    77us) raft_consensus.cc:1685] finished
0709 13:07:59.931681 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.931958 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2529,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":7949,"spinlock_wait_cycles":17982336,"thread_start_us":424355,"threads_started":1,"wal-append.queue_time_us":620039}
W20260709 13:07:59.940244 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.885475 21926 rpcz_store.cc:276] 0709 13:07:58.704605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.704720 (+   115us) service_pool.cc:224] Handling call
0709 13:07:58.705316 (+   596us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:58.746708 (+ 41392us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4800241ed8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:07:59.011088 (+264380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.011357 (+   269us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:59.039847 (+ 28490us) log.cc:844] Serialized 78 byte log entry
0709 13:07:59.626961 (+587114us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:59.701596 (+ 74635us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.719769 (+ 18173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.784165 (+ 64396us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:59.784223 (+    58us) raft_consensus.cc:1685] finished
0709 13:07:59.800406 (+ 16183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.804184 (+  3778us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":125354,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":7642,"raft.queue_time_us":12529,"raft.run_cpu_time_us":6041,"raft.run_wall_time_us":52204,"spinlock_wait_cycles":34961024,"thread_start_us":904147,"threads_started":3,"wal-append.queue_time_us":575125}
I20260709 13:07:59.965121 28539 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:07:59.969511 28586 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.390s	user 0.002s	sys 0.000s
I20260709 13:07:59.971649 27861 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.972707 27861 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:07:59.973562 28348 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:07:59.976815 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:07:59.977139 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.981031 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:07:59.981356 22593 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
W20260709 13:07:59.981608 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.979426 28348 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:07:59.979926 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:07:59.982935 28502 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.983784 21860 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.952142 21859 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:07:59.979809 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:07:59.983999 28502 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:07:59.986145 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 174771us
I20260709 13:07:59.987097 28567 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.987226 27861 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:07:59.994029 28584 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:00.007901 21855 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:00.024456 22589 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:08:00.012921 28341 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.032502 28341 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:00.034667 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 169619us
I20260709 13:08:00.035688 28116 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:00.036408 28116 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.036888 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.038453 28065 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.038944 28116 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:00.039271 28065 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.017113 28360 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:00.041953 21926 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:08:00.041714 28360 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.045425 28569 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.046229 28554 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.046144 28569 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.048357 28502 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.048530 28569 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.043989 28360 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.019750 27861 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.050424 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:00.062932 28065 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:00.068884 28577 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.071s	user 0.000s	sys 0.002s
W20260709 13:08:00.070551 27970 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.233s	user 0.013s	sys 0.018s
I20260709 13:08:00.006472 28567 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.073450 28341 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.073800 27861 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.074257 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.074962 22590 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
W20260709 13:08:00.075219 28590 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.149s	user 0.000s	sys 0.003s
W20260709 13:08:00.075978 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 60303us
I20260709 13:08:00.080933 21934 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:00.082535 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.088107 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.099449 21860 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:00.100369 21927 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:08:00.090771 28341 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:00.119431 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:00.120476 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.124904 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.130635 22007 raft_consensus.cc:2410] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:08:00.136240 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.089838 28370 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:08:00.140475 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.146064 21934 raft_consensus.cc:2410] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:00.111974 28341 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.131768 27861 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.117830 28533 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:08:00.130244 28397 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:08:00.160324 21927 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:00.149945 28539 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.172190 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 148718us
I20260709 13:08:00.173727 28052 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:00.175186 28323 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:00.175678 28323 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:00.177318 28502 raft_consensus.cc:993] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: : Instructing follower 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:08:00.178246 28540 raft_consensus.cc:1081] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Signalling peer 7d07aa0dcdd1484ab87925c9a4604b81 to start an election
I20260709 13:08:00.179484 28397 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:00.180310 28397 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:00.181514 28278 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.182263 28278 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:00.189911 28545 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.281s	user 0.003s	sys 0.000s
I20260709 13:08:00.191764 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:00.197266 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1323) took 1333 ms. Trace:
I20260709 13:08:00.199774 28381 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:00.200876 28381 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.178309 28065 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.167860 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:00.175768 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50494.6us
I20260709 13:08:00.214659 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:00.217279 28493 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:00.217156 28212 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:08:00.223304 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50855.4us
I20260709 13:08:00.171677 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:00.226434 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59543.5us
I20260709 13:08:00.227922 27861 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:00.171341 28492 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.198460 22002 rpcz_store.cc:276] 0709 13:07:58.767016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.767164 (+   148us) service_pool.cc:224] Handling call
0709 13:07:58.767508 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:58.810596 (+ 43088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:58.810625 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:58.844060 (+ 33435us) log.cc:844] Serialized 26 byte log entry
0709 13:07:59.162626 (+318566us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b180039f520. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:07:59.756130 (+593504us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:59.794244 (+ 38114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.797215 (+  2971us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.076687 (+279472us) raft_consensus.cc:1685] finished
0709 13:08:00.076715 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.100366 (+ 23651us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42687,"raft.queue_time_us":42898,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11746,"spinlock_wait_cycles":28168064,"thread_start_us":414749,"threads_started":1,"wal-append.queue_time_us":1101294}
I20260709 13:08:00.245059 21931 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
W20260709 13:08:00.248553 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78192.3us
I20260709 13:08:00.239746 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:00.230366 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.204124 28381 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:00.254762 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.265214 27861 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.269475 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.270094 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.271083 22007 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:08:00.271831 28323 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.255821 28492 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.273902 28323 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:00.225404 28052 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:08:00.275434 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:00.276692 21857 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:00.277413 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.277850 28533 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:00.278375 22593 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:08:00.285449 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 44041.4us
I20260709 13:08:00.261756 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:00.299075 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:00.302330 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.239207 28065 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.286381 28541 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:08:00.308187 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.305421 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:00.289988 28502 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:00.199657 28397 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.230867 28539 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.316642 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:00.317988 21860 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:00.318980 28001 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.320129 28341 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:00.319777 28001 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:00.321906 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 82390.7us
W20260709 13:08:00.325222 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.308682 27861 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:00.310616 28360 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.331501 22588 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:00.331776 21855 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:00.332170 28532 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:00.332774 28532 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.334486 28065 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.335435 28532 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.335898 28360 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.338812 28360 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:00.341800 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.351570 28492 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:00.353638 28595 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.187s	user 0.000s	sys 0.002s
I20260709 13:08:00.353848 28405 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:00.354698 28405 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:00.359514 28493 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:08:00.361408 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 124082us
I20260709 13:08:00.360443 28493 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:00.373121 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.374523 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:00.386754 22005 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.375334 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:00.315891 28547 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:00.376379 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.377714 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.394914 22009 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:08:00.379773 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.329414 28341 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:00.381699 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.317411 22002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:32904
I20260709 13:08:00.383379 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.384301 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.399111 22002 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:00.385165 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.399680 22002 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.326900 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:00.387202 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70017.9us
I20260709 13:08:00.397882 28554 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:00.402984 22003 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:00.403249 28554 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.404337 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.406329 22008 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:00.406703 28584 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.406867 22588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1262) took 1874 ms (client timeout 1747 ms). Trace:
I20260709 13:08:00.402402 22001 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.407186 22006 raft_consensus.cc:2410] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:08:00.398355 28547 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:00.403595 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.317762 28502 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:00.340601 28521 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.409413 28554 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.402573 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.399338 22586 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:00.402374 22007 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:08:00.433435 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.434971 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:00.447750 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:00.431823 22588 rpcz_store.cc:269] 0709 13:07:58.532231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.540036 (+  7805us) service_pool.cc:224] Handling call
0709 13:08:00.406517 (+1866481us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54005325d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:00.406772 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3891947776}
I20260709 13:08:00.393457 28404 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:00.440589 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 136093us
W20260709 13:08:00.440833 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 114968us
I20260709 13:08:00.451167 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.398329 28341 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.447554 22590 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:00.456929 28502 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.458388 28539 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.439584 28547 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:00.481563 22584 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:00.485562 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.466238 28341 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:00.493321 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 2947) took 1601 ms. Trace:
I20260709 13:08:00.441265 28584 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.463686 27861 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:00.356693 28405 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.497226 22000 raft_consensus.cc:2410] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:08:00.491242 22003 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:00.475786 28554 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:00.515650 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 104809us
I20260709 13:08:00.512287 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.521363 22589 raft_consensus.cc:2410] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:08:00.491663 28541 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:00.527380 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:00.530429 28001 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.496555 27998 consensus_queue.cc:1048] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:08:00.531550 28375 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.508173 28065 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.542081 22002 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.495640 21858 rpcz_store.cc:276] 0709 13:07:58.696433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.628279 (+931846us) service_pool.cc:224] Handling call
0709 13:07:59.714120 (+ 85841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.761119 (+ 46999us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:07:59.900460 (+139341us) log.cc:844] Serialized 14 byte log entry
0709 13:08:00.296639 (+396179us) log.cc:844] Serialized 14 byte log entry
0709 13:08:00.297507 (+   868us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:00.297553 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:00.297599 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.297695 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.297813 (+   118us) inbound_call.cc:177] Queueing success response
0709 13:08:00.409251 (+111438us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b1800377ec0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
Metrics: {"mutex_wait_us":2834,"prepare.queue_time_us":368708,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":9933,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":4849,"spinlock_wait_cycles":11097344,"thread_start_us":418152,"threads_started":2,"wal-append.queue_time_us":446703}
W20260709 13:08:00.559046 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 123788us
I20260709 13:08:00.525349 28554 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:00.594336 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.548981 28001 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.605643 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.491942 28493 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.612685 21928 raft_consensus.cc:2410] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:08:00.557821 28404 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:00.606730 22002 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:00.615839 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.615810 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:00.620172 21932 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: [12.12-14.13]
W20260709 13:08:00.627328 28603 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:08:00.628561 28604 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:08:00.633224 28608 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.633881 28608 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.635303 28554 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.636333 28608 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.637537 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.638648 22589 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
W20260709 13:08:00.633143 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 130093us
I20260709 13:08:00.640782 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:00.641296 28212 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:00.642828 28454 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:00.643352 28454 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:00.643679 28349 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:00.633119 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:00.662341 22001 raft_consensus.cc:2410] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
I20260709 13:08:00.660715 22005 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:00.670583 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:00.671563 22584 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:00.671931 22584 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:00.673599 22003 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:00.561137 28065 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:00.676128 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 57900.4us
I20260709 13:08:00.619859 28360 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:00.688071 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.687187 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:00.644408 28454 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.618040 28001 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.700335 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:00.701253 28001 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.673485 28533 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:00.702184 28518 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:00.702433 28533 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:00.702773 28518 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:00.702944 28533 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:00.703881 28518 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.617744 28375 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.706864 28278 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.706926 28375 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.707849 21934 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [17.9-20.10]
I20260709 13:08:00.702425 21932 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:00.698971 27861 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:00.712646 28323 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:00.715544 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.719424 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:00.697402 28541 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.721191 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.723070 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.712621 28614 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.002s
W20260709 13:08:00.724347 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.725180 28116 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.725757 28116 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.726166 28584 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.728084 28116 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:00.728314 21935 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:08:00.735838 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.744235 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:00.744624 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:00.745399 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:00.745980 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.746703 22585 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:08:00.750903 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:00.756740 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 69689.5us
I20260709 13:08:00.765826 22589 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:00.766451 22003 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:00.754506 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:00.755631 22584 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.763615 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.770619 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:00.771243 22005 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:08:00.694425 28360 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.776042 22008 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:00.770366 22009 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:00.779016 22008 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:00.776182 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67465.3us
I20260709 13:08:00.774610 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:00.738481 27861 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:00.773552 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:00.777298 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:00.788357 28579 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.649s	user 0.000s	sys 0.006s
W20260709 13:08:00.790560 28610 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:08:00.790908 28370 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.791936 28609 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:00.768826 28493 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:00.797206 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 77865.4us
I20260709 13:08:00.791643 28370 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.797866 21935 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:00.769200 28278 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:00.806828 21929 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:00.810329 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:00.782176 28618 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.821730 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.771946 28323 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:08:00.825511 22001 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:00.825834 28278 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:00.782814 27861 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:00.833747 22584 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.834913 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.804744 28387 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:00.841208 28618 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:00.842720 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 44685us
I20260709 13:08:00.804411 22008 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.850323 21859 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:08:00.812276 28614 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:00.870853 28349 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.803977 28001 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:08:00.870621 21859 pending_rounds.cc:85] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 12
I20260709 13:08:00.871512 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.872515 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:00.875258 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:00.838197 28387 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:00.868026 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60016.5us
I20260709 13:08:00.875905 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:00.867681 22004 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:00.804989 28547 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.892025 22008 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:00.894920 28589 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.827s	user 0.003s	sys 0.007s
I20260709 13:08:00.891510 21859 pending_rounds.cc:107] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:08:00.902511 21859 raft_consensus.cc:2889] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:00.884831 28387 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:00.873304 28541 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.889134 28375 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:00.910908 21929 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:00.910352 27861 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.911471 28278 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.913106 28541 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.912333 28533 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.915220 21928 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:00.916695 28616 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:00.935141 22003 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:00.935457 28616 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.935920 28616 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.918243 28547 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.919056 28483 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:00.938694 28483 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:00.939605 28483 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.871470 28349 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.940848 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:00.926028 28612 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:00.922358 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.934226 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:00.915836 28341 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.928510 28584 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.929157 28623 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:00.929322 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.908062 28454 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.929975 28065 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.933569 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1268) took 1664 ms. Trace:
I20260709 13:08:00.934338 21930 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:00.934279 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:00.944876 22584 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:00.925089 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:00.952386 21927 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:00.952507 21929 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:00.942713 22007 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:00.962512 28370 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.963685 28608 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.942728 28569 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.965687 28349 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:00.965480 28569 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.966581 28541 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:00.966735 28584 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:00.978453 28622 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:08:00.979801 21933 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:00.980808 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.981869 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.983336 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:00.984002 21933 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:00.988922 28454 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:00.967778 21931 raft_consensus.cc:2393] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:00.964387 28608 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:00.968873 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:00.980697 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:00.954610 22008 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:08:01.024226 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 99379us
I20260709 13:08:00.958043 22587 rpcz_store.cc:276] 0709 13:07:59.178723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.178861 (+   138us) service_pool.cc:224] Handling call
0709 13:07:59.184946 (+  6085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.185424 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.185472 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:59.186046 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:07:59.750605 (+564559us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:59.785145 (+ 34540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.797951 (+ 12806us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.681093 (+883142us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:00.797366 (+116273us) raft_consensus.cc:1685] finished
0709 13:08:00.797391 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.843040 (+ 45649us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7213,"raft.queue_time_us":5446,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":218,"spinlock_wait_cycles":4628224,"thread_start_us":420316,"threads_started":1,"wal-append.queue_time_us":826694}
I20260709 13:08:01.036684 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.047494 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:00.904909 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:01.022979 28635 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:01.056851 28348 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:01.057603 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:01.061115 28454 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.062042 22005 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:01.067509 21935 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:08:01.028383 28349 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:01.105813 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 208371us
I20260709 13:08:01.118196 28454 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:01.127146 28381 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:01.127801 28381 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:01.133239 27970 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:01.134095 27970 consensus_queue.cc:1243] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:08:01.068449 28348 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.038813 28370 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:08:01.189518 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:01.144438 28381 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:01.190586 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.200282 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.243599 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 186051us
I20260709 13:08:01.157590 28348 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:01.144289 27970 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:01.246659 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:01.258113 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.258899 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.259075 28341 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:01.183701 28634 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:08:01.261831 21933 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 12.
I20260709 13:08:01.177068 28635 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:01.262071 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:01.270676 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.242771 28483 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:01.190193 28569 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:01.272387 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 83041.5us
I20260709 13:08:01.190850 28454 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:01.276686 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:01.281494 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:01.282495 28521 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:01.274797 27970 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.283027 28521 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:01.283537 28521 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:01.285130 28569 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:01.285643 22009 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:01.278714 28616 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.286448 27970 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.265277 28387 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.288375 28616 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.288543 22003 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:01.289149 21926 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:01.292261 28521 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.271198 28628 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:01.270386 27861 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:01.295615 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:01.278980 28349 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:01.321444 28349 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:01.323077 28349 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.325750 28052 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.257234 28584 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.294323 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:01.303097 28638 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.305354 28628 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.296322 28635 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.305451 28624 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:01.294246 28387 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.330500 28628 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:01.332267 28624 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.327894 28583 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:01.329558 21899 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:01.326334 28052 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.353506 28518 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:01.358408 28518 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.359926 28052 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:01.363119 28624 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.330165 28638 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.367079 28638 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.367489 28360 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.369859 28397 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:01.373090 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 77580.6us
I20260709 13:08:01.378757 27998 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:01.379395 27998 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:01.373148 28518 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.437285 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:01.439409 28584 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:01.440795 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.380474 27998 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.439929 28584 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.448091 28584 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:01.448271 27998 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:01.457312 28360 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:01.459254 28647 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.459944 28647 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.461582 21858 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 18. Ops: []
I20260709 13:08:01.463125 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.447937 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.354671 28616 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:01.442066 28624 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:08:01.445147 28583 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.468214 28583 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.470533 28583 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.471575 28584 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.472244 28647 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.473905 27970 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:01.474488 27970 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.474788 28348 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.476622 27970 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:01.476835 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1181) took 1781 ms (client timeout 2243 ms). Trace:
I20260709 13:08:01.477408 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.479378 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.480175 22584 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:01.481866 28492 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:01.482296 28492 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:01.477321 21930 rpcz_store.cc:269] 0709 13:07:59.718328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.748538 (+ 30210us) service_pool.cc:224] Handling call
0709 13:08:01.476681 (+1728143us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002a7454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:01.476779 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556553984}
I20260709 13:08:01.500219 21859 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:01.363119 28618 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:01.505108 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67908.3us
I20260709 13:08:01.508025 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.509745 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:01.514259 22591 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:08:01.514357 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.524770 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:01.525969 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:01.520010 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:01.527334 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:01.446691 28628 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.529567 21851 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:01.531231 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 157248us
I20260709 13:08:01.515601 28616 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:01.532603 21858 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:01.532948 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:01.540804 22003 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:01.543679 22004 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:01.522204 28583 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.547940 21859 raft_consensus.cc:2393] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:01.526218 28454 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:01.550237 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 303755us
I20260709 13:08:01.525532 28649 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:01.536576 28616 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.557713 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.537658 28001 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.544377 28348 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.543648 28360 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:01.564577 21854 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:01.567065 28212 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:01.573613 28212 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.575413 21859 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 17. Ops: []
W20260709 13:08:01.576505 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61216.7us
I20260709 13:08:01.535257 28492 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.581723 28492 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:01.562213 28616 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:01.569947 28001 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.553956 27861 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:08:01.582540 28650 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:01.592581 28376 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:01.578905 28584 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.553815 28454 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.552207 28583 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.570557 28521 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.599380 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.594818 28628 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.611578 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:01.613082 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.596848 28647 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:01.614337 21854 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:01.589438 28348 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:08:01.623126 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.623960 21859 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:01.643101 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.643359 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.643887 21926 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:01.643800 21934 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:01.640158 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 95751.3us
I20260709 13:08:01.646349 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.651193 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.651890 21860 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:01.648357 21857 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:01.643108 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.654059 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.660461 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.634974 28278 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:01.637401 28608 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.657218 22590 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:01.637565 28567 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.639125 28584 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.668463 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.667423 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.669731 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.669170 21856 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:01.670264 21859 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:01.673122 21852 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 13:08:01.675822 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.676414 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.676944 21859 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:01.666072 28376 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:01.678246 28651 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:08:01.679746 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.677912 28376 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.677973 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.681321 21859 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:01.688333 21856 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:01.698271 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.702139 28348 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:01.700711 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.703161 21859 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:01.680241 28349 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:08:01.704366 21857 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:08:01.622587 28650 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.579274 28649 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.707108 28650 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.673122 28278 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:01.712810 21859 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:01.668601 28584 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.715278 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.715741 28655 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:08:01.716447 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.717420 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:01.718032 28583 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.718779 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.719022 28001 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.719214 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.719885 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.720672 21926 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:01.720017 21932 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:08:01.721318 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.726864 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.719995 28454 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.729619 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.731725 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:01.732060 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56663us
I20260709 13:08:01.732429 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.733225 22003 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:01.733583 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.734449 21931 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:01.735061 28397 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.735720 27861 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:01.721294 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:01.736506 28001 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:01.737134 21852 raft_consensus.cc:2410] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:08:01.737807 22002 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:08:01.726253 21929 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:01.735601 28397 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.736748 27861 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:01.734812 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:01.742468 28646 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:01.743722 28278 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.744270 21859 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:08:01.771433 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.745761 21754 catalog_manager.cc:5697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 5 to 8, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 8 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:01.717448 28212 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.747694 28404 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:01.748726 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.750162 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.752436 28648 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:01.754595 28584 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.755074 21930 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:01.771142 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.852283 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.745668 28376 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.770988 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.810652 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:01.859252 22006 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:01.860267 28637 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:01.811559 21851 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:08:01.865896 22003 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:01.866976 27998 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.846980 21932 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:08:01.802141 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3095) took 1023 ms. Trace:
I20260709 13:08:01.853070 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:01.869626 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.867637 27998 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.868810 21935 rpcz_store.cc:276] 0709 13:08:00.732030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.733322 (+  1292us) service_pool.cc:224] Handling call
0709 13:08:01.755401 (+1022079us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b5400429fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:01.755537 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65286912}
I20260709 13:08:01.755784 28649 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:01.797933 28618 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:01.801069 28646 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:01.887682 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.802021 27861 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:01.837045 28648 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:01.892134 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.899405 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:01.898317 22007 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:01.901862 21928 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:01.899693 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:01.906471 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 171730us
I20260709 13:08:01.907724 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:01.908293 22003 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:01.755941 28656 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.163s
I20260709 13:08:01.811947 28521 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:08:01.802768 28608 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 13:08:01.875615 22006 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:01.924113 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.926553 22001 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:01.927753 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.929075 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.929840 21929 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:01.933557 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.938352 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:01.940268 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:01.943249 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 263599us
I20260709 13:08:01.947943 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.926450 28521 consensus_queue.cc:1243] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:08:01.954282 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.927259 28052 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:01.955083 22593 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:01.921152 28614 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:01.921089 28613 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:01.957512 28614 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.958266 28613 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.938712 27970 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:08:01.952036 22001 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:01.962370 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:01.968453 22005 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:01.962996 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3130) took 1043 ms. Trace:
I20260709 13:08:01.968817 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.942335 28655 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.902993 28618 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.973667 22591 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:01.972390 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:01.968883 22004 rpcz_store.cc:276] 0709 13:08:00.909803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.909950 (+   147us) service_pool.cc:224] Handling call
0709 13:08:00.923563 (+ 13613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.544171 (+620608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.953103 (+408932us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b540042c2d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:01.953267 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425305344}
I20260709 13:08:01.980732 22589 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:01.945788 28518 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:01.977190 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.983038 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:01.960883 28614 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:01.984149 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:01.989362 21929 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:01.956537 28052 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.955509 28646 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:01.994372 28646 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:01.961966 28492 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:01.997327 28492 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.966601 28349 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.988206 28212 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:01.981181 28502 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:02.001636 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:01.993070 22585 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:08:02.002555 22585 pending_rounds.cc:85] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 13
I20260709 13:08:02.002169 28502 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:01.985934 22587 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:08:02.002811 22585 pending_rounds.cc:107] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:08:02.004086 22585 raft_consensus.cc:2889] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:02.004539 28397 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.002830 22593 raft_consensus.cc:2410] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 15 because replica has last-logged OpId of term: 14 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 13:08:02.005899 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.006265 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.006793 22005 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:01.966938 28454 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:02.008188 28454 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.011195 28454 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:02.011857 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.012384 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.012907 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:02.013219 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 74998.5us
I20260709 13:08:02.014374 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.015226 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.016295 28116 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:02.016599 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.015264 22006 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:08:02.017925 28116 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:02.012713 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1456) took 1536 ms. Trace:
I20260709 13:08:01.989132 28583 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:08:02.019519 22590 rpcz_store.cc:276] 0709 13:08:00.476174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.980114 (+503940us) service_pool.cc:224] Handling call
0709 13:08:02.012535 (+1032421us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54006009d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:02.012634 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524829952}
I20260709 13:08:02.001749 28212 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.022672 28370 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:08:02.023882 22591 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:08:02.024369 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.018146 28616 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.013062 22001 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:08:02.027935 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.023327 28212 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.030763 28650 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:02.031237 28650 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.031601 28650 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.045302 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:02.048473 22584 raft_consensus.cc:2410] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:08:02.036206 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.045436 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.050071 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.051867 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:02.049961 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:02.052589 22593 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:08:01.984148 28649 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.054723 22005 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:02.053102 28637 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.057085 28637 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:02.057682 28637 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.051222 28648 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.058172 28001 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:02.023607 28349 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:02.058717 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.058552 28648 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:02.063429 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.064554 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.062243 22590 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:02.073606 22586 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:02.059598 28635 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:02.074632 28635 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:02.057112 28663 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:08:02.075074 28635 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.061638 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:02.047776 28539 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.062283 28647 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.075834 28539 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.076372 28647 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.076783 28647 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.063529 28360 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:02.077359 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.077922 28539 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.078112 21858 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:08:02.050843 28492 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.078555 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.079164 28624 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:01.961362 28613 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:02.079727 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.079862 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:02.080327 22592 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:02.080490 22584 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.081377 21851 pending_rounds.cc:85] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 11
I20260709 13:08:02.081670 21851 pending_rounds.cc:107] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:08:02.081907 21851 raft_consensus.cc:2889] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:02.050608 28518 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:02.082635 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:02.082890 28518 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.083323 28518 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:02.084184 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.074721 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.084508 28349 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.077430 28360 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:02.051090 28116 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:08:02.084853 21857 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:02.085914 28649 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.086318 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.086167 28360 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.061964 28348 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.087112 21858 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:02.084925 22586 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:02.087328 28348 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.088040 28116 consensus_queue.cc:1243] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:02.082654 28405 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:01.955065 27861 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.087190 28360 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.089989 27861 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.034243 28655 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.068288 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.093917 28616 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.071535 21972 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.095361 28662 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:02.095778 28662 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:02.097265 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.097896 21852 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:02.101641 21858 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:02.096617 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.105209 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:02.105693 28502 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:02.106207 22586 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.107152 28116 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:02.108201 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53588.9us
I20260709 13:08:02.060673 28618 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.110507 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.069866 21900 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:02.115078 22003 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:02.110746 28637 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.115960 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:02.118600 21928 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:02.122843 22008 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:08:02.127592 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 57856.4us
I20260709 13:08:02.044497 28533 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:02.131270 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.044718 28052 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.138178 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.139232 22005 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:02.140738 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.149159 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.147773 21860 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:02.151262 21857 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:08:02.155031 22584 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:02.155622 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.159688 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:02.175622 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.177161 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:02.170842 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.179721 22590 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:08:02.181113 21900 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:02.173715 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:02.173044 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.147845 28616 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:02.190989 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.188176 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.152887 28052 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.189425 28065 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:02.189909 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.152881 28661 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:02.195837 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.143285 28370 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:08:02.129364 28624 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:08:02.174610 28116 raft_consensus.cc:2764] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:08:02.185202 28348 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.190738 28539 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:02.096676 28662 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:02.257507 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 102083us
I20260709 13:08:02.246660 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.259927 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.261058 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.250746 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.252400 22584 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:02.252599 22590 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:02.191431 28583 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:02.243803 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 55718.5us
I20260709 13:08:02.195556 28609 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:02.195040 28387 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:02.243333 28661 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:02.193656 28533 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:02.275246 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.286105 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:02.295964 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.299763 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.300735 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.301944 22003 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:02.282457 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.288151 22005 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:02.264775 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:02.281437 21855 pending_rounds.cc:85] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 9
I20260709 13:08:02.291755 22001 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:02.244644 28483 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:02.307780 22556 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:02.311175 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48142.4us
I20260709 13:08:02.313467 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:02.197165 28608 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:08:02.257051 28539 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.312160 28483 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:02.314291 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:02.428936 21754 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
W20260709 13:08:02.594205 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 280053us
I20260709 13:08:02.444352 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.274142 28502 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:02.436165 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.386866 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:02.463430 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 165042us
W20260709 13:08:02.627087 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.279793 28609 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:02.643515 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:02.546129 22001 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:02.355895 21855 pending_rounds.cc:107] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 20.10
W20260709 13:08:02.502466 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 227279us
I20260709 13:08:02.354090 22590 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:02.427291 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.425873 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.645187 21855 raft_consensus.cc:2889] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:02.476506 21756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 193) took 39523 ms (client timeout 15000 ms). Trace:
I20260709 13:08:02.388316 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:02.505700 22589 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.001s
I20260709 13:08:02.491986 22000 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
W20260709 13:08:02.501273 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 193890us
I20260709 13:08:02.371246 28212 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:02.406675 22008 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:02.504161 22586 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:02.527279 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.388801 28662 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.427518 28584 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.352500 28492 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.237s
I20260709 13:08:02.264654 28583 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:02.650156 28609 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.281615 28387 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.589640 22005 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:02.658552 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:02.670378 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:02.518802 28646 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.423401 28397 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:02.477826 28376 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:02.686153 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.688113 22005 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:08:02.410074 28637 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:02.690785 28648 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:02.353451 28665 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:08:02.691560 28648 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:02.650161 21756 rpcz_store.cc:269] 0709 13:07:22.953026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.953317 (+   291us) service_pool.cc:224] Handling call
0709 13:08:02.476143 (+39522826us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:55.569508 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:55.721436 (+151928us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:55.721492 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:07:55.721511 (+    19us) tablet.cc:662] Decoding operations
0709 13:07:55.723738 (+  2227us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:55.723831 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:55.723861 (+    30us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:07:55.731725 (+  7864us) tablet.cc:701] Row locks acquired
0709 13:07:55.731819 (+    94us) write_op.cc:260] PREPARE: finished
0709 13:07:55.732105 (+   286us) write_op.cc:270] Start()
0709 13:07:55.732319 (+   214us) write_op.cc:276] Timestamp: P: 1783602475731999 usec, L: 0
0709 13:07:55.732350 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:07:55.772927 (+ 40577us) log.cc:844] Serialized 10642 byte log entry
0709 13:07:56.828752 (+1055825us) op_driver.cc:464] REPLICATION: finished
0709 13:07:57.304668 (+475916us) write_op.cc:301] APPLY: starting
0709 13:07:57.328355 (+ 23687us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:57.662864 (+334509us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:57.715779 (+ 52915us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:58.553825 (+838046us) spinlock_profiling.cc:243] Waited 7.03 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cc32a 00007f3b70751f14 00007f3b70751ec3 00007f3b70751e38 00007f3b70751df3 00007f3b707510dc 00007f3b7077ae16 00007f3b70762cbb 00007f3b7074d39d 00007f3b70750d75
0709 13:07:59.173256 (+619431us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:59.173737 (+   481us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:07:59.173785 (+    48us) write_op.cc:312] APPLY: finished
0709 13:07:59.174394 (+   609us) log.cc:844] Serialized 236 byte log entry
0709 13:07:59.741224 (+566830us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:59.841832 (+100608us) write_op.cc:454] Released schema lock
0709 13:07:59.842053 (+   221us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:59.842968 (+   915us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:59.843006 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10770935,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":440749,"apply.run_cpu_time_us":71418,"apply.run_wall_time_us":2557444,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117149,"num_ops":27,"prepare.queue_time_us":334238,"prepare.run_cpu_time_us":17435,"prepare.run_wall_time_us":391195,"raft.queue_time_us":449187,"raft.run_cpu_time_us":20753,"raft.run_wall_time_us":1763713,"replication_time_us":1079009,"spinlock_wait_cycles":16174848,"thread_start_us":926190,"threads_started":5,"wal-append.queue_time_us":797758,"wal-append.run_cpu_time_us":18485,"wal-append.run_wall_time_us":2560857}]]}
I20260709 13:08:02.685365 28376 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.694325 28648 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:02.695252 28376 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.673861 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:02.696856 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:02.697672 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:02.698678 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.699682 22009 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:02.440596 28635 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.590996 28539 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.195701 28052 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.420285 28647 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.517249 28533 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:02.357740 28655 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.294175 28661 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.461609 22007 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:02.595939 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.669562 28567 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:02.420475 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.604779 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.721488 21851 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
W20260709 13:08:02.431595 28667 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:08:02.312736 28613 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:02.371359 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 180484us
I20260709 13:08:02.731071 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:02.477979 28521 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:02.422024 28628 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.650540 22589 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:02.354365 28116 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:02.653602 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:02.538597 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55423.1us
I20260709 13:08:02.567015 22593 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:02.305804 28670 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:02.654320 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.724960 28567 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.425421 28649 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.650151 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.657557 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:02.598591 28502 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.598929 27861 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:02.614118 28483 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:02.662645 22586 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:02.665407 28492 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:02.426506 27998 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:02.666739 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 230654us
W20260709 13:08:02.754783 22006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.000s	sys 0.021s
I20260709 13:08:02.273857 28618 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.461983 28650 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.489734 28616 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:08:02.765878 28533 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.684783 28608 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:02.767035 28608 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.767402 28608 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:02.704802 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.666045 28583 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.771018 28360 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:02.708760 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.774119 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.775429 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:02.741801 28628 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.710443 28656 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:02.678112 28387 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:02.780230 21858 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:08:02.780757 21858 pending_rounds.cc:85] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 7
I20260709 13:08:02.781199 21858 pending_rounds.cc:107] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:08:02.781491 21858 raft_consensus.cc:2889] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:02.710493 21754 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:02.722411 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.724900 22003 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:02.724062 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.728935 21974 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:02.733547 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.784883 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:02.785494 21851 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:02.730237 22585 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:02.706156 28065 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:02.735190 21860 raft_consensus.cc:2368] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:08:02.741381 22588 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:02.693485 28397 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.704625 28646 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.746388 22008 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:02.745823 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:02.716080 28677 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:08:02.730286 28613 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:02.489715 28518 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.756807 28376 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:02.759148 28492 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:02.756657 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:02.757296 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 107254us
I20260709 13:08:02.828469 21853 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.829118 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.829159 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.748533 28567 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.766486 28616 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:02.775564 28052 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.667157 28584 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:02.705978 22590 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:08:02.844365 22590 pending_rounds.cc:85] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 12
I20260709 13:08:02.844832 22590 pending_rounds.cc:107] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:08:02.845124 22590 raft_consensus.cc:2889] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:02.776549 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.731946 27970 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:08:02.650404 21852 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:02.787462 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 58570.5us
W20260709 13:08:02.798269 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.747885 27861 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.747792 28502 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.756703 28618 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:02.755399 22006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.000s	sys 0.021s
I20260709 13:08:02.781968 28278 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:02.826790 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 118123us
I20260709 13:08:02.830734 28001 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:02.830331 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:02.862282 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.836557 22008 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:02.818590 21754 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 2 to 10, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 10 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.743603 28521 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.790480 28065 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.853818 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.805932 28648 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:02.855249 22004 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:08:02.818392 28613 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.856987 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:02.819036 28376 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:02.871238 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 218872us
I20260709 13:08:02.844977 28616 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.858747 27861 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.841034 28637 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:08:02.839569 28404 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.858364 28650 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:02.894517 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.892674 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.903223 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:02.896847 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:02.874428 27970 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.863691 28370 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:02.889500 28376 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:02.892414 28656 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:02.857983 28618 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.871656 28065 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:02.929311 27970 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:02.906260 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.885047 28212 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:02.932961 28397 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.939893 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:02.946489 21858 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:08:02.947278 28052 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:02.947880 21858 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:02.941934 28518 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:02.950150 28518 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:02.950516 28518 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.950856 28518 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:02.942139 28635 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:02.864708 28001 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:02.913820 21851 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
W20260709 13:08:02.892596 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 195003us
W20260709 13:08:02.905872 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.522s (SENT) [suppressed 10 similar messages]
I20260709 13:08:02.932219 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:02.897545 28649 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:02.898082 28404 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:02.910748 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 179749us
I20260709 13:08:02.896736 28616 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:02.968910 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:02.942548 28212 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:02.978608 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 239293us
I20260709 13:08:02.973717 28647 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:02.983134 28212 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:02.954097 28381 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:02.895318 28116 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:02.954455 28650 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:03.005206 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:02.972087 28001 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.976656 28404 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.984014 28624 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:03.014670 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:03.030536 21754 catalog_manager.cc:5697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 5 to 9, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:03.003618 28694 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:02.984200 28677 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:08:03.015374 28647 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:03.003736 28652 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:08:03.033919 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 171824us
I20260709 13:08:02.984339 28656 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:08:03.017061 28650 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:03.041934 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 136069us
I20260709 13:08:03.022799 28624 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.049978 28624 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.023937 28648 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:03.032600 28637 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:03.045984 28646 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:03.065776 21754 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 5 to 8, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 8 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:03.066125 28376 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.069262 21853 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:03.071314 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 66286.3us
I20260709 13:08:03.047449 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:03.025660 28688 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:08:03.041427 28690 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:03.078086 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:03.082904 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 358847us
W20260709 13:08:03.084120 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69532us
I20260709 13:08:03.084758 21825 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.086050 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.089859 28405 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:03.090406 28405 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:03.063517 28647 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.091261 28405 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.093238 28647 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.094992 28706 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:03.097549 28583 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.098330 28628 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.100021 21859 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:03.057104 28648 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.103775 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:03.096141 21754 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 4 to 10, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 10 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.108505 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:03.110185 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.123214 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.123790 22587 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.120154 28608 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.084340 28397 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.127197 21754 catalog_manager.cc:5697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 6 to 7, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 7 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.134207 28521 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.134912 28608 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.136132 28539 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:03.141189 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93838.7us
I20260709 13:08:03.142130 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.142875 28376 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.143296 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.143656 28376 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.143909 28278 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:03.144655 28278 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:03.106863 28533 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:03.145697 21927 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:03.168192 28533 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:03.168638 28533 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:03.172092 28518 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.147697 28397 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.150454 28360 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:03.204851 28502 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.206247 22588 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:08:03.206522 28616 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.207363 28502 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:03.151520 27861 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.120479 28628 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:08:03.106771 28649 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:03.153321 28698 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:03.134931 28692 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:08:03.107199 28583 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.155567 28387 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.156733 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.089105 28650 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.103590 28584 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.122834 28690 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:03.159739 28212 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.126418 28687 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:03.161729 28693 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:08:03.166865 28618 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.242252 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.091275 27970 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.103588 28116 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.173919 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3021) took 1107 ms. Trace:
I20260709 13:08:03.176685 28404 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:03.179229 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2569) took 1461 ms (client timeout 1550 ms). Trace:
W20260709 13:08:03.263252 21927 rpcz_store.cc:269] 0709 13:08:01.718301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.278595 (+560294us) service_pool.cc:224] Handling call
0709 13:08:03.179022 (+900427us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54002a7e54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:03.179165 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288114432}
I20260709 13:08:03.231469 28387 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.183319 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.264672 22005 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:08:03.147270 28670 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:03.144570 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:03.184620 22587 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:03.184217 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1300) took 1081 ms. Trace:
W20260709 13:08:03.270850 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 126333us
I20260709 13:08:03.270623 22592 rpcz_store.cc:276] 0709 13:08:02.102621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.408391 (+305770us) service_pool.cc:224] Handling call
0709 13:08:02.441444 (+ 33053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:02.832114 (+390670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:02.832143 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:02.832182 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:02.832212 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.183825 (+351613us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b5400668354. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:03.183893 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:03.184138 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897934720}
I20260709 13:08:03.130584 28701 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:08:03.106977 28696 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:03.144256 28376 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.190577 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.197821 28492 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.200935 28609 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.201710 28518 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.275616 28696 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.291402 28696 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.291258 28613 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:03.202955 28662 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.001s
I20260709 13:08:03.204028 28397 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.208806 22585 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
W20260709 13:08:03.221661 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 136088us
I20260709 13:08:03.295413 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:03.222283 28707 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:03.279661 28698 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:03.297379 28707 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.300396 28707 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:03.222663 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.259738 28690 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.228834 28656 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.233989 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:03.344769 21860 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.229236 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.345886 21929 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:03.158380 28065 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:03.220249 28682 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
W20260709 13:08:03.236268 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 203359us
I20260709 13:08:03.242297 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.243301 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.232008 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41410
I20260709 13:08:03.350888 21928 raft_consensus.cc:606] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Received request to transfer leadership
I20260709 13:08:03.351397 21928 raft_consensus.cc:614] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:03.351591 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:03.253196 22004 rpcz_store.cc:276] 0709 13:08:02.066218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.115090 (+ 48872us) service_pool.cc:224] Handling call
0709 13:08:02.115434 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:02.872393 (+756959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.173666 (+301273us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b5400426154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:03.173864 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700818688}
I20260709 13:08:03.352293 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.254910 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.257604 21926 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:03.166882 28483 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.226280 28647 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:03.272231 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.238198 28706 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.279198 21853 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:08:03.279198 22590 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:08:03.256075 28661 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 13:08:03.280727 28492 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:03.281385 28404 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.282618 28609 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.286397 27861 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.286592 28376 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.287753 28624 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.353144 22589 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:08:03.346179 28711 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.006s
I20260709 13:08:03.217929 28689 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 13:08:03.294556 21857 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:08:03.343153 28616 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.297401 28698 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.303874 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.407270 28698 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:03.408237 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:03.222448 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 112365us
I20260709 13:08:03.372300 28710 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:08:03.342736 28584 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:08:03.315815 28533 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.321599 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.426514 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.427003 21860 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:03.334231 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.402879 28624 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.339115 22001 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:08:03.340312 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.228609 21935 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:03.343878 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:03.344085 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.342240 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:03.352720 21932 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:08:03.218919 28649 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.372128 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.429315 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.352293 21851 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.430593 22005 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:03.397197 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44786.2us
I20260709 13:08:03.435595 21898 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.397387 21931 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:03.437259 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2654) took 1307 ms. Trace:
I20260709 13:08:03.437592 22589 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
W20260709 13:08:03.397730 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 125607us
I20260709 13:08:03.377928 28539 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.373600 28065 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.398622 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:03.381711 22584 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:03.288273 28618 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:03.296712 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:03.369922 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.350301 21934 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:03.441433 22005 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:08:03.435331 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.432614 22004 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:03.353804 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:03.377331 28212 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.415185 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.436293 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:03.375818 28567 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.000s
I20260709 13:08:03.403131 28387 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:03.445891 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.460379 28678 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.373921 28690 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.461707 28533 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.350286 28656 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.464746 28656 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.427135 22008 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.332297 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:03.427845 22587 raft_consensus.cc:2368] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:08:03.338061 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.428233 22009 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
W20260709 13:08:03.406581 28716 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:08:03.429315 28613 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.346979 28001 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.458298 28567 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.475307 28001 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:03.475646 21852 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:03.476054 22000 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:03.476598 28567 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.444394 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.450088 28387 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.425145 28360 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:03.481900 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 185237us
I20260709 13:08:03.432165 28649 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.458838 22002 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:03.468914 22007 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.438776 21855 rpcz_store.cc:276] 0709 13:08:02.130004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.280830 (+150826us) service_pool.cc:224] Handling call
0709 13:08:02.281228 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:02.655905 (+374677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:02.655933 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:02.665614 (+  9681us) log.cc:844] Serialized 52 byte log entry
0709 13:08:02.928661 (+263047us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:02.928703 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.928786 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:03.436928 (+508142us) raft_consensus.cc:1685] finished
0709 13:08:03.436946 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:03.437118 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6164,"raft.queue_time_us":11348,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":14205,"thread_start_us":262574,"threads_started":1,"wal-append.queue_time_us":317822}
I20260709 13:08:03.470232 22585 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:03.441798 28689 consensus_queue.cc:1243] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:03.472100 22004 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:08:03.490404 22008 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:03.473037 21851 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:03.469586 27998 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:03.477521 28714 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:03.477849 28713 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:03.478430 28688 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:08:03.494407 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50086.1us
I20260709 13:08:03.349993 28662 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.483704 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.403837 28661 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.404019 28647 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:03.484464 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48959us
I20260709 13:08:03.519102 21900 leader_election.cc:400] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:08:03.519646 21900 leader_election.cc:403] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:08:03.520084 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.520525 28708 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.491928 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.523780 28348 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:03.524165 28348 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:03.524535 28348 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.524967 28360 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.525174 28583 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:03.525650 28360 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.526332 28583 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:03.459507 28370 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.529174 28677 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:03.529599 28677 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:03.529481 28370 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.530507 28677 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.531662 28370 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.529218 28652 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:03.534538 28652 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:03.482636 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:03.459405 28637 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:03.492717 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.476540 28696 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.494140 22005 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:08:03.537961 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.541625 22002 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:03.498997 28714 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.499616 28713 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:03.502614 22004 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:03.553293 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:03.553941 22004 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:03.466972 28678 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:03.506672 28709 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.006s
I20260709 13:08:03.504503 28405 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:08:03.507836 28608 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:03.557641 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.502468 28688 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.481668 27861 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:03.481753 28387 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:03.531736 28677 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.548789 28616 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:03.538606 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:03.492524 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.539561 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:03.540809 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48164.4us
I20260709 13:08:03.581640 22008 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.498121 28707 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:03.498608 27998 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:03.583299 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44762.3us
I20260709 13:08:03.520467 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:03.550419 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:03.584437 22585 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:08:03.554294 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:03.584867 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64496us
I20260709 13:08:03.569710 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.528679 28583 consensus_queue.cc:1059] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:08:03.535337 28652 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.563853 22557 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.494148 28065 raft_consensus.cc:2764] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:08:03.576920 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.519136 28647 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.578588 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:03.540792 28376 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:03.606850 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1298) took 1504 ms. Trace:
I20260709 13:08:03.540548 28637 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:03.536520 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53972.7us
I20260709 13:08:03.553418 28670 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:03.495477 22000 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.551383 22007 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:03.558420 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.575274 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.638725 22587 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.585199 22002 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:03.588364 22591 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:03.589107 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.589869 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:03.591339 27861 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:03.587158 28521 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:08:03.643810 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53986.6us
I20260709 13:08:03.555224 28689 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:08:03.505628 28662 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.548847 28706 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.576421 28608 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.608623 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.617537 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.610844 22584 rpcz_store.cc:276] 0709 13:08:02.102107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.311545 (+209438us) service_pool.cc:224] Handling call
0709 13:08:02.311892 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:03.440676 (+1128784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.606606 (+165930us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54005631d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:03.606787 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459628416}
I20260709 13:08:03.621228 22588 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:03.633559 22008 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:03.648209 28001 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:03.580484 28713 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.643976 28649 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.599741 28688 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.599927 28714 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.637086 28637 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:03.576556 28650 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:08:03.608625 28710 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:03.654592 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.656657 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.682263 21934 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:03.683100 21929 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:08:03.663049 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:03.666545 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.608572 28618 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:03.669039 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51570.1us
I20260709 13:08:03.670634 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.606274 28696 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:03.637319 28405 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:03.596036 28661 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.672864 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.672786 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.673354 21927 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:08:03.654608 28609 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:08:03.676460 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87444.1us
I20260709 13:08:03.662964 28567 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:03.672770 28376 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:08:03.671581 28708 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20.
I20260709 13:08:03.643275 27861 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:03.677590 28649 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.680110 21928 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:03.673939 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67641.7us
I20260709 13:08:03.686333 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.718587 28618 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:03.599277 28348 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.719105 28618 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:03.719173 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:03.720053 21853 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:03.720116 28618 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.594043 28715 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:08:03.690551 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.693586 22587 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:03.684448 28710 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:03.680902 28616 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.663398 28670 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:03.670789 28492 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:03.708320 28405 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.713064 28688 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.707726 21935 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:03.701975 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.673906 28521 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.711896 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.598901 27998 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.744084 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:03.721087 28348 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:03.714093 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 102139us
I20260709 13:08:03.722764 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.746628 22008 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.748695 22005 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:03.727563 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.728145 21927 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:03.704820 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.698655 28539 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:03.705698 21933 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:08:03.751883 22004 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:03.734046 22587 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:03.752245 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.752344 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:03.711336 28609 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:03.753203 22009 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.753544 28609 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:03.716111 27861 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.741564 28690 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:03.751457 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.759778 28690 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.761336 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.716503 28714 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.697705 28707 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:08:03.773818 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.745080 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.755462 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:03.728770 28710 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.756016 22002 raft_consensus.cc:1240] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:03.758437 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.737238 28405 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:03.768267 21858 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:03.714800 28387 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.782783 28387 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:03.786880 28721 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.335s	user 0.002s	sys 0.001s
I20260709 13:08:03.775604 22590 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:03.786896 22006 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:08:03.791254 28708 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:03.791764 28708 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:03.792382 21934 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:03.742631 28483 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.720914 28694 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:08:03.722368 28650 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:08:03.713618 28567 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.772940 22008 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:03.708408 28608 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.762636 28001 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:03.745697 28584 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:08:03.779479 28278 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:03.817194 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 95545.9us
I20260709 13:08:03.833957 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:03.800652 28686 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:03.843191 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.760596 28521 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.845399 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.846042 21934 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:03.847014 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.848130 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.792568 28708 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.851922 28360 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.853240 21935 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:03.853780 21927 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:03.854858 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.856052 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.756914 28116 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.055s
W20260709 13:08:03.857504 28727 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:08:03.858441 28370 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.859228 28370 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.862440 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.867866 22000 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:08:03.873306 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 168525us
I20260709 13:08:03.856644 21927 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:08:03.883559 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.885098 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.893865 28116 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:03.897071 28637 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:03.907230 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 216722us
W20260709 13:08:03.907939 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 144424us
I20260709 13:08:03.911615 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:03.861924 28649 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:03.840312 28001 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:03.910115 22001 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 15: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:03.910184 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:03.897890 28628 raft_consensus.cc:3055] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:08:03.920465 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.923734 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 153013us
I20260709 13:08:03.893217 28650 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:03.838820 28483 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.944113 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:03.800915 28701 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:08:03.945451 21927 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.909453 28637 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:03.950979 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:03.953416 22556 leader_election.cc:336] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.243s (SENT)
I20260709 13:08:03.957697 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:03.900355 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.940733 28700 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:08:03.913661 28608 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.970897 21898 leader_election.cc:400] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:08:03.972973 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:03.976429 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 225033us
I20260709 13:08:03.977619 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:03.952975 28712 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:03.949651 28628 raft_consensus.cc:740] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:03.945298 28725 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:03.954977 28650 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:03.962966 28649 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:03.839460 28616 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:03.912978 28670 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:03.973110 28376 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:04.028170 21934 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:03.987300 28662 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:04.019780 28539 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:04.043654 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:04.024073 28656 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:04.042966 28662 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:04.025733 28696 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:04.020414 28694 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.019904 28521 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:08:03.970113 28483 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.059499 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:04.029384 28742 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:08:04.107604 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:04.113346 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:04.114202 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:04.031517 28698 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.006s
I20260709 13:08:04.072889 28628 consensus_queue.cc:260] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.107199 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:04.074770 28662 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:04.134697 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 122111us
I20260709 13:08:04.139740 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:04.135684 21926 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:04.090808 28730 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:08:04.142853 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 193839us
I20260709 13:08:04.144747 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.145349 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.146179 28065 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:04.154677 28065 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:04.155630 28065 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.146759 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.158006 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.159070 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:04.146871 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:04.147482 22003 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:04.153264 22556 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:04.147710 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:04.162714 22007 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:04.161815 22003 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:08:04.057590 28656 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:04.163647 22002 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:04.153132 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:04.166544 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:04.168169 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 190655us
I20260709 13:08:04.168205 22009 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:04.170647 22006 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:04.139096 28698 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.175508 28698 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.176023 22000 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:04.143258 28662 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:04.165891 28628 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:04.147702 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:04.150911 28370 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:04.093743 28694 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.189492 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:04.198048 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50413.2us
I20260709 13:08:04.165361 28567 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.168715 28696 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:04.110186 28649 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.175197 28710 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:04.223176 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 43553.8us
I20260709 13:08:04.091648 28001 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.200747 21927 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
W20260709 13:08:04.226848 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 266829us
I20260709 13:08:04.189318 28360 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:04.230172 21934 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:08:04.231190 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:04.205833 28539 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.242034 21857 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:04.179006 28609 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.232460 21858 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:04.232594 21934 pending_rounds.cc:85] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 13
I20260709 13:08:04.252789 21934 pending_rounds.cc:107] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:08:04.253115 21934 raft_consensus.cc:2889] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:04.242480 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.253883 21851 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:04.230240 28370 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:04.249912 21931 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:04.250119 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.240774 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:04.252905 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:04.168934 28650 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.258662 21853 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:04.258744 21933 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:04.259924 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.263687 21851 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:04.230103 28696 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:04.259459 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:04.271478 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:04.270471 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.269727 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.270254 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:04.243355 28718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.017s	sys 0.017s
I20260709 13:08:04.250823 28539 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.256935 28678 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.312060 22009 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:04.309298 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.645852 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:04.309190 21859 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:08:04.328444 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.328840 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.307325 21929 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:04.388304 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.650483 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:04.651218 22557 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:04.309723 28348 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:04.328668 28650 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:04.536123 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:04.665320 22003 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:08:04.460408 22008 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:08:04.271745 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:04.666821 21852 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:04.240958 28483 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.232035 28736 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:04.634032 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:04.668210 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:04.651506 22592 raft_consensus.cc:2410] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:08:04.623629 22590 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:04.668918 22588 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:04.635121 28718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.171s	user 0.025s	sys 0.018s
I20260709 13:08:04.629634 28614 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:04.272212 28753 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.365s
I20260709 13:08:04.305799 28567 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.301s
W20260709 13:08:04.598384 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 129961us
W20260709 13:08:04.668272 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:04.688949 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:04.689491 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.670168 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:04.673280 21853 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:04.620479 28715 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.677743 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:04.622334 28360 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:04.687145 21927 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:08:04.630723 28370 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:04.703442 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:04.704066 22002 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:04.701808 22591 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:04.664057 28746 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.435s	user 0.000s	sys 0.001s
W20260709 13:08:04.671025 27861 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.905s	user 0.009s	sys 0.005s
I20260709 13:08:04.702940 22008 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:04.310375 28656 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.710505 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:04.679780 28584 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:04.706249 28714 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:04.708750 28656 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:04.705425 28677 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:04.768051 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.718761 28348 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:04.774515 22002 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:04.701839 28614 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.722129 28698 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:08:04.709115 28360 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:04.801301 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.709338 27861 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.747157 28616 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:08:04.800462 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.768191 28618 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:04.805523 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:04.712039 28715 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.773962 28735 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:04.777804 28769 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.788092 28694 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:04.795909 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 174461us
I20260709 13:08:04.777778 28737 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:04.705147 28696 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:04.839633 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 131527us
I20260709 13:08:04.793733 28751 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:04.847836 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:04.848930 28518 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:04.849365 28738 raft_consensus.cc:476] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Not starting pre-election -- already a leader
I20260709 13:08:04.849491 28518 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:04.850029 28738 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.850518 28738 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.850561 28518 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.844008 28737 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:04.853183 28694 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.853121 28737 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.854267 27861 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.853827 28694 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.805598 28710 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.773265 28677 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:04.807792 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 97364.3us
W20260709 13:08:04.844516 28758 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.579s	user 0.000s	sys 0.001s
I20260709 13:08:04.864506 28698 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:04.818615 28609 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:04.825700 28404 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:04.852844 28738 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:04.880688 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.847992 28618 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:04.878307 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:04.871248 28756 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.414s	user 0.000s	sys 0.001s
I20260709 13:08:04.845825 28769 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.869643 28698 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:04.874708 28616 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:08:04.882309 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.804282 28370 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.846976 28483 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:08:04.878489 22591 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:08:04.936981 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58778us
W20260709 13:08:04.881557 28405 consensus_meta.cc:287] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.091s	user 0.012s	sys 0.000s
I20260709 13:08:04.877554 28710 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:04.847406 28735 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.958122 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:04.959044 21853 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:04.952205 28775 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:04.961812 28614 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:04.955808 28770 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:04.970814 28735 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.971145 28770 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:04.801187 28750 consensus_queue.cc:1175] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" responder_term: 20 status { last_received { term: 17 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.13-19.13]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:08:04.973814 28757 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.383s	user 0.000s	sys 0.002s
W20260709 13:08:04.976529 28760 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:08:04.979511 28769 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:04.981376 28764 log.cc:927] Time spent T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:08:04.982980 28747 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.581s	user 0.001s	sys 0.000s
I20260709 13:08:04.983762 28714 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:04.937331 28616 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:04.985283 28771 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.985165 28616 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.986506 28616 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:04.986074 28771 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.952893 28539 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:04.990059 28628 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.793s	user 0.008s	sys 0.003s
I20260709 13:08:04.993934 28360 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.881570 28348 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:04.947527 28405 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:04.997561 28405 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:04.998575 21860 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:08:04.999619 21852 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
W20260709 13:08:04.984696 28761 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:08:05.003655 22588 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:08:05.004882 21854 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:08:05.007095 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 355969us
I20260709 13:08:05.008363 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:05.013073 28738 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:05.015578 28738 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.015947 28738 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:05.017053 28762 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:05.017694 28762 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.018558 28405 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.019076 28405 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.019860 28762 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:05.021703 28405 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:04.961829 28775 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:08:04.996991 28688 consensus_meta.cc:287] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.256s	user 0.001s	sys 0.017s
W20260709 13:08:04.968798 28608 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.990s	user 0.015s	sys 0.008s
I20260709 13:08:05.023407 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:05.029475 21853 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:05.018896 28708 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:05.029549 22588 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:05.039592 22591 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:08:05.040135 22591 pending_rounds.cc:85] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 12
I20260709 13:08:05.040403 22591 pending_rounds.cc:107] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:05.040688 22591 raft_consensus.cc:2889] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:05.041857 22556 leader_election.cc:336] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.522s (SENT)
I20260709 13:08:05.042114 22590 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:08:05.042480 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:05.006740 28768 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:05.019934 28652 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:05.046555 21932 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:05.043584 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:05.054543 22591 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
W20260709 13:08:05.046566 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 198764us
I20260709 13:08:05.067363 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:05.025434 28775 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:04.972709 28770 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.000696 28618 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.240166 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:05.239323 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:05.399075 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:05.043032 28771 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:05.358816 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.261422 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3126) took 1530 ms. Trace:
I20260709 13:08:05.326223 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:05.166533 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:05.345575 22001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 1358) took 1930 ms (client timeout 2557 ms). Trace:
I20260709 13:08:05.262589 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:05.345511 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.327879 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3119) took 1666 ms (client timeout 1987 ms). Trace:
W20260709 13:08:05.105976 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.160887 28662 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.954s	user 0.018s	sys 0.003s
W20260709 13:08:05.514540 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.248944 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3109) took 1594 ms (client timeout 1634 ms). Trace:
W20260709 13:08:05.155490 28713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.034s
I20260709 13:08:05.126904 28751 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:05.097862 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:05.242861 22587 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:05.198632 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:05.578771 22001 rpcz_store.cc:269] 0709 13:08:03.414671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.430695 (+ 16024us) service_pool.cc:224] Handling call
0709 13:08:05.345318 (+1914623us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540040c254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.345501 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034149632}
I20260709 13:08:05.581650 22000 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:05.581769 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:05.072204 22005 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:05.200261 28763 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:05.410960 28609 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:08:05.160002 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:05.292534 28707 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:05.344177 28656 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.003s
I20260709 13:08:05.580925 22590 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:05.280252 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:05.321967 28737 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:05.568282 22007 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
W20260709 13:08:05.640256 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 360096us
I20260709 13:08:05.640437 27861 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:05.177673 28715 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:05.513470 21935 rpcz_store.cc:276] 0709 13:08:03.730708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.744816 (+ 14108us) service_pool.cc:224] Handling call
0709 13:08:05.261147 (+1516331us) spinlock_profiling.cc:243] Waited 78.4 ms on lock 0x7b54002e9ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.261348 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180397312}
I20260709 13:08:05.157840 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:05.071034 22002 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:05.567559 21934 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:05.324188 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3108) took 1670 ms (client timeout 2013 ms). Trace:
I20260709 13:08:05.218101 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:05.272063 22004 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:05.177357 28694 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:05.578713 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 179761us
W20260709 13:08:05.576834 21929 rpcz_store.cc:269] 0709 13:08:03.661133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.683819 (+ 22686us) service_pool.cc:224] Handling call
0709 13:08:05.327667 (+1643848us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b540003be54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.327808 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307492608}
I20260709 13:08:05.583693 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:05.169263 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:05.579331 21932 rpcz_store.cc:269] 0709 13:08:03.654090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.654304 (+   214us) service_pool.cc:224] Handling call
0709 13:08:05.248763 (+1594459us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54003066d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.248877 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137265920}
W20260709 13:08:05.315482 28734 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.276s	user 0.004s	sys 0.009s
I20260709 13:08:05.570393 28742 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.129462 21853 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:08:05.225800 28677 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:05.110199 28533 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.519s
I20260709 13:08:05.635715 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:05.321269 28661 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.321s
I20260709 13:08:05.321352 28583 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:05.630576 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:05.155323 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:05.056021 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.539s (SENT)
W20260709 13:08:05.324826 21900 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:33888 dropped due to backpressure: service queue is full with 50 items
I20260709 13:08:05.641281 27861 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:05.643795 21851 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:08:05.080729 28767 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:05.641240 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:05.274827 28650 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
W20260709 13:08:05.580343 28713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.012s	sys 0.052s
W20260709 13:08:05.649027 21931 rpcz_store.cc:269] 0709 13:08:03.653755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.667330 (+ 13575us) service_pool.cc:224] Handling call
0709 13:08:05.323985 (+1656655us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b540017e654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.324130 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297511168}
I20260709 13:08:05.658281 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:05.567674 28647 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:05.659765 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 441755us
W20260709 13:08:05.660043 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 500091us
I20260709 13:08:05.662889 22004 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:08:05.678371 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.663231 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:05.384344 28712 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.264s
I20260709 13:08:05.686329 22004 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:05.585068 28616 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:05.605476 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:05.666711 22587 raft_consensus.cc:2410] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:08:05.210638 28405 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.461s
I20260709 13:08:05.665587 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.670373 22008 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:05.669961 22001 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:05.671751 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:05.691202 21929 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:05.670017 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.645351 28370 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.230695 28649 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.436s
I20260709 13:08:05.126935 28768 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:05.670516 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 614489us
W20260709 13:08:05.744503 22556 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.562s (SENT)
W20260709 13:08:05.745790 22556 leader_election.cc:336] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.597s (SENT)
I20260709 13:08:05.746306 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:05.674831 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:05.691624 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:05.671150 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:05.678987 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.644917 28707 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.388536 28782 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:05.752516 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3029) took 1615 ms. Trace:
I20260709 13:08:05.683192 28483 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:08:05.691874 28790 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:08:05.263692 28738 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.690652 21935 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:08:05.691659 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2595) took 1437 ms (client timeout 1857 ms). Trace:
I20260709 13:08:05.588020 28662 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.693809 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3144) took 1061 ms. Trace:
I20260709 13:08:05.805397 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3123) took 2136 ms. Trace:
I20260709 13:08:05.809399 21932 rpcz_store.cc:276] 0709 13:08:04.632147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.657549 (+1025402us) service_pool.cc:224] Handling call
0709 13:08:05.693745 (+ 36196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.671511 28613 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:05.246834 28690 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.669791 28583 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:05.666529 28742 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.812446 28583 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.813570 28742 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:05.813767 28583 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:05.759966 28782 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.816509 28782 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:05.817648 28803 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:08:05.819188 28613 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:05.820514 28613 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.809538 21928 rpcz_store.cc:276] 0709 13:08:03.668637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.717750 (+ 49113us) service_pool.cc:224] Handling call
0709 13:08:03.718084 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:03.718373 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:03.718397 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:03.718864 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:08:03.862255 (+143391us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:03.862288 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.862349 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:05.314777 (+1452428us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:05.780599 (+465822us) raft_consensus.cc:1685] finished
0709 13:08:05.780627 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:05.805306 (+ 24679us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16860,"raft.queue_time_us":11229,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":7477,"spinlock_wait_cycles":46734080,"thread_start_us":143041,"threads_started":1,"wal-append.queue_time_us":239386}
I20260709 13:08:05.821913 28613 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:05.669479 28785 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:05.825335 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 247064us
I20260709 13:08:05.685304 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:05.754789 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80031.1us
I20260709 13:08:05.746913 28609 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:05.753053 22006 rpcz_store.cc:276] 0709 13:08:04.136729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.136886 (+   157us) service_pool.cc:224] Handling call
0709 13:08:04.170462 (+ 33576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:04.171045 (+   583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.752235 (+1581190us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b540038aed0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.752441 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76416768}
W20260709 13:08:05.759303 21929 rpcz_store.cc:269] 0709 13:08:04.254015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.663908 (+1409893us) service_pool.cc:224] Handling call
0709 13:08:05.691600 (+ 27692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.683022 28647 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.258777 28775 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.679142 28767 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.799145 22004 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:05.785457 22008 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:08:05.780396 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1199) took 1211 ms (client timeout 1539 ms). Trace:
W20260709 13:08:05.835173 21935 rpcz_store.cc:269] 0709 13:08:04.569090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.646766 (+1077676us) service_pool.cc:224] Handling call
0709 13:08:05.780305 (+133539us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.665587 28735 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:05.785497 28483 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:05.684155 27861 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:05.826290 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 141057us
W20260709 13:08:05.690927 28652 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.558s	user 0.006s	sys 0.005s
W20260709 13:08:05.802037 28784 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:08:05.783176 28753 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.865279 21857 raft_consensus.cc:2393] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:05.750777 28370 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.672180 21900 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:33888 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3145)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1202)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2591)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2594)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1194)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2589)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2593)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1198)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2598)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2606)
I20260709 13:08:05.839308 28742 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:05.826303 21823 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.013s (SENT)
I20260709 13:08:05.879396 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:05.880560 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 54269.5us
I20260709 13:08:05.836062 21931 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:05.842809 28533 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:05.838874 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1191) took 1633 ms (client timeout 1522 ms). Trace:
W20260709 13:08:05.884272 21927 rpcz_store.cc:269] 0709 13:08:04.205430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.230927 (+ 25497us) service_pool.cc:224] Handling call
0709 13:08:05.838629 (+1607702us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.838794 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042352896}
I20260709 13:08:05.885764 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.886495 21927 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:05.887027 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1206) took 1010 ms. Trace:
I20260709 13:08:05.887391 21927 rpcz_store.cc:276] 0709 13:08:04.876871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.885427 (+1008556us) service_pool.cc:224] Handling call
0709 13:08:05.886967 (+  1540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.755481 28713 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:05.760870 28001 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:05.710537 28656 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.131s
I20260709 13:08:05.843485 28735 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.913276 28735 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:05.838917 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.914736 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:05.882690 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2592) took 1632 ms (client timeout 1915 ms). Trace:
I20260709 13:08:05.916003 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:05.916366 21931 rpcz_store.cc:269] 0709 13:08:04.250331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.675554 (+1425223us) service_pool.cc:224] Handling call
0709 13:08:05.882613 (+207059us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.917248 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.897895 28533 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.917608 28649 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:05.918032 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1204) took 1117 ms. Trace:
I20260709 13:08:05.918355 21931 rpcz_store.cc:276] 0709 13:08:04.800500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.917038 (+1116538us) service_pool.cc:224] Handling call
0709 13:08:05.917975 (+   937us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.918702 28789 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:05.919227 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.919579 28533 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:05.919863 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1205) took 1079 ms. Trace:
I20260709 13:08:05.920241 21931 rpcz_store.cc:276] 0709 13:08:04.840317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.919018 (+1078701us) service_pool.cc:224] Handling call
0709 13:08:05.919814 (+   796us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.850044 21858 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:08:05.921079 28789 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:05.921509 21855 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:08:05.922013 21855 pending_rounds.cc:85] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 12
I20260709 13:08:05.922286 21855 pending_rounds.cc:107] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:08:05.922560 21855 raft_consensus.cc:2889] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:05.917191 28677 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:08:05.923528 28677 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.761272 28751 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.923976 28677 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:05.926908 28677 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:05.927448 28677 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:05.927757 28677 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:05.878098 28775 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:05.929046 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:05.930243 21852 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 22: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:05.899577 28001 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.931246 28001 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:05.931649 28370 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:05.933598 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:05.934284 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:05.897200 28647 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:05.868788 28609 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:05.873184 28785 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.898380 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.941980 21927 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:08:05.899847 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 55830.7us
I20260709 13:08:05.903676 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:05.883756 28652 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:05.909502 28483 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.912071 28707 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:05.921286 28567 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:05.936462 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 188792us
I20260709 13:08:05.939280 21934 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:05.936731 22590 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:05.895983 21926 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:05.937407 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:05.928526 28677 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:05.943142 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1207) took 1027 ms. Trace:
I20260709 13:08:05.956372 21927 rpcz_store.cc:276] 0709 13:08:04.915893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.898129 (+982236us) service_pool.cc:224] Handling call
0709 13:08:05.943078 (+ 44949us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.957604 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.958406 21927 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:05.959210 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1202) took 1230 ms. Trace:
I20260709 13:08:05.959645 21927 rpcz_store.cc:276] 0709 13:08:04.729124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.957244 (+1228120us) service_pool.cc:224] Handling call
0709 13:08:05.959142 (+  1898us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.960989 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:05.949189 28768 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:05.961730 21927 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:05.949990 28799 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:08:05.872882 28753 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:05.954727 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2588) took 1720 ms (client timeout 1861 ms). Trace:
W20260709 13:08:05.980201 21926 rpcz_store.cc:269] 0709 13:08:04.234440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.262229 (+1027789us) service_pool.cc:224] Handling call
0709 13:08:05.954657 (+692428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.981212 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.981835 21926 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:05.982298 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1208) took 1033 ms. Trace:
I20260709 13:08:05.982718 21926 rpcz_store.cc:276] 0709 13:08:04.949411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.980923 (+1031512us) service_pool.cc:224] Handling call
0709 13:08:05.982235 (+  1312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:05.983588 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:05.984169 21926 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:08:05.984586 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1203) took 1216 ms. Trace:
I20260709 13:08:05.940536 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2601) took 1306 ms. Trace:
W20260709 13:08:05.955467 21934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2597) took 1688 ms (client timeout 1827 ms). Trace:
I20260709 13:08:05.923978 28662 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:05.964563 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2664) took 1694 ms (client timeout 1832 ms). Trace:
W20260709 13:08:05.989521 21860 rpcz_store.cc:269] 0709 13:08:04.269858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.269972 (+   114us) service_pool.cc:224] Handling call
0709 13:08:05.964335 (+1694363us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:05.964476 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569133952}
W20260709 13:08:05.978835 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 138211us
I20260709 13:08:05.985219 21931 rpcz_store.cc:276] 0709 13:08:04.634494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.920862 (+1286368us) service_pool.cc:224] Handling call
0709 13:08:05.940453 (+ 19591us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:05.985438 21934 rpcz_store.cc:269] 0709 13:08:04.267106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.325911 (+1058805us) service_pool.cc:224] Handling call
0709 13:08:05.955360 (+629449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.002780 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.003929 21931 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:08:05.827497 22001 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
W20260709 13:08:05.987787 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50508.5us
I20260709 13:08:06.004702 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.005487 21934 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:08:06.005884 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3146) took 1373 ms. Trace:
I20260709 13:08:05.946106 28583 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:08:06.007495 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 331659us
I20260709 13:08:06.006273 21934 rpcz_store.cc:276] 0709 13:08:04.632953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.004474 (+1371521us) service_pool.cc:224] Handling call
0709 13:08:06.005826 (+  1352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.036749 21859 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:06.036969 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.038206 21931 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:08:05.977941 28765 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.041702 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1210) took 1027 ms. Trace:
I20260709 13:08:06.042348 21931 rpcz_store.cc:276] 0709 13:08:05.013915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.036647 (+1022732us) service_pool.cc:224] Handling call
0709 13:08:06.038808 (+  2161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.005888 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:05.976145 28618 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:06.000391 28647 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:05.963644 28772 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:05.984978 21926 rpcz_store.cc:276] 0709 13:08:04.767788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.983353 (+1215565us) service_pool.cc:224] Handling call
0709 13:08:05.984539 (+  1186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.047009 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.048538 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.072700 21852 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:06.052829 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:05.945261 28785 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.054749 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:06.058761 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.017319 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.059290 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:06.062569 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 56717us
I20260709 13:08:06.035738 28370 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.086915 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.073418 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.074141 22001 raft_consensus.cc:2368] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:08:05.961982 28567 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.076459 21927 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:06.081553 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.082895 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.099903 22003 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:06.071820 22004 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:06.010133 28740 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:08:06.060389 28799 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:08:06.109483 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3120) took 2436 ms. Trace:
I20260709 13:08:06.111783 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3147) took 1478 ms. Trace:
I20260709 13:08:06.112288 21931 rpcz_store.cc:276] 0709 13:08:04.633211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.043200 (+1409989us) service_pool.cc:224] Handling call
0709 13:08:06.111718 (+ 68518us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:06.101179 21927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2600) took 1478 ms (client timeout 1916 ms). Trace:
I20260709 13:08:06.106216 28817 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:06.066170 28677 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:06.087847 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:06.113464 21927 rpcz_store.cc:269] 0709 13:08:04.623324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.960702 (+1337378us) service_pool.cc:224] Handling call
0709 13:08:06.101125 (+140423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.126905 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.039180 28712 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:08:06.133471 21934 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:06.130280 22008 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:08:06.134223 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.141819 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.145531 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.146763 21927 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:08:06.145593 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.148319 21931 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
W20260709 13:08:06.147222 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.110283 21930 rpcz_store.cc:276] 0709 13:08:03.661451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.701597 (+ 40146us) service_pool.cc:224] Handling call
0709 13:08:03.723473 (+ 21876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:03.723715 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:05.519645 (+1795930us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18003b0f20. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf
0709 13:08:05.570812 (+ 51167us) log.cc:844] Serialized 14 byte log entry
0709 13:08:06.087500 (+516688us) log.cc:844] Serialized 14 byte log entry
0709 13:08:06.097585 (+ 10085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:06.097660 (+    75us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:06.097732 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.097822 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.097937 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38931,"prepare.queue_time_us":171345,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":1778,"raft.queue_time_us":35462,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4921,"spinlock_wait_cycles":32841088,"thread_start_us":682106,"threads_started":3,"wal-append.queue_time_us":436127}
I20260709 13:08:06.152139 22000 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:08:06.154177 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2591) took 1908 ms. Trace:
I20260709 13:08:06.154770 21927 rpcz_store.cc:276] 0709 13:08:04.245614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.145230 (+1899616us) service_pool.cc:224] Handling call
0709 13:08:06.154094 (+  8864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5888}
I20260709 13:08:06.107563 28742 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:06.060199 28609 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.156920 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.109591 28583 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:06.068861 28618 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.160370 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:06.169576 22003 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:06.068331 28765 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.159456 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.169955 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.186287 21930 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:06.183053 28618 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.177600 21934 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:06.087455 28772 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.189494 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.193718 22003 raft_consensus.cc:2410] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:06.193898 21931 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:06.192028 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2599) took 1649 ms. Trace:
I20260709 13:08:06.151255 28811 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:06.195858 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.112247 28740 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.198215 28811 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:06.199527 28765 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:06.202342 21858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2661) took 1942 ms (client timeout 1857 ms). Trace:
W20260709 13:08:06.225682 21858 rpcz_store.cc:269] 0709 13:08:04.259535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.259684 (+   149us) service_pool.cc:224] Handling call
0709 13:08:06.202129 (+1942445us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000fc654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:06.202267 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3636471040}
I20260709 13:08:06.226889 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1325) took 1018 ms. Trace:
I20260709 13:08:06.227124 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.227440 22588 rpcz_store.cc:276] 0709 13:08:05.208184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.238471 (+ 30287us) service_pool.cc:224] Handling call
0709 13:08:05.581042 (+342571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:05.581222 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.581245 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:05.581287 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:05.581324 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.581388 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.226648 (+645260us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54005019d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:06.226816 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622149504}
W20260709 13:08:06.205147 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.146370 28637 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:08:06.194794 21934 rpcz_store.cc:276] 0709 13:08:04.542671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.085243 (+1542572us) service_pool.cc:224] Handling call
0709 13:08:06.191816 (+106573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.109995 28370 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.231907 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.232815 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.167599 28767 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.249490 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.188766 28613 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.248963 21859 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:08:06.265885 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:06.234391 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:06.274586 21933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2590) took 2033 ms (client timeout 1709 ms). Trace:
I20260709 13:08:06.193349 28583 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:06.276330 21933 rpcz_store.cc:269] 0709 13:08:04.241461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.235774 (+1994313us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:06.274499 (+ 38725us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:06.137269 28817 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.277829 22006 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:06.279939 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.286321 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1201) took 1580 ms. Trace:
I20260709 13:08:06.286998 21934 rpcz_store.cc:276] 0709 13:08:04.705747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.234069 (+1528322us) service_pool.cc:224] Handling call
0709 13:08:06.286222 (+ 52153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.289685 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2675) took 1282 ms. Trace:
I20260709 13:08:06.290764 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:06.291059 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 125498us
I20260709 13:08:06.290185 21854 rpcz_store.cc:276] 0709 13:08:05.007388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.011542 (+  4154us) service_pool.cc:224] Handling call
0709 13:08:05.011973 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:05.022369 (+ 10396us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:05.022836 (+   467us) log.cc:844] Serialized 14 byte log entry
0709 13:08:05.346754 (+323918us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:05.384603 (+ 37849us) log.cc:844] Serialized 26 byte log entry
0709 13:08:05.384777 (+   174us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:05.384816 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.384889 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:05.518309 (+133420us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b1800174160. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:06.279084 (+760775us) raft_consensus.cc:1685] finished
0709 13:08:06.289374 (+ 10290us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.289595 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113210,"prepare.queue_time_us":2163,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":14176,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":29701,"spinlock_wait_cycles":9651456,"thread_start_us":189751,"threads_started":1,"wal-append.queue_time_us":602486}
I20260709 13:08:06.288012 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1215) took 1023 ms. Trace:
I20260709 13:08:06.295202 21930 rpcz_store.cc:276] 0709 13:08:05.264406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.225788 (+961382us) service_pool.cc:224] Handling call
0709 13:08:06.287918 (+ 62130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.311221 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.311257 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.313588 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.311933 21933 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:06.314832 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1216) took 1050 ms. Trace:
W20260709 13:08:06.302443 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 215575us
I20260709 13:08:06.315281 21933 rpcz_store.cc:276] 0709 13:08:05.264806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.284413 (+1019607us) service_pool.cc:224] Handling call
0709 13:08:06.314771 (+ 30358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.319686 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.297318 28742 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.310236 21931 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 13:08:06.323901 21931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2596) took 2066 ms (client timeout 2655 ms). Trace:
W20260709 13:08:06.325419 21931 rpcz_store.cc:269] 0709 13:08:04.257805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.165897 (+1908092us) service_pool.cc:224] Handling call
0709 13:08:06.323756 (+157859us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.326112 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.352377 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.354532 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.356837 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.357664 21934 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:06.283895 28613 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.361092 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.332082 28712 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.361932 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1219) took 1096 ms. Trace:
I20260709 13:08:06.351640 28649 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:06.363921 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.363474 21859 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:06.366385 28712 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.364884 21934 rpcz_store.cc:276] 0709 13:08:05.265689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.329343 (+1063654us) service_pool.cc:224] Handling call
0709 13:08:06.361870 (+ 32527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.355680 28741 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:06.376645 21854 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:06.370154 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3191) took 1099 ms. Trace:
I20260709 13:08:06.368986 28825 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:06.374871 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.377708 22009 rpcz_store.cc:276] 0709 13:08:05.270853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.660764 (+389911us) service_pool.cc:224] Handling call
0709 13:08:05.661373 (+   609us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:05.661893 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.661920 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:05.674119 (+ 12199us) log.cc:844] Serialized 590 byte log entry
0709 13:08:06.083240 (+409121us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:06.083283 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.083362 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.369884 (+286522us) raft_consensus.cc:1685] finished
0709 13:08:06.369906 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.370085 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2215,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3444,"thread_start_us":336985,"threads_started":1,"wal-append.queue_time_us":637575}
I20260709 13:08:06.358807 28753 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.380854 28809 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.389714 28809 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.390729 28001 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.392132 28809 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.393210 28370 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.384753 28827 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:06.394152 28613 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.393993 28827 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:06.395104 28817 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.395766 28785 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.395387 28827 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.396272 28712 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.396829 28583 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:06.397269 28583 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:06.373526 28811 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.397655 28583 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:06.399482 28795 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.821s	user 0.008s	sys 0.000s
I20260709 13:08:06.400496 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.370990 28649 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.401434 28649 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:06.401475 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1213) took 1192 ms. Trace:
I20260709 13:08:06.401929 21934 rpcz_store.cc:276] 0709 13:08:05.209108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.373909 (+1164801us) service_pool.cc:224] Handling call
0709 13:08:06.401393 (+ 27484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.402858 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.406658 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:06.382575 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.412369 21859 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.369916 28824 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:08:06.388963 28737 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.733s	user 0.003s	sys 0.008s
I20260709 13:08:06.380173 21931 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 13.
I20260709 13:08:06.386653 28740 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.413950 21934 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:08:06.420720 21854 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:06.420982 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1209) took 1420 ms. Trace:
I20260709 13:08:06.414736 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.421339 21934 rpcz_store.cc:276] 0709 13:08:05.001033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.402605 (+1401572us) service_pool.cc:224] Handling call
0709 13:08:06.420927 (+ 18322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.375851 28741 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:06.422672 22586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1318) took 1790 ms. Trace:
I20260709 13:08:06.422780 28741 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:06.411863 28736 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:06.430413 28736 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:06.433193 21858 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:08:06.423059 22586 rpcz_store.cc:276] 0709 13:08:04.631995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.632239 (+   244us) service_pool.cc:224] Handling call
0709 13:08:04.662902 (+ 30663us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:04.663325 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:04.663362 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:04.664229 (+   867us) log.cc:844] Serialized 52 byte log entry
0709 13:08:05.086214 (+421985us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:05.086277 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.086383 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.398880 (+1312497us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:06.418555 (+ 19675us) raft_consensus.cc:1685] finished
0709 13:08:06.418597 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.422605 (+  4008us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52826,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":15669,"spinlock_wait_cycles":12311808,"thread_start_us":373462,"threads_started":1,"wal-append.queue_time_us":800109}
I20260709 13:08:06.392664 28001 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.419525 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1217) took 1154 ms. Trace:
I20260709 13:08:06.443181 21931 rpcz_store.cc:276] 0709 13:08:05.265112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.353771 (+1088659us) service_pool.cc:224] Handling call
0709 13:08:06.419459 (+ 65688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.444552 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:06.444365 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.447563 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.459473 21858 raft_consensus.cc:2393] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:06.448642 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:06.431519 28736 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.452620 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:06.485740 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.505936 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.507138 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2605) took 1762 ms. Trace:
I20260709 13:08:06.507661 21934 rpcz_store.cc:276] 0709 13:08:04.744578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.445769 (+1701191us) service_pool.cc:224] Handling call
0709 13:08:06.507054 (+ 61285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.509526 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.510416 21931 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
W20260709 13:08:06.527984 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 75442.9us
I20260709 13:08:06.451928 28809 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.450502 28712 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.452157 28618 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:06.452099 28765 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:06.545998 21859 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:06.544916 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:06.419800 28613 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:08:06.452844 28826 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:06.567464 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 203615us
I20260709 13:08:06.566716 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:06.555331 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.568576 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.572530 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.573573 21933 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:06.577648 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.578281 21859 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.576659 21858 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:06.539688 28830 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:06.559342 28736 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.581604 21858 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:06.584009 21858 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:08:06.564661 28765 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:06.589588 21858 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:06.585600 28609 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:06.595566 21858 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:08:06.596956 28735 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.002s
I20260709 13:08:06.597414 28735 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.599094 21858 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:06.599583 28735 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.604540 21858 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: [20.13-22.14]
I20260709 13:08:06.576594 28827 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:06.575088 28826 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.608584 21858 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:06.609684 28826 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.577160 28829 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:06.611721 21858 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:08:06.612522 28738 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:06.617801 21935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2604) took 1907 ms (client timeout 1683 ms). Trace:
W20260709 13:08:06.618275 21935 rpcz_store.cc:269] 0709 13:08:04.710884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.617527 (+1906643us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:06.617698 (+   171us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:06.618059 21858 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:08:06.619465 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.622244 28763 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:06.622802 28763 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:06.623226 28763 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:06.563984 28712 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.559988 28809 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.635874 28698 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.590152 28765 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:06.639328 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.640096 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.640452 28775 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.711s	user 0.012s	sys 0.000s
I20260709 13:08:06.641216 28613 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:06.642235 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:06.642620 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.642784 22590 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:08:06.644553 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1200) took 1976 ms (client timeout 1562 ms). Trace:
W20260709 13:08:06.645084 21928 rpcz_store.cc:269] 0709 13:08:04.667739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.644380 (+1976641us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:06.644463 (+    83us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:06.646150 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.646338 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:06.642237 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.648662 21930 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:06.649510 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.650314 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2594) took 2396 ms. Trace:
I20260709 13:08:06.650794 21930 rpcz_store.cc:276] 0709 13:08:04.253777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.649257 (+2395480us) service_pool.cc:224] Handling call
0709 13:08:06.650239 (+   982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.652026 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.652221 28770 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.652734 28770 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.653537 28777 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:06.654299 28754 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.654601 28698 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.654793 28754 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.655314 28662 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:06.655704 28662 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:06.656756 28348 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:06.657859 28798 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.984s	user 0.000s	sys 0.001s
I20260709 13:08:06.657370 28348 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.656529 28662 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.659518 28662 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.654050 28777 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.667057 21935 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:06.669201 28616 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.672322 28616 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:06.673609 28794 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.169s	user 0.019s	sys 0.006s
I20260709 13:08:06.674964 28616 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.687244 28376 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:06.687758 28376 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:06.690843 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.692515 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.693048 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2660) took 2437 ms. Trace:
I20260709 13:08:06.693243 21856 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 21. Current term is 22. Ops: [19.10-21.11]
I20260709 13:08:06.693447 21857 rpcz_store.cc:276] 0709 13:08:04.256086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.264408 (+  8322us) service_pool.cc:224] Handling call
0709 13:08:06.692813 (+2428405us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400017f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:06.692965 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3662871808}
I20260709 13:08:06.670693 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.698139 21934 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:06.699134 28805 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.616s	user 0.001s	sys 0.001s
I20260709 13:08:06.699430 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.671655 21931 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:06.701288 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.701771 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1193) took 2482 ms. Trace:
I20260709 13:08:06.702311 21931 rpcz_store.cc:276] 0709 13:08:04.219622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.671103 (+2451481us) service_pool.cc:224] Handling call
0709 13:08:06.671447 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.701457 (+ 30010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.701706 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.711149 21927 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:08:06.714885 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45894.6us
I20260709 13:08:06.717800 22000 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:08:06.718197 21932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2603) took 1978 ms (client timeout 1708 ms). Trace:
W20260709 13:08:06.718807 21932 rpcz_store.cc:269] 0709 13:08:04.667538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.644353 (+1976815us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:06.644463 (+   110us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:06.720700 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.722261 22000 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:08:06.730701 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.731616 22000 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
W20260709 13:08:06.738854 21823 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.634s (SENT)
I20260709 13:08:06.739503 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.740087 21859 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:06.741057 21858 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:06.591536 28811 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.641812 28613 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:06.750739 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 302186us
I20260709 13:08:06.753108 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:06.753527 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:06.755939 28649 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.756810 28613 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.757411 28613 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.757905 28809 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.758348 28712 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.759235 28770 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.760035 28839 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:06.757855 28613 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.761106 28754 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.760686 28839 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.762025 28777 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.729156 28750 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.762482 28613 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.763010 28613 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:06.762781 28750 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.763401 21853 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:06.763477 28839 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.764346 28789 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:06.755156 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:06.767874 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.782384 21927 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 17. Current term is 19. Ops: [17.16-17.16]
I20260709 13:08:06.783900 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1196) took 2552 ms. Trace:
I20260709 13:08:06.764750 28789 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:06.743410 28652 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:06.804811 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.784394 21927 rpcz_store.cc:276] 0709 13:08:04.231695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.745531 (+2513836us) service_pool.cc:224] Handling call
0709 13:08:06.781849 (+ 36318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.783527 (+  1678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.783823 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.837342 28775 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.837634 28789 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.837915 28775 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.838559 28649 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:06.817854 28707 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.839880 28707 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.840341 28775 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:06.842507 28707 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:06.844197 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3145) took 2211 ms (client timeout 1634 ms). Trace:
I20260709 13:08:06.844782 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.845613 21857 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:06.844723 21929 rpcz_store.cc:269] 0709 13:08:04.632618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.843971 (+2211353us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:06.844120 (+   149us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:06.850248 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.688838 28376 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:06.851814 28376 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:06.852774 28725 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.853997 28725 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.857764 21857 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
W20260709 13:08:06.858742 21851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2662) took 2589 ms (client timeout 2655 ms). Trace:
I20260709 13:08:06.856688 28725 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:06.859109 21851 rpcz_store.cc:269] 0709 13:08:04.269291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.269490 (+   199us) service_pool.cc:224] Handling call
0709 13:08:06.858544 (+2589054us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540010c054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:06.858690 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047639296}
I20260709 13:08:06.864604 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:06.867432 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.869683 21857 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:06.869661 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:06.870144 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:06.870675 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.871310 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.871402 21858 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:06.871817 21851 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:08:06.871327 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.872960 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.873565 21851 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:08:06.877071 28348 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.877398 28690 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:06.879307 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.879981 21929 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:06.880824 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.882261 21927 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:06.882197 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.883517 21935 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:06.883530 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2586) took 2651 ms. Trace:
I20260709 13:08:06.884284 21927 rpcz_store.cc:276] 0709 13:08:04.232323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.836696 (+2604373us) service_pool.cc:224] Handling call
0709 13:08:06.882093 (+ 45397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.883321 (+  1228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.883481 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.885774 28842 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:06.886245 28842 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:06.886607 28842 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:06.819249 28839 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.882082 21929 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:06.890134 21858 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:06.903370 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:06.904098 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.904565 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.906025 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:06.914104 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 229773us
I20260709 13:08:06.912803 21857 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:06.916636 21899 leader_election.cc:400] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:08:06.904021 21935 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:06.919641 21899 leader_election.cc:403] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:08:06.920303 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.927723 21899 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.932139 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.937291 21929 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:06.879350 28618 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:06.881559 28212 raft_consensus.cc:2764] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:08:06.888947 28750 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:06.764746 28613 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.752928 28811 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.950721 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:06.954881 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.956259 21927 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:06.927050 28725 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:06.938722 28616 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:06.955721 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:06.958938 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.960211 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.968518 21823 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.972177 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:06.960158 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 186565us
I20260709 13:08:06.963395 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:06.973014 21935 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:06.954353 28712 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:06.897258 28707 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:08:06.984670 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:06.981534 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.982686 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.986342 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.987427 21932 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:08:06.987910 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:06.988703 21927 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:06.913463 28839 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:06.987243 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:06.964498 28750 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:06.993547 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.994138 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:06.994179 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.995621 21927 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:06.995963 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:06.982422 28652 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:06.997150 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:06.990979 28583 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.005s
I20260709 13:08:06.999163 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.004371 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:07.007907 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:07.011901 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.013636 21930 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:08:06.977360 28751 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:06.997056 22008 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:07.005350 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2589) took 2764 ms. Trace:
I20260709 13:08:07.018078 21927 rpcz_store.cc:276] 0709 13:08:04.241174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.995026 (+2753852us) service_pool.cc:224] Handling call
0709 13:08:06.995292 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.005096 (+  9804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.005298 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.019335 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.002780 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:07.002729 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:06.990128 28533 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.008560 22584 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:06.999023 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:06.972940 28618 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:07.014446 28001 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:07.007544 28809 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.003429 28775 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:07.003609 28715 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:06.987353 28742 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:07.016078 22005 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:08:07.000195 28387 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:07.016440 22593 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:08:07.006352 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:07.008518 28838 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:08:07.025163 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.026894 28001 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:07.000155 28754 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.001614 28750 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:07.003906 28656 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:07.102787 21930 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:07.103559 28001 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:07.104249 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.011229 28779 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.100975 28829 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.011837 28652 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:07.098546 28775 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:07.108381 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.108636 28533 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:07.109859 22004 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:07.107699 21929 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:07.112432 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.113165 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.114063 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:07.116601 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:07.086894 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.117617 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.034170 21933 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
W20260709 13:08:07.118386 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.013s	sys 0.009s
I20260709 13:08:07.128989 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1198) took 2820 ms. Trace:
I20260709 13:08:07.024042 28729 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.131829 22009 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:07.132660 22006 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
W20260709 13:08:07.134153 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.135912 22589 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:07.140465 21934 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:07.141217 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2610) took 1160 ms. Trace:
I20260709 13:08:07.140719 21930 rpcz_store.cc:276] 0709 13:08:04.308779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.022693 (+2713914us) service_pool.cc:224] Handling call
0709 13:08:07.102515 (+ 79822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.128733 (+ 26218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.128936 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:07.140857 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.013s	sys 0.009s
I20260709 13:08:07.175896 21933 rpcz_store.cc:276] 0709 13:08:05.980534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.559773 (+579239us) service_pool.cc:224] Handling call
0709 13:08:07.141134 (+581361us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:07.177059 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 610338us
I20260709 13:08:07.178165 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:07.182695 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:07.183513 22004 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:07.223169 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:07.229745 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:07.230741 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 116768us
W20260709 13:08:07.231264 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53215.8us
I20260709 13:08:07.145988 28829 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.106419 28387 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.021541 28725 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.156s
I20260709 13:08:07.025825 28567 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:07.274482 22005 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:07.277066 21930 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:07.183504 28708 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:07.125772 28750 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:07.021924 28583 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.280486 21933 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:07.309963 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2602) took 2668 ms. Trace:
I20260709 13:08:07.310407 21933 rpcz_store.cc:276] 0709 13:08:04.642422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.176962 (+2534540us) service_pool.cc:224] Handling call
0709 13:08:07.280176 (+103214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.309611 (+ 29435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.309888 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.267112 28782 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:07.286441 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:07.287236 28376 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:07.288033 28850 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:07.289872 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.293990 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:07.327035 22589 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:07.328106 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.293856 28001 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.126983 28742 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:07.032676 28712 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 13:08:07.277675 28387 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:07.131677 28775 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:07.131510 28652 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:07.126164 28212 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:07.300067 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2598) took 2971 ms. Trace:
I20260709 13:08:07.341176 21930 rpcz_store.cc:276] 0709 13:08:04.328353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.176341 (+2847988us) service_pool.cc:224] Handling call
0709 13:08:07.276713 (+100372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.290966 (+ 14253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.299992 (+  9026us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:07.342314 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48410.4us
I20260709 13:08:07.313580 22004 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:07.315944 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3201) took 1217 ms. Trace:
W20260709 13:08:07.318470 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.100302 28618 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.267433 28830 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:07.323143 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:07.350728 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.353524 28618 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:07.354277 28618 raft_consensus.cc:2764] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:08:07.354859 28618 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:08:07.357440 28843 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:08:07.358843 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3003) took 1010 ms. Trace:
I20260709 13:08:07.279927 28809 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:08:07.359239 21854 rpcz_store.cc:276] 0709 13:08:06.348986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.421540 (+ 72554us) service_pool.cc:224] Handling call
0709 13:08:06.452586 (+ 31046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.453131 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.453170 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.522087 (+ 68917us) log.cc:844] Serialized 26 byte log entry
0709 13:08:06.896330 (+374243us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:07.324450 (+428120us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.327277 (+  2827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.327359 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:07.358631 (+ 31272us) raft_consensus.cc:1685] finished
0709 13:08:07.358650 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.358786 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18183,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":27348,"thread_start_us":705663,"threads_started":2,"wal-append.queue_time_us":442565}
I20260709 13:08:07.360869 28742 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:07.323774 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.328812 22589 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:07.303449 28770 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:07.369472 28770 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:07.309101 28845 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:08:07.120544 28715 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:07.373319 28698 consensus_meta.cc:287] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.718s	user 0.002s	sys 0.009s
I20260709 13:08:07.373783 28849 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:07.374270 28849 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:07.374641 28712 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:07.375028 28712 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:07.375277 28715 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:07.376214 22007 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:07.375949 28712 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.337375 21933 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:07.377252 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2606) took 2053 ms. Trace:
I20260709 13:08:07.377322 28811 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:07.377628 21933 rpcz_store.cc:276] 0709 13:08:05.323785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.336721 (+2012936us) service_pool.cc:224] Handling call
0709 13:08:07.337124 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.376958 (+ 39834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.377195 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.378374 28712 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:07.379088 21933 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:07.379657 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2611) took 1368 ms. Trace:
I20260709 13:08:07.379938 21933 rpcz_store.cc:276] 0709 13:08:06.011473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.378490 (+1367017us) service_pool.cc:224] Handling call
0709 13:08:07.378801 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.379414 (+   613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.379610 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.393905 28839 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:07.128072 28754 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.397415 28829 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:07.398167 28816 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.398854 28816 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.400419 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:07.359288 28741 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.936s	user 0.013s	sys 0.000s
I20260709 13:08:07.332835 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.413180 22005 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:07.344700 22001 rpcz_store.cc:276] 0709 13:08:06.099001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.100965 (+  1964us) service_pool.cc:224] Handling call
0709 13:08:06.108649 (+  7684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.135860 (+ 27211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.135906 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.149578 (+ 13672us) log.cc:844] Serialized 26 byte log entry
0709 13:08:06.451753 (+302175us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:06.451809 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.452323 (+   514us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:07.315658 (+863335us) raft_consensus.cc:1685] finished
0709 13:08:07.315689 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.315866 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26770,"raft.queue_time_us":91793,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"spinlock_wait_cycles":48384,"thread_start_us":301847,"threads_started":1,"wal-append.queue_time_us":630414}
I20260709 13:08:07.417292 22001 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:07.404944 28001 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:07.396667 28754 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:07.425491 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2608) took 1475 ms. Trace:
W20260709 13:08:07.426844 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.426122 21930 rpcz_store.cc:276] 0709 13:08:05.950287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.342233 (+1391946us) service_pool.cc:224] Handling call
0709 13:08:07.424709 (+ 82476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.424976 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.425041 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.425124 (+    83us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:07.425184 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.425266 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.425412 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:07.413753 28765 consensus_meta.cc:287] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.000s
W20260709 13:08:07.420924 28763 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.000s
I20260709 13:08:07.436532 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.441675 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:07.442782 22004 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:07.419945 28768 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.349s	user 0.012s	sys 0.049s
W20260709 13:08:07.430914 28777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.045s
I20260709 13:08:07.400614 28782 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:08:07.432353 28770 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:07.452032 28690 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.003s
I20260709 13:08:07.474189 21930 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:07.403566 28212 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.520483 22005 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:08:07.452397 28768 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.397s	user 0.012s	sys 0.049s
I20260709 13:08:07.436213 28613 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:07.560218 21926 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
W20260709 13:08:07.565711 22585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.010s
I20260709 13:08:07.428607 28741 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:07.566186 22585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.011s	sys 0.010s
W20260709 13:08:07.566007 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 300523us
W20260709 13:08:07.567272 21932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.013s	sys 0.013s
W20260709 13:08:07.567651 21932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.013s	sys 0.013s
W20260709 13:08:07.456892 28777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.046s
I20260709 13:08:07.568531 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:07.558061 28795 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:08:07.579675 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:07.580037 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:07.573136 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1197) took 3322 ms. Trace:
I20260709 13:08:07.567847 28690 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:08:07.375295 28849 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.561837 28807 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.451746 28765 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.454452 28763 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.588667 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:07.586493 21932 rpcz_store.cc:276] 0709 13:08:04.250238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.992807 (+2742569us) service_pool.cc:224] Handling call
0709 13:08:07.013163 (+ 20356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.013424 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:07.027488 (+ 14064us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.572656 (+545168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.572730 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:07.572799 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.572889 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.573065 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18297,"prepare.queue_time_us":14258,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":4203,"thread_start_us":540210,"threads_started":1}
I20260709 13:08:07.590448 21859 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:07.570322 28842 consensus_meta.cc:287] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.003s
I20260709 13:08:07.615756 21930 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:07.631466 28782 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:07.632458 28782 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:07.568280 28715 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:08:07.633926 28712 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:07.634424 28712 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.634845 28712 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:07.634867 28715 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:07.635618 28715 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:07.638695 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.033s	sys 0.023s
W20260709 13:08:07.639163 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.033s	sys 0.023s
W20260709 13:08:07.639531 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 654885us
I20260709 13:08:07.643325 22587 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:07.646981 22591 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:07.649286 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:07.650082 21932 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:07.678968 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:07.645587 28808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.025s
W20260709 13:08:07.645902 28650 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.012s	sys 0.016s
W20260709 13:08:07.682446 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.684161 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.686486 22586 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:07.729956 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:07.735678 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.003s	sys 0.047s
W20260709 13:08:07.736277 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.003s	sys 0.047s
W20260709 13:08:07.736675 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 710499us
I20260709 13:08:07.639387 28772 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:07.764528 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 77183.5us
I20260709 13:08:07.765353 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:07.623360 28807 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.676710 28583 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:07.780067 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:07.763639 28650 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.013s	sys 0.017s
I20260709 13:08:07.770026 28738 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:08:07.786198 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:07.787381 28278 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.787978 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:07.788954 28278 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:07.789533 28278 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:07.790408 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:07.790699 28278 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.791441 28698 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:07.791869 28387 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:07.792759 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:07.793212 28001 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:07.791844 28698 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.795095 28698 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.653858 28613 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 13:08:07.626724 28842 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:07.763173 28808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.007s	sys 0.026s
I20260709 13:08:07.813601 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.809413 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2587) took 3575 ms. Trace:
I20260709 13:08:07.780830 22588 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:07.851706 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:07.853503 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 12.
I20260709 13:08:07.852316 21929 rpcz_store.cc:276] 0709 13:08:04.233647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.992693 (+2759046us) service_pool.cc:224] Handling call
0709 13:08:07.011879 (+ 19186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.132237 (+120358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.809153 (+676916us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b5400097ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:07.809356 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219397888}
I20260709 13:08:07.855163 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:07.856017 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87540us
W20260709 13:08:07.800745 28616 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.824s	user 0.007s	sys 0.000s
I20260709 13:08:07.863256 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:07.865727 21860 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:07.867738 21857 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:08:07.856012 21929 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:07.771474 28736 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.184s	user 0.009s	sys 0.005s
I20260709 13:08:07.869810 22586 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
W20260709 13:08:07.870551 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.007s	sys 0.019s
W20260709 13:08:07.874537 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.007s	sys 0.019s
W20260709 13:08:07.875229 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 531935us
I20260709 13:08:07.874583 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:07.636626 28715 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.876825 21857 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
W20260709 13:08:07.877444 22584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.033s	sys 0.089s
I20260709 13:08:07.879005 22590 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:07.880342 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:07.885394 21860 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:07.774106 28779 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:07.881613 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:07.886157 21855 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:08:07.802157 28741 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:07.881757 21928 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:08:07.882710 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:07.802852 28212 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:07.876035 22557 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:07.881676 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:07.888195 22584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.033s	sys 0.090s
I20260709 13:08:07.809202 28742 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:07.800487 28712 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.873311 28616 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18.
I20260709 13:08:07.767175 28696 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.108s
W20260709 13:08:07.903033 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.807958 28750 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:07.910259 28763 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:07.855075 28772 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:08:07.894923 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:07.935035 21859 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:07.914881 28712 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:07.808758 28698 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.901177 28715 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:07.947634 28851 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:08:07.948259 28698 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:07.884626 28652 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:07.954941 21974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.018s	sys 0.046s
W20260709 13:08:07.955977 21974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.018s	sys 0.046s
I20260709 13:08:07.925825 28736 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.911108 28742 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:07.970598 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:07.957463 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 368821us
I20260709 13:08:07.907312 28212 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
W20260709 13:08:07.960268 28852 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 13:08:07.949580 28698 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.976560 28765 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.021121 28613 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.972968 28736 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:07.974980 28854 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 13:08:08.033164 28866 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:08.102746 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1332) took 1108 ms. Trace:
I20260709 13:08:08.025157 28698 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.048895 28867 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:08.119467 22592 rpcz_store.cc:276] 0709 13:08:06.994518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.994643 (+   125us) service_pool.cc:224] Handling call
0709 13:08:06.994940 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.995171 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:07.401487 (+406316us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.856324 (+454837us) log.cc:844] Serialized 14 byte log entry
0709 13:08:08.101502 (+245178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.101570 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:08.101628 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.102543 (+   915us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.102684 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7935,"prepare.queue_time_us":811558,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":4308,"raft.queue_time_us":219068,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9998,"spinlock_wait_cycles":106500224,"thread_start_us":1084349,"threads_started":3,"wal-append.queue_time_us":591452}
I20260709 13:08:08.143304 28854 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:08.144021 28854 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.152319 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.153085 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:08.153468 22007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 3193) took 2879 ms (client timeout 2488 ms). Trace:
I20260709 13:08:08.153896 21857 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
W20260709 13:08:08.154130 22007 rpcz_store.cc:269] 0709 13:08:05.273730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.658303 (+384573us) service_pool.cc:224] Handling call
0709 13:08:08.153193 (+2494890us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400166f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:08.153377 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3740498816}
I20260709 13:08:08.155328 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.156129 21856 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:08.153086 21853 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:08.160763 28858 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.161695 28858 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.165439 28858 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:08.166378 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1212) took 2999 ms (client timeout 1562 ms). Trace:
W20260709 13:08:08.166931 21926 rpcz_store.cc:269] 0709 13:08:05.166793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.087597 (+920804us) service_pool.cc:224] Handling call
0709 13:08:08.166124 (+2078527us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400207954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:08.166292 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3281315072}
W20260709 13:08:08.175223 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.184974 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3059) took 1150 ms. Trace:
I20260709 13:08:08.186033 22590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1328) took 1196 ms. Trace:
I20260709 13:08:08.186553 22590 rpcz_store.cc:276] 0709 13:08:06.989279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.989452 (+   173us) service_pool.cc:224] Handling call
0709 13:08:06.989747 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.884658 (+894911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.185720 (+301062us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400595454. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:08.185952 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044402048}
I20260709 13:08:08.187218 21935 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:08.188127 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3161) took 1339 ms. Trace:
I20260709 13:08:08.190065 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1194) took 3969 ms. Trace:
I20260709 13:08:08.190570 21934 rpcz_store.cc:276] 0709 13:08:04.221136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.700622 (+2479486us) service_pool.cc:224] Handling call
0709 13:08:07.140227 (+439605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.175142 (+ 34915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.189828 (+1014686us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540003b950. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:08.189998 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010149760}
I20260709 13:08:08.192559 21934 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:08.185587 22008 rpcz_store.cc:276] 0709 13:08:07.035283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.035455 (+   172us) service_pool.cc:224] Handling call
0709 13:08:07.035863 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.266917 (+231054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.266960 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.267015 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:07.267060 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.184610 (+917550us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540045abd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:08.184675 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.184889 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530760064}
W20260709 13:08:08.200212 28868 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
W20260709 13:08:08.204304 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.209689 28830 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:08.210275 28830 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:08.211410 28830 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.216960 28830 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:08.221958 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 213216us
I20260709 13:08:08.223857 28849 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:08.224253 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:08.225224 28690 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:08.226179 28690 consensus_queue.cc:1243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:08.226796 28690 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:08.227702 21933 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:08:08.228183 21933 pending_rounds.cc:85] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 14
I20260709 13:08:08.228451 21933 pending_rounds.cc:107] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:08:08.228797 21933 raft_consensus.cc:2889] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:08.229398 28874 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:08.229900 28874 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:08.230980 28874 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.231909 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1221) took 2167 ms. Trace:
I20260709 13:08:08.232497 28874 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:08.233523 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:08.236027 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.236120 22591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1373) took 1120 ms. Trace:
I20260709 13:08:08.236624 22591 rpcz_store.cc:276] 0709 13:08:07.115843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.116123 (+   280us) service_pool.cc:224] Handling call
0709 13:08:07.116437 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.654152 (+537715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.235844 (+581692us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54005839d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:08.236053 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219664128}
I20260709 13:08:08.228547 28690 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:08:08.243443 28829 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:08.244053 28829 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:08.245164 28829 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.246524 28829 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:08.250595 28809 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.251384 28809 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:08.255445 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.256518 28707 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.257232 28707 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.260043 28707 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.261160 28725 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:08.261966 28725 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.264874 28725 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.255008 28809 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.271471 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:08.272359 22008 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:08.160944 22000 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:08.235066 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1330) took 1244 ms. Trace:
W20260709 13:08:08.229506 21974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.839s (SENT) [suppressed 31 similar messages]
I20260709 13:08:08.234550 28869 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:08.281379 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:08.274796 22008 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:08.284610 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:08.188508 21935 rpcz_store.cc:276] 0709 13:08:06.848298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.186564 (+1338266us) service_pool.cc:224] Handling call
0709 13:08:08.186984 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.187836 (+   852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.188060 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.286475 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:08.287002 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.168817 21926 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:08:08.287889 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3160) took 1439 ms. Trace:
I20260709 13:08:08.288895 21926 rpcz_store.cc:276] 0709 13:08:06.848077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.168099 (+1320022us) service_pool.cc:224] Handling call
0709 13:08:08.168526 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:08.287548 (+119022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.287825 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.290262 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.292244 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3163) took 1443 ms. Trace:
I20260709 13:08:08.293071 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.302573 21934 rpcz_store.cc:276] 0709 13:08:06.848744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.191747 (+1343003us) service_pool.cc:224] Handling call
0709 13:08:08.192124 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.192947 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.292176 (+ 99229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.304091 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.305085 21934 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:08.232434 21933 rpcz_store.cc:276] 0709 13:08:06.064317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.380620 (+1316303us) service_pool.cc:224] Handling call
0709 13:08:08.227223 (+846603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.229416 (+  2193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.231618 (+  2202us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b5400207950. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:08.231827 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1946185984}
I20260709 13:08:08.306138 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.307314 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:08.307343 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 77862.3us
I20260709 13:08:08.320834 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.326078 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.329255 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:08.337726 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.301713 21926 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 12.
I20260709 13:08:08.110569 28866 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:08.141404 28772 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:08.347864 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.378372 28772 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:08.379935 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:08.380779 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3207) took 1270 ms. Trace:
I20260709 13:08:08.138104 28867 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:08.353272 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.396034 21859 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:08.396154 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:08.354030 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.396791 22007 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:08.352582 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.381320 22001 rpcz_store.cc:276] 0709 13:08:07.110240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.418245 (+308005us) service_pool.cc:224] Handling call
0709 13:08:07.430981 (+ 12736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:07.431360 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.431399 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:07.437299 (+  5900us) log.cc:844] Serialized 26 byte log entry
0709 13:08:07.978987 (+541688us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:07.979035 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.979129 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.380471 (+401342us) raft_consensus.cc:1685] finished
0709 13:08:08.380492 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.380633 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":685,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5155,"thread_start_us":427209,"threads_started":1,"wal-append.queue_time_us":921141}
I20260709 13:08:08.348192 21935 raft_consensus.cc:2368] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21.
W20260709 13:08:08.374457 21932 consensus_meta.cc:287] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.001s
I20260709 13:08:08.376040 22003 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:08:08.379748 28772 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.344509 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:08.274780 22585 rpcz_store.cc:276] 0709 13:08:06.991035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.991191 (+   156us) service_pool.cc:224] Handling call
0709 13:08:06.991596 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:07.566801 (+575205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.566837 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:07.586671 (+ 19834us) log.cc:844] Serialized 26 byte log entry
0709 13:08:07.982141 (+395470us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:07.982200 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.982296 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.234798 (+252502us) raft_consensus.cc:1685] finished
0709 13:08:08.234825 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.234996 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":349815,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":905856,"threads_started":2,"wal-append.queue_time_us":617006}
I20260709 13:08:08.339236 21860 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:08:08.402854 21932 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:08.048067 28613 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.340091 22008 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:08.348795 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:08.342514 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.380645 21851 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:08.351917 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.353389 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:08.354858 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.356051 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:08.381978 22556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.087s	sys 0.197s
I20260709 13:08:08.403949 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:08.396905 21856 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:08.308343 28842 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.514251 21926 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:08.338939 21933 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:08.517789 22004 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:08.519134 21855 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
W20260709 13:08:08.448840 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.520193 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.523231 21935 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:08.513852 22556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.095s	sys 0.205s
I20260709 13:08:08.362371 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:08.404605 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.034s	sys 0.096s
W20260709 13:08:08.540689 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.037s	sys 0.115s
I20260709 13:08:08.540742 21934 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:08.541540 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 665539us
I20260709 13:08:08.542776 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:08.525964 28652 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.528496 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1195) took 4302 ms. Trace:
W20260709 13:08:08.358310 28865 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 13:08:08.102272 28742 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.365250 28869 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:08.351112 28650 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.537140 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.514896 28855 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:08.557029 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1327) took 1568 ms. Trace:
I20260709 13:08:08.557598 22587 rpcz_store.cc:276] 0709 13:08:06.988838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.989193 (+   355us) service_pool.cc:224] Handling call
0709 13:08:07.009172 (+ 19979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.643860 (+634688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.556696 (+912836us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54006a04d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:08.556932 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457810688}
I20260709 13:08:08.559356 22557 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:08.563110 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:08.563476 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:08.563827 22007 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:08.564280 21859 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:08:08.513259 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.083s	sys 0.146s
W20260709 13:08:08.543254 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.531596 28772 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:08.543771 21926 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:08:08.349395 28405 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:08.342903 28647 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.349529 28678 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.349690 28348 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.382373 28866 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:08.354027 28782 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:08.641299 28782 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:08.543753 21931 rpcz_store.cc:276] 0709 13:08:04.225843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.703651 (+2477808us) service_pool.cc:224] Handling call
0709 13:08:06.744130 (+ 40479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:06.744612 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:06.793042 (+ 48430us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.303970 (+510928us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:07.331453 (+ 27483us) log.cc:844] Serialized 52 byte log entry
0709 13:08:07.331648 (+   195us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:07.331696 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.331783 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.528200 (+1196417us) raft_consensus.cc:1685] finished
0709 13:08:08.528228 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.528416 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36004,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":45735,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":4212,"thread_start_us":388050,"threads_started":1,"wal-append.queue_time_us":707772}
I20260709 13:08:08.642282 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.642927 21931 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
W20260709 13:08:08.565997 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.084s	sys 0.147s
I20260709 13:08:08.575541 21934 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:08.588129 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.644088 21926 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:08.644719 21972 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:08.646317 28001 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:08.646775 28001 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:08.648263 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 82447.6us
I20260709 13:08:08.655349 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.656136 21926 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:08.554821 28652 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.600973 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:08.634419 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.637498 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:08.638175 21929 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.006s
I20260709 13:08:08.636023 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:08.660108 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.652177 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.599615 21851 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:08.660938 21935 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
W20260709 13:08:08.698794 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51301.7us
I20260709 13:08:08.700971 28001 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:08.699259 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.601320 28616 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.668665 21932 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:08.707675 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.639554 28763 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:08.704939 28001 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:08.640698 28696 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:08:08.705797 21929 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:08.704234 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.706029 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1372) took 1604 ms. Trace:
I20260709 13:08:08.653571 28858 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:08.709664 21935 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:08.692817 21926 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:08.705353 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:08.742646 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.725747 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.726758 22007 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:08:08.744131 22007 pending_rounds.cc:85] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 9
I20260709 13:08:08.744550 22007 pending_rounds.cc:107] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:08:08.744869 22007 raft_consensus.cc:2889] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:08.744899 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.708534 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.633205 28880 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:08.600947 28405 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.742553 21931 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:08.746491 21934 raft_consensus.cc:2376] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18. Re-sending same reply.
I20260709 13:08:08.732645 28001 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:08.699096 28844 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:08.747953 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.761943 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.648306 28348 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.666474 28770 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:08:08.751077 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.738988 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.754448 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.699826 28768 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:08:08.565327 28650 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.594031 28855 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.754711 28707 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.761669 21926 raft_consensus.cc:2376] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19. Re-sending same reply.
I20260709 13:08:08.753285 28736 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.706853 28652 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.792531 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.707226 28533 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:08:08.729938 22593 rpcz_store.cc:276] 0709 13:08:07.108036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.119621 (+ 11585us) service_pool.cc:224] Handling call
0709 13:08:07.185106 (+ 65485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:07.185553 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:07.185592 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:07.289283 (+103691us) log.cc:844] Serialized 26 byte log entry
0709 13:08:07.657107 (+367824us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:07.657150 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.657220 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.705771 (+1048551us) raft_consensus.cc:1685] finished
0709 13:08:08.705800 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.705959 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":168543,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":333844,"threads_started":1,"wal-append.queue_time_us":664522}
I20260709 13:08:08.763116 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.794994 21934 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:08.795938 21934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:08.762617 21859 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:08.764014 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.772344 21932 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:08.772779 21933 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:08.797936 21935 raft_consensus.cc:2376] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19. Re-sending same reply.
I20260709 13:08:08.769101 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:08.741695 28816 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:08.773854 28613 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:08.774070 28879 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:08:08.650835 28867 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:08.774820 28878 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
W20260709 13:08:08.767891 28873 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.645s	user 0.000s	sys 0.001s
I20260709 13:08:08.771551 28869 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:08.779311 28850 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.436s	user 0.012s	sys 0.001s
I20260709 13:08:08.746759 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:08.781466 28871 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
I20260709 13:08:08.740682 28854 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:08.636373 28647 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.643335 28678 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.755226 28763 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:08.796638 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:08.803021 21854 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:08.725333 28616 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.806960 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.807749 28348 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:08.788761 28770 consensus_queue.cc:1243] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:08:08.669217 28842 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 13:08:08.768373 28858 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:08.821301 21853 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:08.821741 21975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.018s	sys 0.086s
W20260709 13:08:08.871312 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 526846us
I20260709 13:08:08.825731 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:08.790493 28768 consensus_queue.cc:1243] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:08.871729 21858 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:08.843663 21929 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:08.872306 21926 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:08.872510 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3166) took 2011 ms. Trace:
I20260709 13:08:08.872709 21860 raft_consensus.cc:2376] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13. Re-sending same reply.
I20260709 13:08:08.873155 21929 rpcz_store.cc:276] 0709 13:08:06.860908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.814644 (+1953736us) service_pool.cc:224] Handling call
0709 13:08:08.843257 (+ 28613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.872217 (+ 28960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.872439 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:08.830938 28212 consensus_queue.cc:1175] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "821b3831b8a348c0826bd3a20a857f00" responder_term: 19 status { last_received { term: 18 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:08.877808 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:08.858460 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 62142.9us
I20260709 13:08:08.769408 28844 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.800953 28738 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:08:08.789582 28884 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
W20260709 13:08:08.869814 21825 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.004s	sys 0.025s
W20260709 13:08:08.871523 21975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.018s	sys 0.087s
I20260709 13:08:08.863720 28708 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:08.802979 28707 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:08.792172 28698 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:08.852360 28850 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:08.875123 21929 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:08:08.898499 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3170) took 1945 ms. Trace:
I20260709 13:08:08.898919 21929 rpcz_store.cc:276] 0709 13:08:06.953369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.874363 (+1920994us) service_pool.cc:224] Handling call
0709 13:08:08.874721 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.898213 (+ 23492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.898439 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.844158 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:08.900309 21851 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:08.833555 21932 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:08.802469 28880 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:08:08.785833 28782 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.878857 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:08.905673 21931 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:08.883929 21825 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.004s	sys 0.025s
W20260709 13:08:08.906397 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 542848us
I20260709 13:08:08.884505 21926 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:08.909983 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3167) took 2048 ms. Trace:
I20260709 13:08:08.910360 21926 rpcz_store.cc:276] 0709 13:08:06.861226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.873050 (+2011824us) service_pool.cc:224] Handling call
0709 13:08:08.884088 (+ 11038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.909718 (+ 25630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.909927 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:08.912209 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.914625 21859 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:08:08.864616 28763 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:08.851646 21933 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 13.
I20260709 13:08:08.918133 21933 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
W20260709 13:08:08.926231 21823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.004s	sys 0.035s
I20260709 13:08:08.926893 21931 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
W20260709 13:08:08.928797 21823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.004s	sys 0.035s
I20260709 13:08:08.930967 21859 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:08.938244 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 56848.7us
I20260709 13:08:08.924712 21851 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:08.939677 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:08.902783 28698 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:08.972633 21926 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:08:08.975004 21859 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:08:08.966789 21900 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:09.007577 21859 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: []
W20260709 13:08:08.961877 28628 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.020s	sys 0.024s
I20260709 13:08:08.986181 28708 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:08.919211 28855 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.003553 21926 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:08:08.980012 28782 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:09.009450 28628 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.021s	sys 0.024s
W20260709 13:08:08.924871 28777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.029s
W20260709 13:08:09.007270 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 269179us
I20260709 13:08:08.961494 28882 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:08:08.963770 28871 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:08:08.980444 28698 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:09.014562 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 47866us
I20260709 13:08:09.018055 21933 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 13:08:09.026298 21933 pending_rounds.cc:85] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 13
I20260709 13:08:09.026690 21933 pending_rounds.cc:107] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:08:09.027357 21933 raft_consensus.cc:2889] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.025815 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.033140 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.012432 28708 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:09.021672 28777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.007s	sys 0.030s
W20260709 13:08:09.036161 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.018945 28736 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:09.015689 28892 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:09.047433 21926 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:09.050616 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 330688us
I20260709 13:08:09.045223 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.055393 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.056090 28885 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:09.032663 28650 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.060668 28885 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.052767 28768 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:09.078284 28885 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:08.996121 28858 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.077497 28533 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:09.059615 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:09.042562 28855 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.080279 22556 leader_election.cc:400] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21.
I20260709 13:08:09.042318 28405 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.045497 28708 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:09.087812 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 204523us
I20260709 13:08:09.041962 28789 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:09.083657 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.047668 28647 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.050061 28736 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.042374 28884 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:09.085975 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:09.060128 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 424134us
I20260709 13:08:09.078650 28768 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:09.109423 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:09.084302 28533 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:09.113587 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.113533 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:09.133697 21825 leader_election.cc:336] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.983s (SENT)
W20260709 13:08:09.133456 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47574.6us
I20260709 13:08:09.119958 21824 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.136411 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:09.139922 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:09.143229 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:09.137773 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.145680 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.145702 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.146786 22009 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 13: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:09.147650 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.148469 22593 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:08:09.086660 28888 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:09.114750 28887 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:08:09.152375 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:58544 (request call id 0) took 2237 ms. Trace:
I20260709 13:08:09.086056 28742 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:09.138561 28533 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:09.135228 28768 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:09.092599 28855 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.132387 28736 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:09.170650 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:09.237488 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80013.1us
W20260709 13:08:09.142474 28775 consensus_meta.cc:287] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.731s	user 0.009s	sys 0.005s
I20260709 13:08:09.133104 28616 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.138049 28850 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:09.134274 28770 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:09.137475 28884 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.168807 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.229002 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.230115 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.251380 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:09.303292 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:09.230878 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.304311 22588 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:09.304512 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.231827 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.305289 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:09.156818 21933 rpcz_store.cc:276] 0709 13:08:06.905266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.033235 (+2127969us) service_pool.cc:224] Handling call
0709 13:08:09.142853 (+109618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:09.141454 28886 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.238695 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:09.238940 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:09.309931 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71069.5us
W20260709 13:08:09.239540 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 96384.6us
I20260709 13:08:09.239553 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:09.149833 28707 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.156364 28888 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:09.258069 28893 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
W20260709 13:08:09.306731 28849 consensus_meta.cc:287] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.082s	user 0.007s	sys 0.007s
W20260709 13:08:09.324654 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86047.7us
I20260709 13:08:09.325834 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:09.259853 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:09.326999 21931 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:09.261366 21932 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:09.261868 28650 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:09.273417 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:09.281440 22587 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:09.301982 28850 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.282951 28896 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:08:09.287874 28830 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.070s	user 0.007s	sys 0.002s
I20260709 13:08:09.195525 28895 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:09.288504 28901 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:08:09.289333 28902 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:09.295567 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3164) took 2446 ms. Trace:
I20260709 13:08:09.296677 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2593) took 5043 ms. Trace:
I20260709 13:08:09.342592 21926 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:08:09.249199 28405 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:09.299532 28874 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.063s	user 0.008s	sys 0.002s
I20260709 13:08:09.305842 22588 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:09.305938 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.305780 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:09.308235 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:09.288800 28715 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.335s	user 0.014s	sys 0.004s
I20260709 13:08:09.323505 28888 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:09.335796 28212 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:08:09.315575 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 201488us
W20260709 13:08:09.320274 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 206811us
I20260709 13:08:09.335956 28842 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.337733 28850 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:09.366997 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1504) took 1085 ms. Trace:
I20260709 13:08:09.250236 28897 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:09.282375 28678 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:09.332506 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 188587us
I20260709 13:08:09.240592 28892 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:09.301496 28770 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.307924 28616 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.312108 28628 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:09.386390 28909 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:08:09.387976 28628 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.387897 28616 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.312335 28884 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:09.390406 28616 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.391006 28905 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.340381 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:09.392884 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 52567.7us
I20260709 13:08:09.393616 28533 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.395710 28896 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.341491 21933 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:09.358265 28405 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.342202 21927 rpcz_store.cc:276] 0709 13:08:04.253505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.019178 (+2765673us) service_pool.cc:224] Handling call
0709 13:08:07.019622 (+   444us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:08:07.023483 (+  3861us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:07.419775 (+396292us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.942288 (+522513us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.952954 (+ 10666us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:08:07.953711 (+   757us) log.cc:844] Serialized 78 byte log entry
0709 13:08:07.953878 (+   167us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:07.954083 (+   205us) log.cc:844] Serialized 14 byte log entry
0709 13:08:08.216750 (+262667us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b180047d460. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b746a358f 00007f3b6d6e2571 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd
0709 13:08:08.217606 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.217793 (+   187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.239221 (+ 21428us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b180047d460. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1
0709 13:08:08.240931 (+  1710us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b180047d460. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:09.296380 (+1055449us) raft_consensus.cc:1685] finished
0709 13:08:09.296411 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.296595 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":533008,"prepare.queue_time_us":482213,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":158,"raft.queue_time_us":285701,"raft.run_cpu_time_us":566,"raft.run_wall_time_us":28532,"spinlock_wait_cycles":37640064,"thread_start_us":721379,"threads_started":2,"wal-append.queue_time_us":598627}
I20260709 13:08:09.347796 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.350059 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:09.396772 21825 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:09.332933 28707 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:09.248773 28775 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:09.352857 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47191.3us
I20260709 13:08:09.372730 28212 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:09.085096 28698 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.408699 28775 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:09.409936 22002 raft_consensus.cc:1240] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:09.342115 21935 rpcz_store.cc:276] 0709 13:08:06.848981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.798991 (+1950010us) service_pool.cc:224] Handling call
0709 13:08:08.799824 (+   833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:08.814079 (+ 14255us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:08.814128 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:08.846121 (+ 31993us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.094766 (+248645us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:09.094828 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.094927 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.295266 (+200339us) raft_consensus.cc:1685] finished
0709 13:08:09.295295 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.295484 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7795,"raft.queue_time_us":2586,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":225727,"threads_started":1,"wal-append.queue_time_us":438864}
I20260709 13:08:09.316672 28886 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.412277 28886 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:09.360415 28829 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.113s	user 0.014s	sys 0.000s
I20260709 13:08:09.378669 28903 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:08:09.414292 28900 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:09.415412 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3174) took 2390 ms. Trace:
I20260709 13:08:09.415897 21932 rpcz_store.cc:276] 0709 13:08:07.024532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.901414 (+1876882us) service_pool.cc:224] Handling call
0709 13:08:09.261119 (+359705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.328687 (+ 67568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.415166 (+ 86479us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400228b50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.415339 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826202880}
I20260709 13:08:09.361647 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.343441 28910 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:09.340853 28895 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:09.364161 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.365975 21930 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:08:09.366281 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.237470 28763 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.343751 28849 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:09.368432 22589 rpcz_store.cc:276] 0709 13:08:08.281279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.281613 (+   334us) service_pool.cc:224] Handling call
0709 13:08:08.282008 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:08.282436 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.282474 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:08.346716 (+ 64242us) log.cc:844] Serialized 26 byte log entry
0709 13:08:08.925435 (+578719us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:08.925491 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.925596 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.366770 (+441174us) raft_consensus.cc:1685] finished
0709 13:08:09.366792 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.366934 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29649,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":15673,"thread_start_us":392414,"threads_started":1,"wal-append.queue_time_us":639559}
I20260709 13:08:09.377456 21934 raft_consensus.cc:606] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:09.381038 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 832549us
I20260709 13:08:09.383366 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:09.378278 28387 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:08:09.388507 28628 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:09.401360 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:09.401808 21825 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:09.366425 28830 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.403206 22006 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 22: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:09.405609 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:09.337944 28866 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.697s	user 0.008s	sys 0.005s
I20260709 13:08:09.402778 28707 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.406524 28906 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:09.399062 28854 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:09.396123 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.407761 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.385254 28897 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:09.385556 28869 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:09.409045 28698 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:09.334123 28650 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.337075 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.363739 28708 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.506268 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:09.502326 21928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2613) took 3200 ms (client timeout 2338 ms). Trace:
W20260709 13:08:09.562393 21928 rpcz_store.cc:269] 0709 13:08:06.302513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.645842 (+343329us) service_pool.cc:224] Handling call
0709 13:08:09.502079 (+2856237us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002e86d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.502222 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2839382912}
I20260709 13:08:09.564404 21928 pending_rounds.cc:85] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 13
I20260709 13:08:09.565023 21928 pending_rounds.cc:107] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:08:09.565433 21928 raft_consensus.cc:2889] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.503716 21929 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
W20260709 13:08:09.566623 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 170588us
I20260709 13:08:09.567008 21929 pending_rounds.cc:85] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:08:09.484936 21932 pending_rounds.cc:85] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:08:09.567867 21932 pending_rounds.cc:107] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:08:09.567485 21929 pending_rounds.cc:107] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:08:09.568207 21932 raft_consensus.cc:2889] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.568388 21929 raft_consensus.cc:2889] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.558810 21930 pending_rounds.cc:85] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 9
I20260709 13:08:09.569244 21930 pending_rounds.cc:107] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:08:09.569588 21930 raft_consensus.cc:2889] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.560374 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:09.427186 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.503119 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3018) took 1151 ms. Trace:
I20260709 13:08:09.501897 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.503720 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:09.559108 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 153593us
I20260709 13:08:09.502779 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3028) took 1144 ms. Trace:
I20260709 13:08:09.567464 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:09.618413 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 235123us
I20260709 13:08:09.391584 28905 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.383589 28842 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.505669 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.491997 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:09.399137 28405 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:09.509434 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 140408us
I20260709 13:08:09.387070 28770 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:09.654743 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.649716 21853 rpcz_store.cc:276] 0709 13:08:08.352217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.352338 (+   121us) service_pool.cc:224] Handling call
0709 13:08:09.502988 (+1150650us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400058cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.503074 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974247808}
I20260709 13:08:09.655000 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.655870 22587 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
W20260709 13:08:09.619648 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 52309.4us
I20260709 13:08:09.613564 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.657040 22592 raft_consensus.cc:2393] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:09.621423 21854 rpcz_store.cc:276] 0709 13:08:08.358560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.556945 (+198385us) service_pool.cc:224] Handling call
0709 13:08:09.502622 (+945677us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540008acd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.502727 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309534080}
I20260709 13:08:09.408766 28212 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:09.626760 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 225037us
I20260709 13:08:09.566524 28707 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:09.649333 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.618801 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:09.618893 22590 raft_consensus.cc:2410] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:09.507632 28533 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:09.653322 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:09.666671 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47929.9us
I20260709 13:08:09.668054 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:09.649384 22007 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:08:09.653546 28754 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 13:08:09.392900 28892 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:09.500972 28883 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.667s	user 0.001s	sys 0.001s
W20260709 13:08:09.661399 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 100253us
I20260709 13:08:09.563359 28874 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:09.415900 28772 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.814s	user 0.010s	sys 0.000s
W20260709 13:08:09.456045 28911 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:08:09.654098 28909 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:09.424898 22588 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:09.504210 28750 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.562493 28650 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.619194 28715 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:09.627996 28907 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.498620 28844 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.714535 21927 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:08:09.714753 28844 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:09.722054 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48000 (request call id 30) took 1451 ms. Trace:
I20260709 13:08:09.722810 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3017) took 1372 ms. Trace:
I20260709 13:08:09.722714 21934 rpcz_store.cc:276] 0709 13:08:08.271040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.795787 (+524747us) service_pool.cc:224] Handling call
0709 13:08:09.721840 (+926053us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b540017e654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707c9853 00007f3b7579760a 00007f3b6ed1d7d4 00007f3b6ed1d762 00007f3b6ed1d691 00007f3b6ed1d60c 00007f3b6ed1c8c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.721944 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335640832}
I20260709 13:08:09.723313 21858 rpcz_store.cc:276] 0709 13:08:08.350915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.351674 (+   759us) service_pool.cc:224] Handling call
0709 13:08:09.722635 (+1370961us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b540018f6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.722755 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811423104}
I20260709 13:08:09.724399 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:09.726953 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3054) took 2730 ms. Trace:
I20260709 13:08:09.727367 22000 rpcz_store.cc:276] 0709 13:08:06.996729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.997631 (+   902us) service_pool.cc:224] Handling call
0709 13:08:08.160610 (+1162979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.274470 (+113860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.726718 (+1452248us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400166f50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.726869 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2673932032}
I20260709 13:08:09.729205 28897 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.731832 28867 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:09.742141 28905 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.724400 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:09.747730 21856 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:09.671679 21854 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:09.750358 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2709) took 1016 ms. Trace:
I20260709 13:08:09.750687 21854 rpcz_store.cc:276] 0709 13:08:08.733691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.657774 (+924083us) service_pool.cc:224] Handling call
0709 13:08:09.671494 (+ 13720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.750109 (+ 78615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.750306 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:09.673520 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.628726 28886 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:09.678805 22590 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:09.748914 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:09.752768 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.611701 28869 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.662796 28777 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:08:09.571523 28738 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:09.685820 28918 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.498946 28884 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.663796 22589 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:09.689309 22587 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:09.648561 28895 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:09.691061 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:09.506402 28768 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.694407 22586 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:08:09.695469 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:09.612084 28854 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.476117 28885 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.649809 28903 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:08:09.757076 28900 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:09.677853 28405 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.679287 28855 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:09.628394 28736 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.682061 28678 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:08:09.711189 28829 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:09.798338 28829 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:09.799276 28829 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.799712 28736 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.701480 28616 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.803138 28616 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.805289 28616 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.657938 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:09.806030 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 148178us
I20260709 13:08:09.627368 28763 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:09.807740 22591 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:09.571521 28866 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:09.709743 28782 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.000s
I20260709 13:08:09.704815 28715 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:09.754814 28779 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.865s	user 0.014s	sys 0.000s
I20260709 13:08:09.702857 28917 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:09.670501 28906 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:09.695732 28874 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.757918 28905 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:08:09.789163 28387 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:09.778302 28738 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.812121 28772 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:09.754439 28907 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.809916 28763 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:09.819876 28715 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:09.813865 28900 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:09.845844 28708 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:09.679952 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 342986us
I20260709 13:08:10.043291 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.045352 28652 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.047554 28927 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:08:09.713707 21806 auto_rebalancer.cc:893] Removing replace marker failed: Replica 821b3831b8a348c0826bd3a20a857f00 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:09.939543 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3026) took 1581 ms. Trace:
I20260709 13:08:10.050055 28750 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:09.942607 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2706) took 1453 ms. Trace:
I20260709 13:08:10.050860 28750 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:10.051301 21855 rpcz_store.cc:276] 0709 13:08:08.489451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.426143 (+936692us) service_pool.cc:224] Handling call
0709 13:08:09.487625 (+ 61482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.562203 (+ 74578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.562232 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.562685 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.941720 (+379035us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:09.941862 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.942029 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.942128 (+    99us) raft_consensus.cc:1685] finished
0709 13:08:09.942144 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.942384 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83351,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8911,"thread_start_us":278031,"threads_started":1,"wal-append.queue_time_us":251265}
I20260709 13:08:09.943337 21854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2713) took 1208 ms. Trace:
I20260709 13:08:10.053659 21854 rpcz_store.cc:276] 0709 13:08:08.734901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.941401 (+1206500us) service_pool.cc:224] Handling call
0709 13:08:09.942343 (+   942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.942808 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.942833 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:09.942912 (+    79us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:09.942988 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.943044 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.943149 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88320}
I20260709 13:08:09.945549 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2712) took 1210 ms. Trace:
I20260709 13:08:10.055047 21856 rpcz_store.cc:276] 0709 13:08:08.734621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.943880 (+1209259us) service_pool.cc:224] Handling call
0709 13:08:09.944534 (+   654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.944952 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.944978 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:09.945061 (+    83us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:09.945121 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.945199 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.945332 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:09.751065 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:10.059813 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 308617us
W20260709 13:08:09.962597 21860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2702) took 1589 ms (client timeout 1832 ms). Trace:
I20260709 13:08:09.794584 22001 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:08:09.966845 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3171) took 3013 ms. Trace:
W20260709 13:08:09.967445 21935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.002s	sys 0.007s
W20260709 13:08:10.086057 21935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.003s	sys 0.007s
I20260709 13:08:10.085841 21929 rpcz_store.cc:276] 0709 13:08:06.953665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.899630 (+1945965us) service_pool.cc:224] Handling call
0709 13:08:09.503369 (+603739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.568980 (+ 65611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.966369 (+397389us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54002e86d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.966631 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387679488}
I20260709 13:08:10.088826 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.089169 28782 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.090144 28782 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.976454 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3214) took 1704 ms. Trace:
I20260709 13:08:09.983054 21857 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:09.787218 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1382) took 2493 ms. Trace:
I20260709 13:08:10.099400 22584 rpcz_store.cc:276] 0709 13:08:07.287727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.351670 (+ 63943us) service_pool.cc:224] Handling call
0709 13:08:07.351977 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:07.895591 (+543614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.895635 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:07.896172 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:08:08.395677 (+499505us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:08.395733 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.395831 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.500390 (+1104559us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:09.767727 (+267337us) raft_consensus.cc:1685] finished
0709 13:08:09.767760 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.781313 (+ 13553us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21174,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"spinlock_wait_cycles":63361408,"thread_start_us":1042299,"threads_started":2,"wal-append.queue_time_us":891097}
I20260709 13:08:10.101315 28867 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.786813 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:09.993265 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.103017 28922 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:10.103627 28922 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:09.995887 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.105307 28820 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.106343 28820 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.107103 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:09.997247 28001 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:09.834156 28387 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:10.007647 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.109572 28387 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:10.110791 28850 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.726s	user 0.011s	sys 0.003s
I20260709 13:08:10.111438 28850 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.018688 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2705) took 1619 ms. Trace:
I20260709 13:08:10.113540 22590 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:08:10.022368 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:10.114212 28905 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:08:10.115622 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93643.5us
I20260709 13:08:10.050541 21851 rpcz_store.cc:276] 0709 13:08:08.357885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.970987 (+613102us) service_pool.cc:224] Handling call
0709 13:08:09.001716 (+ 30729us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:09.224412 (+222696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.224461 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:09.316004 (+ 91543us) log.cc:844] Serialized 52 byte log entry
0709 13:08:09.561463 (+245459us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:09.561504 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.561570 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.938281 (+376711us) raft_consensus.cc:1685] finished
0709 13:08:09.938412 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.939057 (+   645us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102073,"raft.queue_time_us":132002,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3939,"thread_start_us":420606,"threads_started":2,"wal-append.queue_time_us":329791}
I20260709 13:08:09.850564 28707 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.856557 28738 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.848016 28770 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.854360 28763 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:10.113190 21859 rpcz_store.cc:276] 0709 13:08:08.399296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.020362 (+621066us) service_pool.cc:224] Handling call
0709 13:08:09.725089 (+704727us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:09.945737 (+220648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.945781 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:09.946886 (+  1105us) log.cc:844] Serialized 52 byte log entry
0709 13:08:09.961481 (+ 14595us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:09.961556 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.961737 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.962044 (+   307us) raft_consensus.cc:1685] finished
0709 13:08:09.962068 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.018157 (+ 56089us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b540018f6d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:10.018463 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":256006,"raft.queue_time_us":302350,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":23457,"spinlock_wait_cycles":1604973824,"thread_start_us":13901,"threads_started":1,"wal-append.queue_time_us":1191}
I20260709 13:08:10.119539 28770 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:09.829439 28779 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:09.850121 28830 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:10.121469 21931 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:10.121969 28830 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:10.122407 28830 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.122624 28768 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:10.122738 28830 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.123883 21926 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:10.125515 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2646) took 1072 ms. Trace:
I20260709 13:08:09.817785 28855 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:08:10.096616 28750 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.129108 28855 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:10.129594 28855 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:10.129927 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 289682us
I20260709 13:08:10.130405 28750 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.131085 28855 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.893937 28866 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.132763 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:09.870604 28874 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.868575 28869 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.880050 28884 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.895195 28885 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:09.868125 28888 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.866173 28896 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:09.889492 28900 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:09.833822 28906 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:09.900319 28907 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.818490 28908 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Committing config change with OpId 18.12: config changed from index 2 to 12, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.153381 28886 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:10.153712 28820 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.779116 28918 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.154139 28886 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:10.155738 28886 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.156417 28918 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:09.876103 28921 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:10.157408 28921 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:10.158727 28921 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.031437 28933 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:09.782322 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93347.2us
W20260709 13:08:10.161425 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 411467us
W20260709 13:08:10.161912 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.162626 21972 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:10.164017 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:10.164209 28896 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:10.056171 28628 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.165518 28907 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:10.165164 28896 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.166271 28907 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:10.168140 28628 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.167822 28907 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:10.050127 21806 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:08:10.169196 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:08:09.815042 28618 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:08:10.138430 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:09.875687 28772 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:10.170150 28618 consensus_queue.cc:1243] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:10.170989 28618 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:10.171376 28618 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:10.171567 28772 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.097798 28678 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:10.102469 28867 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.065795 28708 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:10.077931 21860 rpcz_store.cc:269] 0709 13:08:08.373514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.403295 (+ 29781us) service_pool.cc:224] Handling call
0709 13:08:09.961684 (+1558389us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:09.962337 (+   653us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825401088}
W20260709 13:08:10.145596 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 57087.5us
W20260709 13:08:09.850620 28789 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.000s
I20260709 13:08:10.186254 21852 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:10.187042 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3153) took 3340 ms. Trace:
I20260709 13:08:10.187441 21930 rpcz_store.cc:276] 0709 13:08:06.846395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.626922 (+780527us) service_pool.cc:224] Handling call
0709 13:08:07.648586 (+ 21664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.570241 (+1921655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.186535 (+616294us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540017eb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:10.186839 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3948715264}
I20260709 13:08:09.891597 28854 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.092029 22002 rpcz_store.cc:276] 0709 13:08:08.272309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.272412 (+   103us) service_pool.cc:224] Handling call
0709 13:08:08.272690 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.410476 (+1137786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.975851 (+565375us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002dafd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:09.976234 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2614742272}
I20260709 13:08:10.127072 28001 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:10.177516 28708 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.194512 28707 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.194921 28708 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.194782 28001 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.196209 28001 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:10.196429 28741 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.123029 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2645) took 1070 ms. Trace:
I20260709 13:08:10.197024 28741 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.197695 21931 rpcz_store.cc:276] 0709 13:08:09.052501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.327853 (+275352us) service_pool.cc:224] Handling call
0709 13:08:09.328264 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.122049 (+793785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.122732 (+   683us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b5400229a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:10.122959 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1823067008}
I20260709 13:08:10.199736 28850 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:10.126154 21926 rpcz_store.cc:276] 0709 13:08:09.052831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.346751 (+293920us) service_pool.cc:224] Handling call
0709 13:08:09.365336 (+ 18585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.124273 (+758937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.125064 (+   791us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b5400429fd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:10.125428 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1742810368}
I20260709 13:08:10.112740 28652 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:10.200991 28652 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.201437 28652 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:10.202121 28882 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:10.202634 28882 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.139811 22593 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:08:10.205250 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 562760us
I20260709 13:08:10.140753 28888 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.125435 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.207275 22586 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:10.140712 28869 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:10.103080 22589 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:10.137396 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.137815 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:10.137204 28922 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.147459 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:09.825759 28917 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.212299 28917 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.168371 28866 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:10.140030 28807 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:09.808096 28405 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.218712 28895 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.219823 28405 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.115371 28905 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.220314 22585 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 23: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:10.221100 28741 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.222355 28905 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:10.222631 28212 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.545s	user 0.015s	sys 0.006s
I20260709 13:08:10.223140 28212 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.223865 22590 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:10.224609 22588 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 15: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:10.225221 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:10.225457 28922 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:10.152581 28906 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.160744 28921 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.139990 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.172674 28618 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.173491 28678 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:10.188103 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2686) took 3338 ms (client timeout 1907 ms). Trace:
I20260709 13:08:10.170145 28842 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.208326 28869 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:10.224646 22584 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:10.184516 28867 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.209721 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:10.239049 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58082.6us
W20260709 13:08:10.239027 21852 rpcz_store.cc:269] 0709 13:08:06.849668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.849915 (+   247us) service_pool.cc:224] Handling call
0709 13:08:10.187865 (+3337950us) spinlock_profiling.cc:243] Waited 3.33 s on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:10.188007 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7669867264}
W20260709 13:08:10.246872 28698 consensus_meta.cc:287] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.002s
W20260709 13:08:10.248736 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 106838us
I20260709 13:08:10.249204 28678 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.249797 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.250128 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2707) took 1718 ms. Trace:
I20260709 13:08:10.245110 28869 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:10.250613 21853 rpcz_store.cc:276] 0709 13:08:08.515770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.663244 (+1147474us) service_pool.cc:224] Handling call
0709 13:08:09.690099 (+ 26855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.690470 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.690489 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.696276 (+  5787us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.985591 (+289315us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b18004776a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:08:10.142264 (+156673us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:10.144933 (+  2669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.180526 (+ 35593us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.180601 (+    75us) raft_consensus.cc:1685] finished
0709 13:08:10.180619 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.233936 (+ 53317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24657,"raft.queue_time_us":3377,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":10767,"spinlock_wait_cycles":74032000,"thread_start_us":92866,"threads_started":1,"wal-append.queue_time_us":375823}
W20260709 13:08:10.253778 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 114012us
W20260709 13:08:10.258051 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66207us
I20260709 13:08:10.258987 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.258488 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1392) took 1024 ms. Trace:
I20260709 13:08:10.162647 28933 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.263795 22585 rpcz_store.cc:276] 0709 13:08:09.234004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.366039 (+132035us) service_pool.cc:224] Handling call
0709 13:08:10.247670 (+881631us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b5400684f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:10.258430 (+ 10760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514374528}
I20260709 13:08:10.255172 28855 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:10.256642 28212 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.260471 28807 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:10.256168 28888 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:08:10.280376 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:10.280478 28933 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.282228 28874 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.270057 28906 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:10.284636 28650 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.006s
I20260709 13:08:10.290805 28618 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:10.291919 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.292810 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:10.302721 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 138861us
I20260709 13:08:10.271605 28895 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:10.302191 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 137231us
I20260709 13:08:10.303890 28905 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:10.304283 21749 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 16, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 16 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.304316 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:10.306384 28405 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:10.306855 28896 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.309154 28874 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.342756 28650 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.365530 28807 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:10.366365 28807 consensus_queue.cc:1243] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:10.288765 28754 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.078s
W20260709 13:08:10.310104 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46518.3us
I20260709 13:08:10.347400 22590 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:10.377395 28777 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:10.378021 28405 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:10.378386 28777 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.380465 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.260469 28707 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 13:08:10.342077 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67516.2us
I20260709 13:08:10.307421 28937 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:10.353149 28844 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:10.346805 28918 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:10.357779 28698 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:10.300048 28906 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:10.358790 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 100161us
I20260709 13:08:10.361491 28926 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:10.341058 28905 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.398316 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:10.407639 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:10.408017 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:10.408253 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 50674.2us
I20260709 13:08:10.400432 28905 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:10.410567 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:10.411360 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 131098us
I20260709 13:08:10.412529 21974 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:10.397635 28906 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:10.413753 21858 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:10.359606 21749 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 12, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 12 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.415130 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2710) took 1681 ms. Trace:
I20260709 13:08:10.415486 21858 rpcz_store.cc:276] 0709 13:08:08.734000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.941174 (+1207174us) service_pool.cc:224] Handling call
0709 13:08:09.942509 (+  1335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.414261 (+471752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.414871 (+   610us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b540010c554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:10.415053 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011729152}
I20260709 13:08:10.415422 21749 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.393898 28918 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.407886 28844 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.418768 28918 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.419509 28874 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.401593 28937 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:10.352451 28715 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:10.421552 28849 consensus_meta.cc:287] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.810s	user 0.009s	sys 0.010s
I20260709 13:08:10.395140 28741 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:08:10.398646 28926 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.423630 28707 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:10.424161 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.426407 28001 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:10.428328 28923 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:10.413786 28754 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:10.421309 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.440786 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.451243 28001 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:10.452093 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.452721 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.449319 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.453445 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.450636 28926 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:10.454819 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:10.455405 28849 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:10.456302 28650 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.452127 28896 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:10.443873 28888 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:08:10.456667 28874 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.467417 28923 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.476440 22588 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:08:10.417237 28782 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.486466 22006 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:08:10.488963 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78569.7us
I20260709 13:08:10.489063 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.493451 28001 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.446720 28698 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:08:10.487540 28923 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:10.501571 28919 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.502238 28919 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.504614 28919 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.487948 28869 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.493932 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:10.506531 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.508214 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 240818us
I20260709 13:08:10.493799 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.511113 22592 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
W20260709 13:08:10.511969 28950 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:08:10.506923 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.483518 28896 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:10.520299 22590 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:08:10.494300 28789 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:08:10.523527 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.524112 28908 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.525416 21749 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 12, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.527350 28715 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.527925 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.528738 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.529198 21929 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:08:10.529397 21935 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:10.525400 28908 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.509523 28777 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.519402 28855 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:08:10.535786 22588 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:10.536808 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.535830 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.537515 22008 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:10.540359 28708 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.541586 28212 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:10.541301 28937 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.542243 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.544812 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:10.536517 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58869.8us
I20260709 13:08:10.546661 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:10.524312 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 13.
I20260709 13:08:10.486167 28882 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.528771 22584 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:08:10.548408 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.485746 28652 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:10.549157 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.549640 28945 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:08:10.513034 28917 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.553109 21860 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [16.13-20.14]
I20260709 13:08:10.536823 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.554502 28869 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.555326 21857 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:10.556205 28830 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.547933 21749 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 16, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 16 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.570345 22008 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:10.532987 28908 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.520033 28896 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:10.602263 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:10.579749 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2652) took 1037 ms. Trace:
I20260709 13:08:10.616182 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.522790 28844 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.493258 28874 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:10.607743 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.612605 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.613945 21931 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:10.572100 28212 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:10.579705 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.617208 22006 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:10.620795 28830 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.621563 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:10.623862 28882 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:10.625293 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 86971.7us
I20260709 13:08:10.625599 22590 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:10.626166 21859 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:08:10.603080 28387 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.626521 21853 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 13:08:10.627130 21754 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 307) took 86255 ms (client timeout 15000 ms). Trace:
W20260709 13:08:10.629253 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 84743.6us
I20260709 13:08:10.627786 28708 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:10.636755 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.638784 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.606951 28763 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.644527 22591 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.645569 28874 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.646195 28763 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.647142 28001 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:10.653661 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:10.649803 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:10.650830 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.655310 22592 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:10.657430 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.658304 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.660298 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:10.661842 21929 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:08:10.662809 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:10.663040 28820 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:10.663493 28820 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.663894 28820 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.664597 21825 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.650532 28869 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:10.666509 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.666831 28618 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:10.653826 22008 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:10.672717 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.678155 22588 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:08:10.679158 22588 pending_rounds.cc:85] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 15
I20260709 13:08:10.684151 28926 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:10.641237 21754 rpcz_store.cc:269] 0709 13:06:44.371791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.372123 (+   332us) service_pool.cc:224] Handling call
0709 13:06:44.617520 (+245397us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:10.626839 (+86009319us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:03.460344 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:03.461042 (+   698us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:03.461099 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:08:03.461118 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:03.462061 (+   943us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:03.462138 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:03.462163 (+    25us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:08:03.462867 (+   704us) tablet.cc:701] Row locks acquired
0709 13:08:03.462912 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:08:03.463073 (+   161us) write_op.cc:270] Start()
0709 13:08:03.463224 (+   151us) write_op.cc:276] Timestamp: P: 1783602483463002 usec, L: 0
0709 13:08:03.463240 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:08:03.464071 (+   831us) log.cc:844] Serialized 3316 byte log entry
0709 13:08:04.043409 (+579338us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b6d6c50ac 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b
0709 13:08:05.503010 (+1459601us) op_driver.cc:464] REPLICATION: finished
0709 13:08:06.274580 (+771570us) write_op.cc:301] APPLY: starting
0709 13:08:06.298641 (+ 24061us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:06.423823 (+125182us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:06.423868 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:06.749636 (+325768us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:06.838143 (+ 88507us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b18001914a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:06.938697 (+100554us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:08:06.951486 (+ 12789us) write_op.cc:312] APPLY: finished
0709 13:08:07.026448 (+ 74962us) log.cc:844] Serialized 81 byte log entry
0709 13:08:07.864905 (+838457us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:07.883885 (+ 18980us) write_op.cc:454] Released schema lock
0709 13:08:07.886679 (+  2794us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:07.943737 (+ 57058us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:07.952818 (+  9081us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":987501,"reported_tablets":49,"spinlock_wait_cycles":24375552,"child_traces":[["op",{"apply.queue_time_us":689130,"apply.run_cpu_time_us":50616,"apply.run_wall_time_us":1675103,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37615,"num_ops":8,"prepare.queue_time_us":263910,"prepare.run_cpu_time_us":44593,"prepare.run_wall_time_us":1247374,"raft.queue_time_us":436356,"raft.run_cpu_time_us":13970,"raft.run_wall_time_us":642726,"replication_time_us":2000953,"spinlock_wait_cycles":95456256,"thread_start_us":2500265,"threads_started":5,"wal-append.queue_time_us":1751297,"wal-append.run_cpu_time_us":13223,"wal-append.run_wall_time_us":2806947}]]}
I20260709 13:08:10.676276 22006 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:08:10.689363 28919 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.690090 28919 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.616552 21932 rpcz_store.cc:276] 0709 13:08:09.370060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.418845 (+ 48785us) service_pool.cc:224] Handling call
0709 13:08:09.484656 (+ 65811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:09.963344 (+478688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.963388 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:09.964566 (+  1178us) log.cc:844] Serialized 52 byte log entry
0709 13:08:10.078805 (+114239us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:10.345215 (+266410us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.394289 (+ 49074us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.395608 (+  1319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.403800 (+  8192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.403845 (+    45us) raft_consensus.cc:1685] finished
0709 13:08:10.403856 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.407006 (+  3150us) inbound_call.cc:177] Queueing success response
0709 13:08:10.412114 (+  5108us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b18002ea740. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
Metrics: {"mutex_wait_us":60556,"prepare.queue_time_us":41173,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":109,"raft.queue_time_us":164745,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":12095,"spinlock_wait_cycles":5397888,"thread_start_us":708099,"threads_started":3,"wal-append.queue_time_us":125457}
I20260709 13:08:10.758260 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:10.720016 28750 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.006s
I20260709 13:08:10.722755 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.719151 21853 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:10.717357 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.759159 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.717137 21749 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 14, leader changed from <none> to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 14 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:10.723368 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.827376 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:10.680639 22588 pending_rounds.cc:107] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:08:10.723946 28908 consensus_queue.cc:1059] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:10.723132 28886 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:10.759125 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.661890 28917 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.721546 28741 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:10.720840 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.691866 28715 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:08:10.718236 28387 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:10.844367 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.695252 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.839358 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:10.845079 21934 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:10.722443 28869 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.650269 28844 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:10.770121 22004 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.806090 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.660579 28212 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.772119 21930 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:10.812142 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:10.812989 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.814911 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.762593 21851 raft_consensus.cc:2393] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:10.771097 28830 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.770145 28874 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.801399 28618 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:10.724974 28905 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.765111 28708 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.832316 22586 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.17-19.17]
I20260709 13:08:10.827790 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.834229 22588 raft_consensus.cc:2889] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:10.832940 22584 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:10.833818 21931 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:10.835923 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 207478us
I20260709 13:08:10.832118 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.838697 28652 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:10.839550 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 116931us
I20260709 13:08:10.848809 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.839242 28886 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.863154 21749 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.844918 28789 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:10.864781 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.868718 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:10.892062 22002 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 13:08:10.890455 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.852905 28715 consensus_queue.cc:1243] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:10.893592 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.892530 22002 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:10.893296 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:10.897935 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.879635 28874 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.899355 28905 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.899843 28874 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:10.891487 28789 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:10.900770 28789 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:10.903753 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:10.885989 28906 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.905833 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.941028 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.906477 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.913162 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.942142 28001 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:10.942510 28001 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:10.942839 28001 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:10.887789 28919 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.895342 22008 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:10.895721 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:10.949508 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:10.949266 21858 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:10.925962 28708 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.930573 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:10.935552 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 130908us
I20260709 13:08:10.938239 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:10.951475 22005 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:10.889294 28886 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:10.948729 21926 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:10.917573 21928 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:10.919274 28923 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.894193 21932 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:08:10.958271 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:10.954603 22009 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:08:10.866950 28844 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:10.945088 21934 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:10.887428 28678 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:10.957672 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:10.962443 21930 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:10.961127 21857 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 13:08:10.965195 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 85342.3us
I20260709 13:08:10.967343 22004 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:10.968431 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:10.968245 28918 consensus_meta.cc:287] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
W20260709 13:08:10.970230 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 66604.7us
I20260709 13:08:10.970044 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.973394 21926 raft_consensus.cc:2393] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:10.973696 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.974576 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.975574 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:10.895139 28830 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.973034 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:10.978834 22009 raft_consensus.cc:2393] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:10.980273 22588 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:10.980602 21934 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:08:10.977102 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:10.985936 21932 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:10.984091 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:10.988058 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:10.989641 21926 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:10.990406 22003 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:10.991763 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:10.991966 28869 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:10.917568 28618 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.992656 22587 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 22 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:10.993860 28844 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:10.995023 28001 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:10.997545 21853 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:10.977286 28908 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:10.981817 28963 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:08:10.998238 21929 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:11.002552 28906 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.005726 22000 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:08:11.005951 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:11.007134 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.984468 28741 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:11.013761 21934 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:08:11.019820 28001 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:10.985184 28926 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.021453 22005 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:08:10.978456 28918 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.037576 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:11.039542 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.039302 28937 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:08:11.043658 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.045881 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.046404 28820 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.051293 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:11.053591 28750 raft_consensus.cc:3055] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:11.054400 28001 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:11.069620 28652 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.070333 28652 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.069694 28750 raft_consensus.cc:740] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:11.071641 28750 consensus_queue.cc:260] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.072350 28652 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.072816 28750 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:11.072989 28820 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.003382 21749 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 18 to 19, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 19 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:11.075871 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.077824 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:11.056169 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:11.057333 28919 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.057250 28869 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.058305 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:11.100945 28917 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:11.101140 28869 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.101533 28917 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.101970 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:11.103407 28869 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.103703 28917 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.104107 28869 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:11.104528 28869 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:10.999768 28908 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:11.064198 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 133778us
I20260709 13:08:11.069195 21929 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.070659 28918 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.055795 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.079522 22001 raft_consensus.cc:2410] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:11.081234 21853 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:11.077864 28923 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.088150 28829 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:11.091485 21934 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:11.104768 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:11.092347 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.097905 28708 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.099611 28830 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.100296 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:11.128469 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:11.170881 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70665.1us
W20260709 13:08:11.118430 28966 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:11.074419 21749 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 18 to 19, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:11.171924 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:11.172550 22586 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:11.173473 22003 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.172745 21749 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 13, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 13 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:11.174091 28895 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:11.174569 28895 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.175149 28895 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:11.174602 21749 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:11.100143 28919 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.177670 21749 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 19, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 19 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:11.178723 28919 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.179032 28917 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:08:11.179863 28917 consensus_queue.cc:1243] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:11.179339 21749 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:11.180363 28917 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:11.180881 28917 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.181216 28850 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:11.181743 28850 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:11.182480 22593 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:11.183094 28917 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.182716 28850 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.183640 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:11.162626 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:11.120409 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.184393 22585 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:11.106635 21930 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:11.133669 28937 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:11.107470 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:11.121837 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.169929 22584 pending_rounds.cc:85] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 13
I20260709 13:08:11.187726 22584 pending_rounds.cc:107] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:08:11.188081 22002 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:11.163057 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:11.188270 22584 raft_consensus.cc:2889] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:11.188452 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:11.173211 28829 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.191774 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.105229 28715 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:11.104950 28869 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.193387 28829 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.178318 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:11.194180 22587 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:11.197824 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93145.1us
I20260709 13:08:11.198676 28708 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:11.199771 28715 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:11.200273 28715 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:11.189337 22588 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:08:11.200879 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:11.122419 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.126961 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:11.105553 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.106124 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.121172 28001 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:08:11.119488 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:11.205701 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 98297.7us
I20260709 13:08:11.187765 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:11.170938 22592 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:11.191351 28850 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:11.209919 22001 raft_consensus.cc:2368] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23.
I20260709 13:08:11.121654 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:11.200273 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:11.210855 21858 raft_consensus.cc:2368] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23.
W20260709 13:08:11.211095 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48146.8us
I20260709 13:08:11.170184 28908 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.236439 22586 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:11.201414 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:11.119952 28387 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:08:11.242179 21928 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:11.242988 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:11.254766 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49952.5us
I20260709 13:08:11.243424 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:11.255970 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:11.238701 21825 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:11.247076 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 361954us
I20260709 13:08:11.189114 28830 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:11.262765 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:11.107754 28900 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.277s
I20260709 13:08:11.189038 28937 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:11.281203 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.269963 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.278007 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:11.286108 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.110731 28763 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.144s
I20260709 13:08:11.236395 28906 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:11.296453 22000 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:11.244900 28842 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:11.294448 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:11.302927 21933 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.264029 28708 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:11.310459 22005 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:11.312395 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:11.308370 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46632.4us
W20260709 13:08:11.308765 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 65657.9us
I20260709 13:08:11.317452 22557 leader_election.cc:400] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23.
I20260709 13:08:11.317828 22557 leader_election.cc:403] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:08:11.318019 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:11.303447 28906 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:11.325134 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.325377 28906 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:11.329293 22008 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:11.334352 22585 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:11.342716 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 58575.2us
I20260709 13:08:11.287858 28937 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:11.346513 22003 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:11.346629 22004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49448
I20260709 13:08:11.348461 22004 raft_consensus.cc:606] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Received request to transfer leadership
I20260709 13:08:11.348794 22004 raft_consensus.cc:614] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.329746 28807 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:11.347098 28830 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:11.329903 28707 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:11.349431 28807 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.311758 28708 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.350342 28807 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:11.350598 28708 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.353749 28763 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23.
I20260709 13:08:11.285842 28905 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:11.317322 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:11.357695 21928 raft_consensus.cc:606] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:11.358211 21928 raft_consensus.cc:614] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.318650 28917 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:11.314920 28741 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:11.329937 28820 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:11.362485 28652 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:11.362681 28888 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:11.378489 28807 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.342240 28829 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:11.345036 28874 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:11.386749 28387 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:11.405185 28895 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.406585 28869 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.407622 28895 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.409186 28387 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:11.410684 28906 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.410615 28387 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:11.413362 28906 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:11.414755 28886 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.416149 21859 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:11.417989 28886 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.412783 28387 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:11.425344 28869 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.403244 21933 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:11.405208 28829 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:11.434530 28829 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.437471 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:11.439189 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:11.440560 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:11.365058 28842 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 13:08:11.445490 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.446184 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:11.363301 28789 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.451278 21928 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:08:11.458791 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:11.461790 22586 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:11.384286 28820 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.381819 28905 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.344424 28715 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.241151 21749 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 16, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:11.470534 28869 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:11.472361 28820 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:11.495893 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "430edce57310489986e4f739148461a6"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:11.497201 21928 raft_consensus.cc:606] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:11.499266 21928 raft_consensus.cc:614] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.506223 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.508132 22006 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:11.495909 28715 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:11.533630 21929 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
W20260709 13:08:11.536504 21926 consensus_meta.cc:287] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:08:11.537642 21926 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
W20260709 13:08:11.560252 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.573284 21926 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:11.492815 28869 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:11.586774 21928 raft_consensus.cc:1240] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:08:11.590257 21929 raft_consensus.cc:1240] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:11.591008 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:11.597108 21933 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:08:11.605993 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 350120us
I20260709 13:08:11.612625 21823 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:11.629791 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:11.631726 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.634130 21929 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:11.634660 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:11.635973 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 398622us
I20260709 13:08:11.638702 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:11.647074 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 435209us
I20260709 13:08:11.648171 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.654876 22003 raft_consensus.cc:2410] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:08:11.637626 28896 consensus_meta.cc:287] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.003s	user 0.010s	sys 0.002s
I20260709 13:08:11.650606 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.652977 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.656608 22005 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:11.638365 28918 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:11.641990 28212 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:11.669806 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:11.668900 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 459225us
I20260709 13:08:11.671319 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:11.668210 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 308557us
I20260709 13:08:11.665445 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.673337 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.675410 22001 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:11.646633 28678 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:11.672693 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.677592 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:11.680033 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:11.680318 22000 raft_consensus.cc:2410] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:11.680713 22586 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:11.664377 28652 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:11.670295 28807 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.642303 28844 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:08:11.682178 28789 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.677714 28678 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.682612 28707 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:11.652163 28708 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.718374 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:11.687765 28652 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.666276 28896 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:11.729987 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58745.9us
I20260709 13:08:11.733847 21851 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:08:11.685806 28886 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:11.745623 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 80138.7us
I20260709 13:08:11.692339 28905 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:11.758970 22002 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:08:11.652907 28869 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.713503 28678 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:11.736555 28926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.016s	sys 0.020s
I20260709 13:08:11.766672 22591 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:11.719671 28844 consensus_queue.cc:1243] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:11.768105 22586 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:08:11.746271 28652 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:11.789098 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50636us
W20260709 13:08:11.769598 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 63034.8us
I20260709 13:08:11.794062 22589 raft_consensus.cc:2393] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:08:11.796471 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 67830.4us
I20260709 13:08:11.746184 28906 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:11.809631 22585 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
W20260709 13:08:11.790509 28926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.016s	sys 0.021s
I20260709 13:08:11.813129 21935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:11.816125 21935 raft_consensus.cc:606] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:11.816658 21935 raft_consensus.cc:614] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.818631 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:11.819998 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:11.758776 28900 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 13:08:11.828377 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 233522us
I20260709 13:08:11.801483 28917 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.832214 28001 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:11.842010 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:11.843202 22588 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:11.851678 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:11.857372 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:11.858673 22591 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 23: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:11.846201 21825 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:11.874512 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:11.678037 21749 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 4 to 14, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 14 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:11.900357 22006 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:11.905234 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:11.910429 22007 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:11.911513 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.911978 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.834699 28906 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:11.919950 22592 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:11.922080 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.922860 22007 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:11.841265 28987 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:11.851315 28895 consensus_queue.cc:1059] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:08:11.933509 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.874889 28789 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:11.887970 28970 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 13:08:11.927945 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 81850us
I20260709 13:08:11.931918 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:11.888087 28874 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:11.921475 28919 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:11.941444 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.947410 22001 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 23: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:11.964403 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:11.955571 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 113673us
I20260709 13:08:11.957947 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.939678 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:11.877910 28979 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:08:11.969501 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:11.929517 28989 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:11.929219 28987 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:11.961867 28974 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.265s	user 0.001s	sys 0.001s
I20260709 13:08:11.887085 28917 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.947254 28985 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:11.948676 28789 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:11.924904 28900 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:08:11.996057 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 110995us
I20260709 13:08:11.971442 22002 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 2 to 12, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:11.967330 28886 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:11.925397 28980 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.078s
W20260709 13:08:12.016383 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.977205 28989 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:12.039912 22592 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:08:12.009457 28972 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.332s	user 0.000s	sys 0.002s
I20260709 13:08:12.039472 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1396) took 1067 ms. Trace:
I20260709 13:08:11.913447 28707 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:11.995472 28987 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:12.015578 28789 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:12.039810 28970 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:08:12.057276 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:12.013075 28919 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:12.048477 28977 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.360s	user 0.000s	sys 0.001s
I20260709 13:08:12.010109 28874 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:12.067360 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 216080us
I20260709 13:08:12.070510 22005 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:12.027417 28807 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.049302 22009 rpcz_store.cc:276] 0709 13:08:10.972365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.982994 (+ 10629us) service_pool.cc:224] Handling call
0709 13:08:11.001405 (+ 18411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.020774 (+ 19369us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:08:11.069518 (+ 48744us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.971271 (+901753us) log.cc:844] Serialized 14 byte log entry
0709 13:08:12.038474 (+ 67203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:12.038608 (+   134us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:12.038741 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.038893 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.039335 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19700,"prepare.queue_time_us":87499,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":6578,"raft.queue_time_us":203839,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":14443,"thread_start_us":949710,"threads_started":4,"wal-append.queue_time_us":687939}
W20260709 13:08:12.057956 28973 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 13:08:12.054592 28708 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:12.079458 28886 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:12.062665 28975 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.297s	user 0.001s	sys 0.001s
I20260709 13:08:12.099030 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1238) took 1283 ms. Trace:
I20260709 13:08:12.099634 21927 rpcz_store.cc:276] 0709 13:08:10.815350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.839882 (+ 24532us) service_pool.cc:224] Handling call
0709 13:08:10.879521 (+ 39639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.898004 (+ 18483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.898046 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.898635 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.089970 (+191335us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b18004b64c0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:08:11.572524 (+482554us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:11.572571 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.572658 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.098440 (+525782us) raft_consensus.cc:1685] finished
0709 13:08:12.098471 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.098791 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":432,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":4518528,"thread_start_us":482334,"threads_started":1,"wal-append.queue_time_us":789138}
W20260709 13:08:12.122522 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65342us
I20260709 13:08:12.124132 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:12.072458 28896 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:12.145107 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 175724us
W20260709 13:08:12.132939 28937 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.000s
I20260709 13:08:12.147238 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:12.051309 28906 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:12.170017 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.171054 28971 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.273s	user 0.001s	sys 0.001s
I20260709 13:08:12.171577 29022 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:12.163383 28937 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:12.175745 28937 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:12.119603 28886 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:12.128585 28807 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:12.166103 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:12.182919 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.179800 28997 log.cc:927] Time spent T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:08:12.182323 28908 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.887s	user 0.008s	sys 0.006s
I20260709 13:08:12.130362 28991 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.277s
I20260709 13:08:12.178001 28896 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:12.209244 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 129109us
I20260709 13:08:12.213454 29020 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.213653 29017 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:12.214398 29013 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:12.204744 28980 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:12.224429 28999 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:08:12.207695 28919 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:12.224175 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:12.228190 22007 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:12.092896 21749 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.252627 22007 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:08:12.214146 29020 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:12.258260 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 129366us
W20260709 13:08:12.229658 28990 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:08:12.262085 21824 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:12.263799 22008 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:12.265333 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 118772us
I20260709 13:08:12.214120 29017 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:12.269191 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 145156us
I20260709 13:08:12.270061 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:12.271438 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:12.270814 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:12.210220 28908 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:12.260174 28850 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.050s	user 0.012s	sys 0.002s
I20260709 13:08:12.181993 28989 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:12.284624 28708 consensus_queue.cc:1059] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:08:12.286846 28981 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:12.287492 28994 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.293s	user 0.001s	sys 0.001s
I20260709 13:08:12.211287 28874 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:12.287773 28986 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:08:12.288261 29024 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:12.288733 29015 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:08:12.289558 28850 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:12.289849 28807 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:12.313973 28750 raft_consensus.cc:3055] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:12.292629 28918 raft_consensus.cc:3055] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:12.294157 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:12.294486 28923 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:12.282474 28991 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:12.298676 28983 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.513s	user 0.003s	sys 0.000s
I20260709 13:08:12.299367 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 1095) took 1319 ms. Trace:
I20260709 13:08:12.399849 21851 rpcz_store.cc:276] 0709 13:08:10.979914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.980058 (+   144us) service_pool.cc:224] Handling call
0709 13:08:12.299154 (+1319096us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54000591d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:12.299295 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935747840}
I20260709 13:08:12.300241 28001 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:12.300981 28979 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:12.301764 29011 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:08:12.304282 29004 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:08:12.222365 28985 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:12.313923 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:12.260463 28917 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:12.290114 22593 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:12.274034 29022 raft_consensus.cc:2764] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:08:12.249996 28896 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:12.215144 29013 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:12.297606 28678 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:12.572908 28994 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:08:12.365793 28923 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:12.308864 28652 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:12.344178 28708 consensus_queue.cc:1243] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:12.361956 28918 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:12.376721 29024 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:12.358668 28850 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:12.345181 29019 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:12.570572 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:12.575729 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:12.359417 28937 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 13:08:12.583120 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:12.587102 21851 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:12.365248 28750 raft_consensus.cc:740] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:12.559201 28886 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.588477 28919 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:12.643137 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 329278us
I20260709 13:08:12.644021 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:12.577239 28829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.025s
I20260709 13:08:12.591037 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:12.592020 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.592378 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.592744 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.593520 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.594319 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.594712 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.595105 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:12.588055 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 326028us
W20260709 13:08:12.580035 28715 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.063s	user 0.010s	sys 0.010s
I20260709 13:08:12.608968 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:12.582540 28982 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:12.613520 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1527) took 1443 ms. Trace:
I20260709 13:08:12.616142 22586 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:12.612497 22589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1349) took 1796 ms (client timeout 2090 ms). Trace:
I20260709 13:08:12.620146 28900 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:12.701234 28886 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:12.648463 28829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.011s	sys 0.025s
W20260709 13:08:12.646867 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.611444 29006 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 13:08:12.615092 28850 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:12.707067 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 59036.9us
I20260709 13:08:12.707582 21856 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:08:12.709772 21853 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:12.706779 21860 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:12.645175 29014 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.717314 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:12.716277 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:12.731951 21852 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:13.025795 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 381802us
I20260709 13:08:13.026701 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:13.026881 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.803431 29032 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:12.916682 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.732254 21855 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:12.712415 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 155071us
I20260709 13:08:12.743634 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:12.711068 22584 rpcz_store.cc:276] 0709 13:08:11.166518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.166683 (+   165us) service_pool.cc:224] Handling call
0709 13:08:11.169692 (+  3009us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:11.254993 (+ 85301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.255028 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:11.276397 (+ 21369us) log.cc:844] Serialized 52 byte log entry
0709 13:08:11.693703 (+417306us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:11.693750 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.693833 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.609563 (+915730us) raft_consensus.cc:1685] finished
0709 13:08:12.609586 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.609746 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33358,"raft.queue_time_us":57628,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":14236,"thread_start_us":345755,"threads_started":1,"wal-append.queue_time_us":507147}
W20260709 13:08:12.713197 22589 rpcz_store.cc:269] 0709 13:08:10.810644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.814730 (+  4086us) service_pool.cc:224] Handling call
0709 13:08:12.606626 (+1791896us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400531954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:12.606765 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099595136}
W20260709 13:08:13.039343 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 323143us
I20260709 13:08:13.032333 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:12.642771 28707 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.752511 22586 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:12.780735 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3202) took 1446 ms. Trace:
W20260709 13:08:13.039559 21972 connection.cc:256] RPC call timeout handler was delayed by -2.7120508869999185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
I20260709 13:08:12.606494 21749 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.056752 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:12.870390 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.040822 22591 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:13.057561 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.082069 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:12.870990 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:12.724539 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:12.716372 28212 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.038s	user 0.004s	sys 0.006s
W20260709 13:08:12.725610 29028 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:08:13.080106 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.712414 29023 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.720520 28886 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:12.713863 29026 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:08:12.714417 28993 log.cc:927] Time spent T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
I20260709 13:08:12.717474 29013 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:12.803831 28908 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:13.211392 21826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.008s	sys 0.012s
W20260709 13:08:13.211903 21826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.008s	sys 0.012s
W20260709 13:08:13.212555 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.211421 22592 raft_consensus.cc:1240] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:12.722865 28741 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:12.725312 28650 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:13.094883 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:12.701144 28981 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:13.101991 21856 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:13.021893 29022 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.088965 21926 rpcz_store.cc:276] 0709 13:08:11.334586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.577679 (+243093us) service_pool.cc:224] Handling call
0709 13:08:11.578046 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.594521 (+ 16475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.594567 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.611681 (+ 17114us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.851117 (+239436us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:11.913245 (+ 62128us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.922771 (+  9526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.922875 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.726831 (+803956us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b24001bef18. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d720fb0 00007f3b70397488 00007f3b70397be8 00007f3b7036f398 00007f3b7036a76a 00007f3b70369bb6
0709 13:08:12.728173 (+  1342us) raft_consensus.cc:1685] finished
0709 13:08:12.780482 (+ 52309us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.780662 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10095,"prepare.queue_time_us":74002,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":36281,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":34148,"spinlock_wait_cycles":64475392,"thread_start_us":192176,"threads_started":1,"wal-append.queue_time_us":328791}
I20260709 13:08:12.781845 29015 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:12.869793 28900 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:13.247256 22004 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.009s	sys 0.013s
W20260709 13:08:13.247650 22004 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.009s	sys 0.013s
I20260709 13:08:13.173012 22586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:08:12.713526 28750 consensus_queue.cc:260] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:13.105343 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 78737.5us
I20260709 13:08:12.712106 28919 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.047456 28842 raft_consensus.cc:3055] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:13.057750 28678 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:13.093787 29025 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
I20260709 13:08:13.059109 28917 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:08:13.213511 21974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.004s	sys 0.016s
I20260709 13:08:13.216210 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:13.297587 21974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.004s	sys 0.016s
I20260709 13:08:13.216287 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.220675 22589 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:13.233714 22590 raft_consensus.cc:2410] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:08:13.057183 28896 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:13.241290 22593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1530) took 2063 ms (client timeout 1844 ms). Trace:
W20260709 13:08:13.241765 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.317476 21926 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:13.248883 22586 raft_consensus.cc:606] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Received request to transfer leadership to TS 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:13.319417 22586 raft_consensus.cc:614] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:13.320881 22000 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:13.287667 22584 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:12.806692 29014 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.291627 21933 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 13:08:13.298367 22587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.017s	sys 0.026s
I20260709 13:08:13.347695 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.348407 22002 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:13.348799 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.292464 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.349699 21851 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:13.293347 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:13.245491 29026 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:13.231984 28650 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:13.359330 28650 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.360396 28650 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:13.361374 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:13.362237 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.362574 28708 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:13.362133 21749 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 8 to 15, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 15 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:13.364034 28708 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:13.363778 21749 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 15, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 15 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:13.261656 22591 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:13.365269 28984 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:13.217422 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.366210 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.366495 21860 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:13.366829 21856 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:13.367478 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:12.912246 28923 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:13.297997 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 728808us
I20260709 13:08:13.362133 21926 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:13.369763 21927 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:13.370541 21926 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:13.356683 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.371804 21928 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:13.364578 28708 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.373315 29012 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.373840 29012 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.374994 28708 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.375954 29012 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:13.381496 29018 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.637s	user 0.000s	sys 0.001s
W20260709 13:08:13.384553 21974 leader_election.cc:336] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 2.090s (SENT)
W20260709 13:08:13.317126 22593 rpcz_store.cc:269] 0709 13:08:11.177795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.183377 (+  5582us) service_pool.cc:224] Handling call
0709 13:08:13.241056 (+2057679us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003791d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:13.241194 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543027584}
I20260709 13:08:13.298668 21855 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:13.288244 28789 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.202s	user 0.009s	sys 0.003s
I20260709 13:08:13.296499 28886 consensus_queue.cc:1059] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:08:13.256979 29020 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:13.356242 29016 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.344s
I20260709 13:08:13.357090 29014 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.405079 28917 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:13.357704 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.442772 21851 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:13.358170 21853 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:13.385298 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.387125 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3197) took 2398 ms. Trace:
I20260709 13:08:13.043694 29032 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.389667 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.085547 28707 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:13.417253 28987 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.342s	user 0.005s	sys 0.008s
I20260709 13:08:13.350751 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.399497 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:13.255005 28842 raft_consensus.cc:740] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:13.405077 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 364956us
I20260709 13:08:13.295754 28981 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:13.445385 21931 rpcz_store.cc:276] 0709 13:08:10.988607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.340978 (+352371us) service_pool.cc:224] Handling call
0709 13:08:11.366539 (+ 25561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.385626 (+ 19087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.385678 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.389606 (+  3928us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.714017 (+324411us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:11.714078 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.714171 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.386898 (+1672727us) raft_consensus.cc:1685] finished
0709 13:08:13.386922 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.387064 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12607,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":2647,"thread_start_us":292629,"threads_started":1,"wal-append.queue_time_us":868958}
W20260709 13:08:13.465950 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 409262us
I20260709 13:08:12.845479 28985 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.303268 28652 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.404668 28708 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:08:13.453349 29019 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:13.253791 28820 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.758s	user 0.014s	sys 0.000s
I20260709 13:08:13.417719 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.083030 29024 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.209584 29023 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.349030 28618 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.421844 28919 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:13.429515 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:13.444689 21935 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:13.403043 28984 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:13.392725 21928 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:13.323453 28980 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:08:13.351696 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:13.466873 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:13.468268 22586 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:13.402012 29013 consensus_queue.cc:1059] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:08:13.412801 22589 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:08:13.450489 29020 raft_consensus.cc:3055] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:13.474273 21933 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.442087 28917 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.407307 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:13.484938 21851 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:13.411183 28789 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.489161 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.252077 29022 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.498854 21853 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
W20260709 13:08:13.457937 29006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.008s	sys 0.050s
I20260709 13:08:13.501660 21929 raft_consensus.cc:2410] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:08:13.499074 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.043s	sys 0.076s
I20260709 13:08:13.473778 29014 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:13.510783 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:13.512290 29006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.008s	sys 0.050s
I20260709 13:08:13.538872 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.503222 28652 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:08:13.542737 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.050s	sys 0.078s
W20260709 13:08:13.543570 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 76813.6us
I20260709 13:08:13.466534 28987 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:13.547582 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 740188us
I20260709 13:08:13.488499 28886 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:08:13.552580 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:13.556897 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:13.509791 21898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.007s	sys 0.003s
W20260709 13:08:13.558300 21898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.007s	sys 0.003s
I20260709 13:08:13.548867 21933 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:08:13.506222 29020 raft_consensus.cc:740] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:13.563812 21854 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:13.498756 28820 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.504256 28650 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:08:13.571919 22003 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:13.504370 28618 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.467538 28900 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.579321 21856 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
W20260709 13:08:13.580022 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.022s	sys 0.017s
I20260709 13:08:13.510221 29015 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 13:08:13.581269 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.022s	sys 0.017s
I20260709 13:08:13.547245 28652 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:13.497957 29019 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.580250 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.581604 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.584911 22000 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:08:13.584445 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 199950us
I20260709 13:08:13.586522 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:13.588009 21857 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:13.592012 22586 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:13.601555 21898 connection.cc:256] RPC call timeout handler was delayed by -1.1598492799998894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
I20260709 13:08:13.606889 21928 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:13.609107 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:13.580791 28869 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.572863 29048 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:13.609872 22006 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:13.550238 29014 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.486207 21749 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.611476 22589 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:13.594702 28844 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.555433 28842 consensus_queue.cc:260] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.589710 28708 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:13.590430 28919 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.586108 29006 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:13.595875 29040 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:13.623551 21929 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:13.624387 22003 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:13.633329 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:13.625100 21898 connection.cc:256] RPC call timeout handler was delayed by -1.0734492799998634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 13:08:13.633888 22000 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.565784 29043 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:08:13.618918 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.635146 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:13.638235 21853 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
W20260709 13:08:13.634276 21898 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.596s (SENT) [suppressed 13 similar messages]
I20260709 13:08:13.644243 22006 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:08:13.646922 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.650271 21928 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:13.649194 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:13.654629 22003 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:13.628409 29054 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.656827 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.650039 28212 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:13.669183 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.631531 28844 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.646551 29053 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:13.654083 29048 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.679440 22003 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:13.680030 22003 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:13.620662 29020 consensus_queue.cc:260] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.693014 29015 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:13.694157 29020 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:13.694125 29015 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:13.696079 29015 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:13.696545 29015 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:13.697448 29044 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:08:13.686997 22003 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.699038 22003 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:13.700603 28987 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.687670 29023 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:13.716572 29023 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:13.722528 29023 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.723150 29043 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:13.723585 29043 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:13.690012 21824 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:13.724550 28991 consensus_meta.cc:287] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.275s	user 0.008s	sys 0.000s
I20260709 13:08:13.724417 29043 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.725241 28886 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:13.725646 29055 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:13.725782 29043 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:13.690580 29014 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:13.597517 28984 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.649395 28708 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.672775 29054 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.642614 28869 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.713392 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.681072 28212 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:13.716977 21749 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 11, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 11 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.718257 28789 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:13.718967 28850 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.993s	user 0.004s	sys 0.005s
W20260709 13:08:13.673084 29047 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:08:13.703398 29050 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:13.721534 29015 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.688536 28820 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:13.727269 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3255) took 1004 ms. Trace:
W20260709 13:08:13.794740 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 188890us
I20260709 13:08:13.703061 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1407) took 1117 ms. Trace:
I20260709 13:08:13.728760 22000 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:08:13.727187 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:13.797140 21927 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:13.797791 28789 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:13.797935 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70808.9us
I20260709 13:08:13.798326 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.799664 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.692306 28844 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.731107 22003 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:13.647482 29006 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.803061 29006 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:13.653476 28652 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.803809 28652 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:13.805734 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 171474us
I20260709 13:08:13.646679 28919 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:13.812287 21898 connection.cc:256] RPC call timeout handler was delayed by -1.0285981819999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
I20260709 13:08:13.740316 28650 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:13.742283 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:13.724586 29023 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:13.671298 29056 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.688584 29017 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:13.745651 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 193173us
I20260709 13:08:13.745666 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1408) took 1135 ms. Trace:
I20260709 13:08:13.749962 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.820094 21926 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:13.724987 28991 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:13.767618 21928 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:13.779381 21853 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:08:13.783779 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.795377 22004 rpcz_store.cc:276] 0709 13:08:12.723072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.723285 (+   213us) service_pool.cc:224] Handling call
0709 13:08:12.723673 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:13.247967 (+524294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:13.248005 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:13.255543 (+  7538us) log.cc:844] Serialized 26 byte log entry
0709 13:08:13.543135 (+287592us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:13.543189 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.543274 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.727067 (+183793us) raft_consensus.cc:1685] finished
0709 13:08:13.727087 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.727211 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":658207,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":117,"thread_start_us":680667,"threads_started":2,"wal-append.queue_time_us":257017}
I20260709 13:08:13.783941 29059 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.816787 21931 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:13.820487 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 214246us
I20260709 13:08:13.796559 22587 rpcz_store.cc:276] 0709 13:08:12.586226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.586368 (+   142us) service_pool.cc:224] Handling call
0709 13:08:12.608676 (+ 22308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.609847 (+  1171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.609886 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.613926 (+  4040us) log.cc:844] Serialized 26 byte log entry
0709 13:08:13.348190 (+734264us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:13.348231 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.348288 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.702832 (+354544us) raft_consensus.cc:1685] finished
0709 13:08:13.702860 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.703005 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56908,"raft.queue_time_us":844,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6533,"thread_start_us":602102,"threads_started":1,"wal-append.queue_time_us":786275}
I20260709 13:08:13.827121 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.828282 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.830380 21928 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:13.829963 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:13.828382 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:13.832072 22592 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:08:13.832226 21927 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:13.809863 28707 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.701817 29053 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.727514 29048 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.811781 29006 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:13.838032 21926 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [17.16-19.17]
I20260709 13:08:13.819710 22588 rpcz_store.cc:276] 0709 13:08:12.610388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.610527 (+   139us) service_pool.cc:224] Handling call
0709 13:08:12.643160 (+ 32633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.708178 (+ 65018us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:13.242281 (+534103us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.745431 (+503150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:13.745469 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:13.745499 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.745542 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.745623 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3549,"prepare.queue_time_us":97900,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":55,"raft.queue_time_us":74591,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":22157,"thread_start_us":936765,"threads_started":3,"wal-append.queue_time_us":532511}
W20260709 13:08:13.835481 21898 connection.cc:256] RPC call timeout handler was delayed by -1.0167492799998854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22850000000000004s for a response
I20260709 13:08:13.791698 28850 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.844878 21860 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:08:13.852252 29023 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:13.852960 29041 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
I20260709 13:08:13.856775 21931 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:08:13.859021 28919 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.863233 21927 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:08:13.833879 28820 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:13.865447 21926 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:08:13.866382 21935 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:13.825294 28886 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:08:13.863641 21927 pending_rounds.cc:85] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:08:13.877278 21928 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:08:13.872141 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.877737 22586 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:13.878158 21927 pending_rounds.cc:107] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:08:13.871280 22556 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:13.879313 21927 raft_consensus.cc:2889] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:13.879719 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.879314 21935 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:13.881600 21928 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:13.878392 21926 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:13.877974 28974 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.797s	user 0.009s	sys 0.015s
I20260709 13:08:13.809615 29050 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.894310 29023 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:13.877198 21749 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 17, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:13.897795 28869 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:13.899950 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3056) took 1301 ms. Trace:
I20260709 13:08:13.900496 21860 rpcz_store.cc:276] 0709 13:08:12.598775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.367217 (+768442us) service_pool.cc:224] Handling call
0709 13:08:13.899738 (+532521us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b540018f6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:13.899884 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091289472}
I20260709 13:08:13.901737 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:13.902357 21860 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:13.904424 28918 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.834618 29059 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.825757 29056 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:13.906690 28995 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:13.884474 28789 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:13.907152 28995 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:13.907804 29056 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:13.909474 21928 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:13.869413 28820 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:13.840149 28900 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:08:13.908066 28995 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.917899 29053 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:13.864575 29006 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.871073 28212 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:13.918896 29050 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.029789 21928 raft_consensus.cc:1240] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:13.961524 21857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3058) took 1362 ms (client timeout 1697 ms). Trace:
I20260709 13:08:13.930760 21860 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:08:14.002830 29056 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:14.028029 28995 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:14.030877 21857 rpcz_store.cc:269] 0709 13:08:12.599386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.807678 (+208292us) service_pool.cc:224] Handling call
0709 13:08:13.961277 (+1153599us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b540008b6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:13.961449 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159773056}
I20260709 13:08:14.033432 29056 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.034574 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:14.021489 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 177305us
I20260709 13:08:14.036192 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:13.931975 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.027645 29059 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.037264 29063 raft_consensus.cc:2764] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 13:08:13.970299 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.959790 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.959586 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3195) took 3034 ms. Trace:
I20260709 13:08:14.032783 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.039937 21860 raft_consensus.cc:2376] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20. Re-sending same reply.
W20260709 13:08:14.040756 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 174684us
I20260709 13:08:14.041765 21857 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.039070 21932 rpcz_store.cc:276] 0709 13:08:10.924960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.997243 (+ 72283us) service_pool.cc:224] Handling call
0709 13:08:11.012787 (+ 15544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.047911 (+ 35124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.047943 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.048473 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.388667 (+340194us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:11.419451 (+ 30784us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.419901 (+   450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.419980 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.845275 (+2425295us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b24001bef18. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d5689c4 00007f3b6d566acb 00007f3b6d721dc9 00007f3b6d721a55 00007f3b6d72109d 00007f3b7035310e 00007f3b7036a88c 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221
0709 13:08:13.944757 (+ 99482us) raft_consensus.cc:1685] finished
0709 13:08:13.959315 (+ 14558us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.959507 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30303,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":4507,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":177,"spinlock_wait_cycles":6248832,"thread_start_us":339776,"threads_started":1,"wal-append.queue_time_us":616422}
W20260709 13:08:14.017107 29070 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:08:14.047274 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:14.060467 22002 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:14.048005 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.062407 22587 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:08:14.049561 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:14.049940 21900 connection.cc:256] RPC call timeout handler was delayed by -1.2508845640001027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
I20260709 13:08:14.030836 29022 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.052043 22589 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:08:14.052592 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.064992 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:14.055621 22590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57098 (request call id 1404) took 3401 ms (client timeout 1838 ms). Trace:
I20260709 13:08:14.055540 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.056288 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.056596 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.066807 22006 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:14.054419 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.067867 22004 raft_consensus.cc:2410] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 13:08:14.066206 22590 rpcz_store.cc:269] 0709 13:08:10.654331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.654500 (+   169us) service_pool.cc:224] Handling call
0709 13:08:14.055401 (+3400901us) spinlock_profiling.cc:243] Waited 2.39 s on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.055527 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5491789696}
I20260709 13:08:14.069849 22001 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.070914 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.041543 28918 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:14.039963 28900 consensus_queue.cc:1243] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:08:14.075802 22556 connection.cc:256] RPC call timeout handler was delayed by -2.2063768089999485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
I20260709 13:08:14.073704 21900 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:14.085444 22002 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:14.089932 22557 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:13.956898 29054 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.073374 28869 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:13.967501 29066 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.149s
I20260709 13:08:14.109229 21860 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:14.136183 21856 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:14.133502 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:14.138156 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 55288.1us
W20260709 13:08:14.139115 22556 connection.cc:256] RPC call timeout handler was delayed by -1.4671532819999129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:08:14.139523 22556 connection.cc:256] RPC call timeout handler was delayed by -1.4671532819999129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:08:14.139839 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0554295370000091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 13:08:14.140431 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.106930 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:14.150259 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.123582 28844 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:13.963892 29071 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.258s
I20260709 13:08:14.160267 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.069108 29060 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 13:08:14.123723 29056 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.076892 28919 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:08:14.106648 29022 raft_consensus.cc:2804] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:14.161194 21860 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:14.161938 22556 leader_election.cc:400] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23.
I20260709 13:08:14.138756 29054 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.170898 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3049) took 1574 ms. Trace:
I20260709 13:08:14.171294 21860 rpcz_store.cc:276] 0709 13:08:12.596576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.137395 (+1540819us) service_pool.cc:224] Handling call
0709 13:08:14.170836 (+ 33441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.172613 29074 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:14.175038 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3038) took 1584 ms. Trace:
I20260709 13:08:14.175513 21858 rpcz_store.cc:276] 0709 13:08:12.590176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.592468 (+  2292us) service_pool.cc:224] Handling call
0709 13:08:14.174805 (+1582337us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540028cb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.174969 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2583704320}
I20260709 13:08:14.164660 29052 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.141950 29009 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:08:14.168828 28844 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:14.176863 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.188324 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3047) took 1592 ms. Trace:
I20260709 13:08:14.188231 28844 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.188978 21858 rpcz_store.cc:276] 0709 13:08:12.595909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.176511 (+1580602us) service_pool.cc:224] Handling call
0709 13:08:14.188224 (+ 11713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.190670 21858 raft_consensus.cc:1240] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:14.176797 29052 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.191469 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3050) took 1594 ms. Trace:
I20260709 13:08:14.191972 21858 rpcz_store.cc:276] 0709 13:08:12.596903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.189966 (+1593063us) service_pool.cc:224] Handling call
0709 13:08:14.190422 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.191144 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.191400 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.165115 28212 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:14.193393 28212 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:14.193470 29043 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:14.194619 29043 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.194860 28212 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.195708 29043 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.197170 29043 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:14.183449 29054 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.141261 28869 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:14.185932 29022 raft_consensus.cc:697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:14.216099 29022 consensus_queue.cc:237] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:14.220047 29037 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.859s	user 0.004s	sys 0.000s
W20260709 13:08:14.221493 21806 auto_rebalancer.cc:893] Removing replace marker failed: Replica 7d07aa0dcdd1484ab87925c9a4604b81 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:14.221954 21806 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:08:14.221935 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:14.222491 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:08:14.223001 21932 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.224299 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.142556 28820 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.234705 29017 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:14.235066 28820 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:14.232872 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.234045 29039 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:14.237042 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.243397 21858 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:14.242172 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.244226 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3065) took 1642 ms. Trace:
I20260709 13:08:14.243422 29015 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:14.181066 28387 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:14.262867 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.262975 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:14.260468 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 152829us
W20260709 13:08:14.251905 21825 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Timed out: RequestConsensusVote RPC to 127.20.66.132:46099 timed out after 1.838s (SENT)
I20260709 13:08:14.261245 21856 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:14.267678 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1102) took 1011 ms. Trace:
I20260709 13:08:14.268092 21856 rpcz_store.cc:276] 0709 13:08:13.256659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.216007 (+959348us) service_pool.cc:224] Handling call
0709 13:08:14.236086 (+ 20079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.267379 (+ 31293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.267606 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.244557 21858 rpcz_store.cc:276] 0709 13:08:12.601626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.198474 (+1596848us) service_pool.cc:224] Handling call
0709 13:08:14.237479 (+ 39005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.244002 (+  6523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.244175 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.274194 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.277199 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:14.221536 29069 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:08:14.262733 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.281342 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:14.282734 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.284161 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:14.284834 22590 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.280478 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:14.286464 22589 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:14.287531 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.287551 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.246225 28750 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:14.293114 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.299857 22593 raft_consensus.cc:2410] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:08:14.262485 28387 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.294063 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:14.294174 29059 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:14.294742 29052 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:14.295521 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.310629 22590 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 14.
I20260709 13:08:14.288697 29060 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:14.246152 29039 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.298905 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.300776 21860 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:14.316265 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.305505 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.307802 22584 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:14.280350 28981 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:14.313036 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45609.2us
I20260709 13:08:14.317587 21932 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:14.309136 28707 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:14.327339 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.285174 28987 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:08:14.333655 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:14.342392 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46943.6us
I20260709 13:08:14.335381 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3214) took 1044 ms. Trace:
I20260709 13:08:14.343719 22556 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:14.345351 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:14.336022 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.341869 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.342911 21933 rpcz_store.cc:276] 0709 13:08:13.291382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.348561 (+ 57179us) service_pool.cc:224] Handling call
0709 13:08:14.335135 (+986574us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54002e81d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.335306 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287145216}
I20260709 13:08:14.317288 28895 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.356830 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.357807 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:14.325850 28789 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.368724 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:14.318471 28981 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:14.339766 28707 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.372368 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.373449 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.380264 21860 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.380060 21933 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:08:14.353916 28212 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:14.297241 28869 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.379015 28981 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:14.381629 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.351744 28829 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.406975 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.383440 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.407653 22584 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:14.371104 21972 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:14.408682 21858 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 18: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.354498 28900 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:14.373025 29060 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:14.377631 29052 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:08:14.373782 29020 consensus_meta.cc:287] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
I20260709 13:08:14.409129 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.382582 28707 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:14.411128 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:14.406163 28001 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:14.441385 28001 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.405576 28896 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:14.416499 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:14.417156 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.417873 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.449106 22585 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:14.418654 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:14.449204 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.449854 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.419991 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.418687 22589 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:14.365756 28652 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:08:14.422101 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.386240 28982 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:14.426574 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.459758 21860 raft_consensus.cc:2410] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:08:14.365806 29016 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:08:14.430126 29082 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:08:14.409065 29039 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.492585 29039 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:14.463954 28982 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:14.493099 21930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1248) took 3320 ms (client timeout 2140 ms). Trace:
W20260709 13:08:14.493891 21930 rpcz_store.cc:269] 0709 13:08:11.172784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.238012 (+ 65228us) service_pool.cc:224] Handling call
0709 13:08:14.492755 (+3254743us) spinlock_profiling.cc:243] Waited 2.77 s on lock 0x7b540003cd54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.492929 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6379617152}
I20260709 13:08:14.440264 29052 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.497373 29052 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:14.499444 28850 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.501447 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1415) took 1785 ms. Trace:
I20260709 13:08:14.501861 22592 rpcz_store.cc:276] 0709 13:08:12.715660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.219748 (+504088us) service_pool.cc:224] Handling call
0709 13:08:13.245704 (+ 25956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:13.832458 (+586754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.501146 (+668688us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54005dcad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:14.501381 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347887872}
W20260709 13:08:14.502620 29093 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:08:14.507593 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.508265 22000 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:14.509541 22004 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:08:14.509863 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.510959 29032 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.431836 28618 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.513417 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.514175 21860 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.515296 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.516098 21851 raft_consensus.cc:2376] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20. Re-sending same reply.
I20260709 13:08:14.433841 29049 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:14.517437 29049 raft_consensus.cc:697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.518464 29049 consensus_queue.cc:237] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.520442 21859 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:14.521103 28886 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.521586 28886 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.524073 28981 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.434548 29006 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.525095 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:14.525460 22586 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:14.527405 21853 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:08:14.528371 21855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3060) took 1928 ms (client timeout 2032 ms). Trace:
I20260709 13:08:14.357697 29010 consensus_queue.cc:1059] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:08:14.438517 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:14.444272 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.445689 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64250.4us
I20260709 13:08:14.452822 22593 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:08:14.454232 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:14.422917 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65712.2us
I20260709 13:08:14.454879 22584 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.564002 21858 raft_consensus.cc:2410] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:08:14.534016 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:14.535283 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.469890 28001 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.471015 29016 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:14.368116 28387 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.374819 28895 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:14.473403 29086 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:08:14.476583 29074 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:14.528970 21855 rpcz_store.cc:269] 0709 13:08:12.599992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.396832 (+796840us) service_pool.cc:224] Handling call
0709 13:08:14.528115 (+1131283us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b540010c554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.528291 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1049801984}
I20260709 13:08:14.479545 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:14.480018 21857 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:14.463063 28896 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.564088 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:14.476794 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.562659 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 194759us
I20260709 13:08:14.489655 29054 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:14.536803 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 117504us
I20260709 13:08:14.534634 22587 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:14.591954 22008 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:08:14.592528 22008 pending_rounds.cc:85] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 11
I20260709 13:08:14.592886 22008 pending_rounds.cc:107] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:08:14.593277 22008 raft_consensus.cc:2889] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:14.566618 22002 pending_rounds.cc:85] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 10
I20260709 13:08:14.593997 22002 pending_rounds.cc:107] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:08:14.594377 22002 raft_consensus.cc:2889] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:14.567134 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.581647 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.595395 21858 raft_consensus.cc:2376] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19. Re-sending same reply.
I20260709 13:08:14.595631 21825 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:14.596284 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:14.546680 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3044) took 1954 ms (client timeout 1596 ms). Trace:
W20260709 13:08:14.582710 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.550838 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.597532 21856 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.563436 21934 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: []
W20260709 13:08:14.598166 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.568315 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:14.437024 29085 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.586777 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.593741 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.584858 28001 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:14.536491 21852 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:14.559900 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.453722 28829 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:08:14.587397 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.563556 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:14.597093 21854 rpcz_store.cc:269] 0709 13:08:12.592315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.594058 (+  1743us) service_pool.cc:224] Handling call
0709 13:08:14.546449 (+1952391us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.546594 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967082368}
I20260709 13:08:14.599663 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:14.600180 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.584745 22007 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:14.608245 21851 raft_consensus.cc:2376] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23. Re-sending same reply.
I20260709 13:08:14.606200 22004 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:14.612226 22003 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:14.611941 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:14.613200 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.532982 29010 raft_consensus.cc:3055] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:14.614614 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.615134 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:14.615690 21858 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.538092 29052 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.529800 28982 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.606213 28750 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:14.637714 28750 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:14.635317 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.638579 28896 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.638741 28750 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.636090 28387 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:14.619494 28979 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:08:14.534140 29019 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.641734 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:14.642352 21931 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:14.643025 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:14.644459 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.645114 21934 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:14.645414 29048 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.634711 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.647112 21851 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:14.647212 28708 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:14.647931 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.636658 29016 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.642371 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.650213 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.650775 21856 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:14.648079 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.651562 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.651919 21851 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:14.626930 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.562580 29040 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.656239 21856 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:14.617724 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.657835 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:14.659067 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.661188 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.659806 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.662248 21855 raft_consensus.cc:2410] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:08:14.664130 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:14.658004 29032 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:14.549860 28981 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:14.641618 29089 log.cc:927] Time spent T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
I20260709 13:08:14.676245 22592 raft_consensus.cc:2368] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22.
I20260709 13:08:14.676592 22006 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:14.672856 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2721) took 1304 ms. Trace:
I20260709 13:08:14.677615 21856 rpcz_store.cc:276] 0709 13:08:13.356050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.655667 (+1299617us) service_pool.cc:224] Handling call
0709 13:08:14.656026 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.660342 (+  4316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.660589 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.659246 22003 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 17. (term mismatch)
I20260709 13:08:14.538565 28707 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.672250 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.678542 22003 pending_rounds.cc:85] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 16
I20260709 13:08:14.679009 21851 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:14.679950 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:14.680640 22003 pending_rounds.cc:107] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 19.17
I20260709 13:08:14.680943 22003 raft_consensus.cc:2889] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:14.613448 29022 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:08:14.682318 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 118285us
I20260709 13:08:14.613583 29085 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.657092 29010 raft_consensus.cc:740] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:14.670248 29074 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.692425 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:14.618563 29023 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:14.618155 29060 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.689339 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.689806 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.695197 22003 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:14.683211 22004 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:14.683758 21856 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:14.618257 28917 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:08:14.634303 28820 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:08:14.697999 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.699888 21855 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:14.693766 28981 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 13:08:14.702468 29060 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.676744 29040 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.716568 29085 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.717337 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2727) took 1359 ms. Trace:
I20260709 13:08:14.725267 21855 rpcz_store.cc:276] 0709 13:08:13.357903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.684072 (+1326169us) service_pool.cc:224] Handling call
0709 13:08:14.699646 (+ 15574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.717101 (+ 17455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.717285 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.726305 29060 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.705914 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.726680 29040 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.727140 22008 raft_consensus.cc:2393] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 14: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.677388 29019 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:08:14.727860 22005 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:14.719517 21974 leader_election.cc:400] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22.
I20260709 13:08:14.728541 21974 leader_election.cc:403] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:14.726718 21855 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:08:14.706419 28982 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.706477 28754 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:14.744616 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:14.722419 28917 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:14.745191 29077 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:14.745631 29077 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.745592 28917 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.746622 29077 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.674410 29013 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:08:14.749310 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2732) took 1389 ms. Trace:
I20260709 13:08:14.708902 28923 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:14.711351 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2726) took 1353 ms. Trace:
I20260709 13:08:14.754940 21853 rpcz_store.cc:276] 0709 13:08:13.360057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.826796 (+466739us) service_pool.cc:224] Handling call
0709 13:08:14.749070 (+922274us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b540010b154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.749229 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436226944}
I20260709 13:08:14.679551 29032 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:14.684477 29022 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:14.715478 21859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2730) took 1356 ms (client timeout 1570 ms). Trace:
W20260709 13:08:14.760888 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.760739 21859 rpcz_store.cc:269] 0709 13:08:13.358842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.469643 (+110801us) service_pool.cc:224] Handling call
0709 13:08:14.715241 (+1245598us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400017a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.715407 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326587776}
I20260709 13:08:14.705627 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:14.739054 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.762763 28001 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:14.763286 28001 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.716434 28989 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:14.708390 29052 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.691545 28652 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.752763 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2731) took 1370 ms. Trace:
I20260709 13:08:14.755091 21856 rpcz_store.cc:276] 0709 13:08:13.357624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.683330 (+1325706us) service_pool.cc:224] Handling call
0709 13:08:14.683611 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.696554 (+ 12943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.711243 (+ 14689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.769505 21855 rpcz_store.cc:276] 0709 13:08:13.359577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.726085 (+1366508us) service_pool.cc:224] Handling call
0709 13:08:14.726398 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.729564 (+  3166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.729796 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.712091 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.770617 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:14.771076 22003 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:14.756796 21853 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:14.717741 28820 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:14.762684 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 57173.4us
I20260709 13:08:14.768536 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:14.774469 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.774660 21859 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:14.775071 28001 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.776449 22003 raft_consensus.cc:2410] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:08:14.782315 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2733) took 1412 ms. Trace:
I20260709 13:08:14.782904 21853 rpcz_store.cc:276] 0709 13:08:13.360586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.755994 (+1395408us) service_pool.cc:224] Handling call
0709 13:08:14.756565 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.772225 (+ 15660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.772473 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.784140 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2734) took 1415 ms. Trace:
I20260709 13:08:14.784617 21859 rpcz_store.cc:276] 0709 13:08:13.361172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.762135 (+1400963us) service_pool.cc:224] Handling call
0709 13:08:14.774368 (+ 12233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.775900 (+  1532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.776152 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.752972 28754 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.765333 29040 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:08:14.765986 29022 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:14.771878 29077 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:14.799274 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:14.801047 21855 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:14.805174 22592 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:14.804479 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.768859 29019 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:08:14.807873 29006 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:14.814064 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.811077 29006 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.816476 29006 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:14.817605 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.818185 22586 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:14.819428 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.820640 21855 raft_consensus.cc:2393] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:14.822356 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.823297 21972 connection.cc:256] RPC call timeout handler was delayed by -1.2373991869999372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
I20260709 13:08:14.839113 29041 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:14.839586 29041 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:14.825296 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.840289 28886 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.782481 28652 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.840436 29041 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.841835 29041 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:14.843871 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.844520 29059 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.845402 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:14.845894 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.849264 28979 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:14.781248 29063 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:08:14.853627 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:14.860632 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.861182 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.863039 28979 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:14.863446 28979 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:14.863551 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.863646 29020 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:14.864454 29020 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:14.865648 29020 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:14.866943 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3048) took 2270 ms (client timeout 1706 ms). Trace:
I20260709 13:08:14.867240 29020 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:14.867376 21852 rpcz_store.cc:269] 0709 13:08:12.596271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.026585 (+430314us) service_pool.cc:224] Handling call
0709 13:08:14.866757 (+1840172us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400259c54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.866860 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3469809536}
I20260709 13:08:14.870021 29039 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:14.863881 28979 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:14.864708 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.844724 21860 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.868875 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.878964 21851 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:14.879221 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.883495 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:14.810002 29043 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:14.886943 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.892187 21851 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:14.887162 22008 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:08:14.887693 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.892529 22008 pending_rounds.cc:85] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 15
I20260709 13:08:14.892791 22008 pending_rounds.cc:107] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:08:14.892767 21860 raft_consensus.cc:2376] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15. Re-sending same reply.
I20260709 13:08:14.893193 22008 raft_consensus.cc:2889] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:14.889694 21931 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:14.894071 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.894824 22003 raft_consensus.cc:2410] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:08:14.900110 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.778733 29010 consensus_queue.cc:260] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:14.901106 21852 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:14.902618 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.902935 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:14.905747 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.903739 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.907157 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.811906 29074 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:08:14.863454 29019 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:14.913112 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.914355 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.843914 28652 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:14.916719 21852 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:14.872498 28982 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 13:08:14.883893 29063 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:08:14.918473 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.901844 29007 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.927064 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.914142 28754 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:14.937589 29102 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:08:14.902581 29006 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:08:14.905289 28917 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:14.939992 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:14.950404 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2725) took 1593 ms. Trace:
I20260709 13:08:14.914975 29017 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:08:14.955312 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.977191 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.995242 21851 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:14.952972 29019 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:14.996699 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:14.997357 21860 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:14.998625 29019 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.973958 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:14.961701 28707 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.975098 22001 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:14.981519 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:14.978636 29052 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:14.957849 28895 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:14.958678 29041 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:14.984021 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:14.992883 22000 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:14.992827 21857 rpcz_store.cc:276] 0709 13:08:13.357358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.056360 (+699002us) service_pool.cc:224] Handling call
0709 13:08:14.950114 (+893754us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540008b1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:14.950319 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948797824}
I20260709 13:08:15.008469 29039 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:15.009586 22003 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:14.985882 29013 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:14.887359 29032 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.001454 29007 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:15.010113 29040 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22.
I20260709 13:08:15.016053 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:15.050359 22008 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:15.030290 22587 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:15.029848 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48369.5us
I20260709 13:08:15.050745 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:15.034308 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:15.061901 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 159429us
I20260709 13:08:15.063123 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.064078 21853 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:08:15.035494 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:15.036262 22000 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:15.065575 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:15.069130 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:15.048084 21851 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:15.033303 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:15.009625 29105 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:08:15.029697 21860 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:15.014169 29017 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:15.069959 21859 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:15.084462 21929 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:15.033428 29016 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.076406 21856 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:15.039740 28895 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:15.075982 28896 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.086479 29022 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.087047 28895 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:15.088068 28895 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.090322 21856 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:15.075176 21851 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:15.093191 29040 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:15.065065 28842 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:15.083840 28652 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:15.099973 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.100337 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:15.100970 22002 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
W20260709 13:08:15.105161 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 517787us
I20260709 13:08:15.105958 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.039379 29007 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:15.107064 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.107698 21852 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:15.093585 22004 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:08:15.110636 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47622.7us
I20260709 13:08:15.118424 21860 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:15.118320 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:15.123234 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 559701us
I20260709 13:08:15.126513 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.127862 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.128700 21855 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:15.131587 22006 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:15.132151 22556 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:15.133793 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.135856 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:15.147114 21860 raft_consensus.cc:1240] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:08:15.149134 21852 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:15.162267 21859 raft_consensus.cc:1275] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:08:15.162765 21859 pending_rounds.cc:85] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 15
I20260709 13:08:15.163025 21859 pending_rounds.cc:107] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:08:15.163311 21859 raft_consensus.cc:2889] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:15.164821 21899 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:15.053392 29013 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.078042 29006 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:08:15.130340 29039 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:15.130820 29052 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:15.168354 21899 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:15.137498 28896 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:15.137352 29011 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.188373 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:15.145848 29048 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:15.136899 29097 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.164239 28708 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:15.138862 28829 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:15.138187 28652 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.138337 29060 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:15.145780 28850 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:15.145527 29043 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:15.160826 28900 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.199950 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:15.200775 29015 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:15.171245 29023 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:15.207250 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 81912.4us
W20260709 13:08:15.208209 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 101191us
W20260709 13:08:15.214694 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2657) took 1555 ms (client timeout 1953 ms). Trace:
I20260709 13:08:15.184731 29039 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:15.191610 28919 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:15.196353 28896 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:15.228469 21929 rpcz_store.cc:269] 0709 13:08:13.661417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.668779 (+  7362us) service_pool.cc:224] Handling call
0709 13:08:15.210144 (+1541365us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54004290d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:15.210274 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087206272}
I20260709 13:08:15.228797 21900 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:15.206875 28708 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:15.232533 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.202453 29040 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 13:08:15.242553 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74289.3us
I20260709 13:08:15.250248 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.212611 29048 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:15.259665 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:15.261984 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 199270us
I20260709 13:08:15.206019 29011 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.227739 29015 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.244879 28886 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:15.231439 29023 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:15.245308 28896 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:15.255323 28708 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:15.218590 28652 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.254846 29097 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:15.326225 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 76055.7us
W20260709 13:08:15.289444 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 153662us
I20260709 13:08:15.330783 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.334710 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.210544 29017 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.207053 29007 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.337658 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:15.285970 29015 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:15.243083 28900 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.290432 29023 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:15.378517 21859 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:15.295018 28886 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:15.374863 29092 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:08:15.383062 29052 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.287294 28982 consensus_queue.cc:1059] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.106s
I20260709 13:08:15.342166 28919 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:15.395589 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56761.6us
W20260709 13:08:15.386502 28989 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
I20260709 13:08:15.452122 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.341952 28850 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.375779 29053 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:15.387151 29111 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:08:15.289912 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1257) took 1051 ms. Trace:
I20260709 13:08:15.292754 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:15.290268 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61560.1us
I20260709 13:08:15.333264 29095 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:15.293895 28750 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.456739 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:15.458034 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 157479us
I20260709 13:08:15.339766 29085 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:15.394670 29049 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.396289 28886 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:15.393616 29116 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.386354 29007 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:15.533727 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.461862 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:15.541169 21859 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:15.511632 28979 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.002s
W20260709 13:08:15.538905 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 82256.3us
I20260709 13:08:15.518770 21926 rpcz_store.cc:276] 0709 13:08:14.238751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.238890 (+   139us) service_pool.cc:224] Handling call
0709 13:08:14.239330 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:14.246650 (+  7320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.246698 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:14.264917 (+ 18219us) log.cc:844] Serialized 52 byte log entry
0709 13:08:14.436369 (+171452us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:14.462512 (+ 26143us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.091696 (+629184us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b18004f7b60. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:08:15.261755 (+170059us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.289468 (+ 27713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.289540 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:15.289648 (+   108us) raft_consensus.cc:1685] finished
0709 13:08:15.289667 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.289844 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32662,"prepare.queue_time_us":383998,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":694913,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":21543,"spinlock_wait_cycles":156790656,"thread_start_us":292731,"threads_started":2,"wal-append.queue_time_us":257448}
I20260709 13:08:15.342458 28844 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:15.522109 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 186448us
I20260709 13:08:15.421399 29054 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:15.532248 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 239457us
W20260709 13:08:15.543630 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.543444 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.544303 22587 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:15.461484 28918 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:15.558087 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:15.532606 29102 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:08:15.561014 22586 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:15.568629 22593 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:15.570288 28001 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:15.571995 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:15.544885 29112 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:08:15.378868 29039 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.384446 29043 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.525604 29050 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:15.529778 29085 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:15.554651 28807 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:15.581305 28895 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.450646 29052 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.513495 28850 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.603819 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:15.642887 22003 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:15.607548 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.613921 21926 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:08:15.643733 21926 pending_rounds.cc:85] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:08:15.644049 21926 pending_rounds.cc:107] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:08:15.644345 21926 raft_consensus.cc:2889] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:15.548888 29053 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:15.670174 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.622478 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.556581 29013 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:15.562474 28754 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.598271 28001 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:15.644424 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.685386 21931 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:15.573822 28982 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:15.534690 28750 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:15.646921 22008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.004s	sys 0.000s
I20260709 13:08:15.648064 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:15.697082 22008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.001s
I20260709 13:08:15.635003 29085 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:15.557029 28979 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.544569 29135 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.672639 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.671538 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.536615 29049 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:15.636132 29114 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 13:08:15.684412 28001 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.615445 28869 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.511511 29048 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:15.668025 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.728897 21929 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:15.704246 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:15.747682 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:15.748355 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.752498 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1419) took 1072 ms. Trace:
I20260709 13:08:15.755033 22004 rpcz_store.cc:276] 0709 13:08:14.680400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.887834 (+207434us) service_pool.cc:224] Handling call
0709 13:08:15.093256 (+205422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.117910 (+ 24654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.752235 (+634325us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400425ed0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:15.752422 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431812480}
I20260709 13:08:15.756915 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:15.757689 22004 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:15.692140 29013 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:15.709859 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50337.2us
I20260709 13:08:15.709853 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:15.696760 29014 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.614607 29113 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 13:08:15.726836 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:15.711118 29102 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:08:15.773093 21933 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:08:15.547055 29116 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.778086 21929 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:08:15.637267 29050 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.732157 29135 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.671017 28895 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.800765 21928 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:15.807087 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1430) took 1085 ms. Trace:
I20260709 13:08:15.810458 22584 rpcz_store.cc:276] 0709 13:08:14.721234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.721549 (+   315us) service_pool.cc:224] Handling call
0709 13:08:14.721993 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:14.722554 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:14.722599 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:14.759317 (+ 36718us) log.cc:844] Serialized 26 byte log entry
0709 13:08:14.994931 (+235614us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:14.994976 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.995044 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:15.806838 (+811794us) raft_consensus.cc:1685] finished
0709 13:08:15.806863 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.807020 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":196131,"threads_started":1,"wal-append.queue_time_us":333221}
I20260709 13:08:15.668455 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:15.828974 28001 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:15.829600 28001 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.830099 28001 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:15.844717 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2677) took 1887 ms. Trace:
I20260709 13:08:15.833112 28652 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:15.845680 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2737) took 1035 ms. Trace:
W20260709 13:08:15.849788 28842 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.001s
I20260709 13:08:15.845271 21927 rpcz_store.cc:276] 0709 13:08:13.957353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.970832 (+ 13479us) service_pool.cc:224] Handling call
0709 13:08:14.006490 (+ 35658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:14.047271 (+ 40781us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:14.142586 (+ 95315us) log.cc:844] Serialized 14 byte log entry
0709 13:08:14.489598 (+347012us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b18002d9f40. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:08:15.385907 (+896309us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.397572 (+ 11665us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.462040 (+ 64468us) log.cc:844] Serialized 26 byte log entry
0709 13:08:15.462222 (+   182us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:15.462257 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.462329 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:15.844496 (+382167us) raft_consensus.cc:1685] finished
0709 13:08:15.844518 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.844653 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":476882,"prepare.queue_time_us":494456,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2459,"raft.queue_time_us":476692,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":47944,"spinlock_wait_cycles":3572992,"thread_start_us":480202,"threads_started":3,"wal-append.queue_time_us":599227}
I20260709 13:08:15.851437 22588 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:15.844837 29040 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:08:15.899147 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 251139us
W20260709 13:08:15.831718 29114 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:08:15.846115 21857 rpcz_store.cc:276] 0709 13:08:14.809903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.053464 (+243561us) service_pool.cc:224] Handling call
0709 13:08:15.076582 (+ 23118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.199502 (+122920us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:15.219679 (+ 20177us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.538747 (+319068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:15.538814 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:15.538860 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.845459 (+306599us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000fbc54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:15.845507 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.845625 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39186,"prepare.queue_time_us":14439,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":178292,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":14976,"spinlock_wait_cycles":282027392,"thread_start_us":277349,"threads_started":1,"wal-append.queue_time_us":573481}
I20260709 13:08:15.913722 22004 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:15.916406 28001 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:15.919888 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.922150 21972 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:15.925619 21928 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:15.898729 29040 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.851064 28842 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:15.829890 28678 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:15.853152 29060 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.797359 29014 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:15.928295 29040 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:15.832484 28900 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:15.973734 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.829613 29011 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:15.791359 29013 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:15.912117 29115 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.498s	user 0.004s	sys 0.000s
I20260709 13:08:15.762835 28869 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:16.016685 22002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.014s	sys 0.022s
I20260709 13:08:16.018325 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:16.038578 22002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.015s	sys 0.022s
I20260709 13:08:16.042002 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3279) took 1278 ms. Trace:
I20260709 13:08:16.021970 29121 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:15.986419 28842 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:16.031728 29133 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 13:08:16.011068 29153 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.033599 28896 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.697s	user 0.009s	sys 0.002s
W20260709 13:08:16.034219 29132 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 13:08:16.035097 29137 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:08:16.042476 22002 rpcz_store.cc:276] 0709 13:08:14.763748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.101961 (+338213us) service_pool.cc:224] Handling call
0709 13:08:15.102332 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.102591 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:15.431445 (+328854us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.040738 (+609293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:16.041330 (+   592us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:16.041615 (+   285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.041821 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.041939 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28901,"prepare.queue_time_us":613129,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":10047,"thread_start_us":849575,"threads_started":2}
I20260709 13:08:16.035158 28917 consensus_queue.cc:1059] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:16.033247 29116 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.022908 28900 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.060514 29138 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:08:16.046921 28678 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.096663 29013 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.096834 28678 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.099211 28678 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.100975 28917 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:16.103025 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3277) took 1379 ms. Trace:
I20260709 13:08:16.103472 22005 rpcz_store.cc:276] 0709 13:08:14.724165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.040727 (+316562us) service_pool.cc:224] Handling call
0709 13:08:15.066051 (+ 25324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.066337 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.066376 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.091558 (+ 25182us) log.cc:844] Serialized 26 byte log entry
0709 13:08:15.394144 (+302586us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:15.394195 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.394291 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.102785 (+708494us) raft_consensus.cc:1685] finished
0709 13:08:16.102808 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.102959 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":215686,"threads_started":1,"wal-append.queue_time_us":527438}
I20260709 13:08:16.012204 29060 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.105454 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:16.042606 29155 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.100775 21859 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.001s
I20260709 13:08:16.122412 21859 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:16.106998 29052 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.139650 28979 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.140980 28750 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.186122 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:16.206184 21859 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:16.073743 28896 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.124810 28850 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.221834 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:16.226913 21859 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:16.065935 29153 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.115496 28900 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.130291 29155 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.288586 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3221) took 1728 ms. Trace:
I20260709 13:08:16.387368 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2750) took 1090 ms. Trace:
W20260709 13:08:16.424438 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 403913us
W20260709 13:08:16.428896 22588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57084 (request call id 1370) took 2295 ms (client timeout 1856 ms). Trace:
I20260709 13:08:16.262403 22587 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 21. Current term is 22. Ops: []
W20260709 13:08:16.470872 22588 rpcz_store.cc:269] 0709 13:08:14.133180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.134036 (+   856us) service_pool.cc:224] Handling call
0709 13:08:16.428639 (+2294603us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54004c2f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:16.428794 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3910412160}
I20260709 13:08:16.437237 21860 rpcz_store.cc:276] 0709 13:08:15.296463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.296626 (+   163us) service_pool.cc:224] Handling call
0709 13:08:15.297055 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.297431 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.297472 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.328142 (+ 30670us) log.cc:844] Serialized 26 byte log entry
0709 13:08:15.542470 (+214328us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:15.542510 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.542577 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.387111 (+844534us) raft_consensus.cc:1685] finished
0709 13:08:16.387133 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.387275 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24550,"raft.queue_time_us":159231,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":562,"thread_start_us":147974,"threads_started":1,"wal-append.queue_time_us":869364}
I20260709 13:08:16.496052 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3104) took 4517 ms. Trace:
I20260709 13:08:16.499871 21860 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 15 index: 16. (index mismatch)
I20260709 13:08:16.498960 22009 rpcz_store.cc:276] 0709 13:08:11.909355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.154615 (+245260us) service_pool.cc:224] Handling call
0709 13:08:12.167228 (+ 12613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.524122 (+3356894us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:15.524178 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:15.524245 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:15.524316 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.426631 (+902315us) spinlock_profiling.cc:243] Waited 3.29 s on lock 0x7b540033e5d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:16.426688 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.426927 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62507,"raft.queue_time_us":68774,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":8534,"spinlock_wait_cycles":7575221632}
W20260709 13:08:16.525473 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.533475 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.435508 21930 rpcz_store.cc:276] 0709 13:08:14.560113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.562845 (+  2732us) service_pool.cc:224] Handling call
0709 13:08:14.589533 (+ 26688us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:15.123343 (+533810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.123387 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:15.182084 (+ 58697us) log.cc:844] Serialized 52 byte log entry
0709 13:08:15.408825 (+226741us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:15.408870 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.408964 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.288227 (+879263us) raft_consensus.cc:1685] finished
0709 13:08:16.288256 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.288503 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":545481,"raft.queue_time_us":525838,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":20084,"thread_start_us":207529,"threads_started":1,"wal-append.queue_time_us":508933}
W20260709 13:08:16.564062 29162 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 13:08:16.564538 28387 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:16.564971 28387 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:16.351536 28708 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.014s	user 0.013s	sys 0.000s
I20260709 13:08:16.574218 28708 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:16.383929 28919 consensus_meta.cc:287] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.924s	user 0.007s	sys 0.003s
W20260709 13:08:16.577155 29169 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 13:08:16.577409 28919 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:16.577852 29166 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 13:08:16.389415 29015 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.003s	user 0.009s	sys 0.002s
W20260709 13:08:16.257478 29077 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.455s	user 0.009s	sys 0.005s
W20260709 13:08:16.286437 29007 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.000s
W20260709 13:08:16.385182 29146 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 13:08:16.579032 29168 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:08:16.579388 29015 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.581789 29015 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.262099 28896 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:16.574685 29129 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.020s	user 0.010s	sys 0.000s
I20260709 13:08:16.263446 29059 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.323s
I20260709 13:08:16.586164 29160 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.187036 29052 raft_consensus.cc:3055] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:16.587486 29052 raft_consensus.cc:740] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:16.589188 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 90582.7us
I20260709 13:08:16.571581 28387 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:16.582089 29023 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.159s	user 0.009s	sys 0.003s
I20260709 13:08:16.591035 28900 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.591050 28387 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:16.591243 29023 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.592928 22592 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:16.596267 28896 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.597644 21857 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 22 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:16.597234 28896 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.242820 29158 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:16.269265 28842 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.270334 28869 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.344930 28895 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.287715 28850 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:16.586753 29160 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:16.571661 29040 consensus_meta.cc:287] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.005s
I20260709 13:08:16.588796 29164 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.584321 28708 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.174849 29121 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:16.254340 29172 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.389199 29085 consensus_meta.cc:287] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.664s	user 0.005s	sys 0.006s
I20260709 13:08:16.588595 29052 consensus_queue.cc:260] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:16.618671 29095 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.083s	user 0.009s	sys 0.002s
I20260709 13:08:16.677400 29095 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:16.663429 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.058s	sys 0.041s
I20260709 13:08:16.607297 29158 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.678584 29158 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:16.679247 29155 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.606215 29116 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:16.680874 29116 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.613235 29014 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.257493 29173 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.400s
I20260709 13:08:16.685820 29116 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:16.685966 29023 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:16.665210 28850 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.687071 29118 raft_consensus.cc:3055] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:16.687557 29118 raft_consensus.cc:740] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:16.687706 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.058s	sys 0.041s
I20260709 13:08:16.685726 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1553) took 1144 ms. Trace:
W20260709 13:08:16.690584 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 582391us
I20260709 13:08:16.690373 22592 rpcz_store.cc:276] 0709 13:08:15.540934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.541151 (+   217us) service_pool.cc:224] Handling call
0709 13:08:15.548092 (+  6941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:16.593410 (+1045318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.685489 (+ 92079us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54006266d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:16.685669 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2402279808}
I20260709 13:08:16.652582 28842 consensus_queue.cc:1048] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:08:16.652244 29040 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.677207 29148 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.719924 29154 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.094s	user 0.006s	sys 0.000s
I20260709 13:08:16.954754 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1126) took 1625 ms. Trace:
I20260709 13:08:16.952198 29149 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.762286 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 287537us
W20260709 13:08:16.751953 29136 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.092s	user 0.000s	sys 0.007s
W20260709 13:08:16.778426 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.987528 21859 rpcz_store.cc:276] 0709 13:08:15.329342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.241933 (+912591us) service_pool.cc:224] Handling call
0709 13:08:16.242392 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:16.252190 (+  9798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.252248 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:16.254491 (+  2243us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.620507 (+366016us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:16.620544 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.620650 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.954486 (+333836us) raft_consensus.cc:1685] finished
0709 13:08:16.954510 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.954680 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9343,"raft.queue_time_us":12438,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":365566,"threads_started":1,"wal-append.queue_time_us":698351}
W20260709 13:08:16.951902 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.673425 29172 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.989209 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:16.990466 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:16.990832 29172 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:16.991084 22593 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:16.959184 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3285) took 2015 ms. Trace:
I20260709 13:08:16.949139 29048 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:16.991556 22003 rpcz_store.cc:276] 0709 13:08:14.943377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.805084 (+861707us) service_pool.cc:224] Handling call
0709 13:08:15.810558 (+  5474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.810919 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.810950 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.811428 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.208511 (+397083us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:16.208551 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.208615 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.958939 (+750324us) raft_consensus.cc:1685] finished
0709 13:08:16.958961 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.959101 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81330,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":19814,"thread_start_us":210925,"threads_started":1,"wal-append.queue_time_us":1055416}
I20260709 13:08:16.759774 29121 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:16.993775 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:16.994403 22588 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:16.994800 29060 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.962371 29153 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.671584 29164 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:16.773307 29167 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:08:16.706149 29052 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:16.996404 29163 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.787s	user 0.000s	sys 0.007s
W20260709 13:08:17.000072 29125 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.435s	user 0.011s	sys 0.006s
I20260709 13:08:17.001346 21852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2749) took 1705 ms. Trace:
I20260709 13:08:17.001609 29149 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.001772 21852 rpcz_store.cc:276] 0709 13:08:15.295672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.295843 (+   171us) service_pool.cc:224] Handling call
0709 13:08:15.334339 (+ 38496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:15.648950 (+314611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.648992 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:15.649699 (+   707us) log.cc:844] Serialized 52 byte log entry
0709 13:08:15.666171 (+ 16472us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:15.686859 (+ 20688us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.712871 (+ 26012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.712966 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.683868 (+970902us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b18003f5500. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:17.001067 (+317199us) raft_consensus.cc:1685] finished
0709 13:08:17.001163 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.001286 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38435,"prepare.queue_time_us":17269,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":405434,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":43391,"spinlock_wait_cycles":21598976,"thread_start_us":323831,"threads_started":2,"wal-append.queue_time_us":611224}
I20260709 13:08:16.673246 29085 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.004451 29149 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.986196 29041 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.005126 29085 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.005584 29041 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:16.944235 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 335267us
I20260709 13:08:17.007684 29041 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:16.956527 28850 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:16.949216 29160 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.008536 29022 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:17.008978 29160 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.001369 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3223) took 2440 ms. Trace:
I20260709 13:08:16.974681 21851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1105) took 2117 ms. Trace:
I20260709 13:08:16.977500 29135 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:16.982056 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:17.015880 21974 connection.cc:256] RPC call timeout handler was delayed by -1.1842286639999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 13:08:17.010530 21935 rpcz_store.cc:276] 0709 13:08:14.560778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.560892 (+   114us) service_pool.cc:224] Handling call
0709 13:08:14.561313 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.102589 (+541276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.102623 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.103102 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:08:15.214887 (+111785us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:15.214928 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.214995 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.999433 (+1784438us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:17.001096 (+  1663us) raft_consensus.cc:1685] finished
0709 13:08:17.001117 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.001316 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":540855,"raft.queue_time_us":542757,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":111609600,"thread_start_us":111410,"threads_started":1,"wal-append.queue_time_us":305337}
I20260709 13:08:17.013152 21851 rpcz_store.cc:276] 0709 13:08:14.857286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.112309 (+255023us) service_pool.cc:224] Handling call
0709 13:08:15.112575 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.112789 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:08:15.147549 (+ 34760us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.974082 (+1826533us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b18003ff1c0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287
0709 13:08:16.974227 (+   145us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:16.974310 (+    83us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:16.974381 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.974461 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.974614 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47374,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":113739,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":39757696,"thread_start_us":663201,"threads_started":2,"wal-append.queue_time_us":661695}
I20260709 13:08:16.951398 28896 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:16.950068 28886 consensus_meta.cc:287] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.403s	user 0.017s	sys 0.005s
I20260709 13:08:16.955510 29158 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.985574 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.034894 22584 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 23: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:16.986372 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:16.988268 29148 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:16.988181 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:17.037123 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49060.3us
I20260709 13:08:17.038332 29022 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:17.038615 29176 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:17.039048 29022 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:16.751184 29118 consensus_queue.cc:260] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.039955 29022 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:17.040836 29074 consensus_queue.cc:1175] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34f3b83da84448bf8aac62b37a1486eb" responder_term: 19 status { last_received { term: 19 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 19 index: 15 } }
I20260709 13:08:17.041890 29118 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.042239 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.009473 29160 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.018870 29116 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.019450 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:17.019973 29041 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.045334 29160 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.045929 29160 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:17.045763 29041 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.046312 29160 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:17.046504 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.045616 29116 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.047390 29063 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:17.020018 29077 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.047806 29063 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.048266 29063 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:17.048087 29077 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.048475 29116 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.049886 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:17.050233 29077 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.050609 21851 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:17.047829 29041 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.031692 22000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3278) took 2280 ms. Trace:
I20260709 13:08:17.051419 22000 rpcz_store.cc:276] 0709 13:08:14.751700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.065568 (+313868us) service_pool.cc:224] Handling call
0709 13:08:15.086059 (+ 20491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:16.228613 (+1142554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.228650 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:16.245055 (+ 16405us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.613612 (+368557us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:16.613670 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.031122 (+417452us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400167954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:17.031195 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.031265 (+    70us) raft_consensus.cc:1685] finished
0709 13:08:17.031285 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.031602 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1693,"raft.queue_time_us":4460,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7500,"spinlock_wait_cycles":2623520000,"thread_start_us":338547,"threads_started":1,"wal-append.queue_time_us":431986}
I20260709 13:08:17.035569 22592 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:17.046916 29160 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:17.054191 21852 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:17.057312 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:17.057837 21852 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.057906 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.058379 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:17.058769 21860 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:17.059633 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.060177 21860 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:17.061553 29059 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.062212 29059 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.062662 28850 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:17.063372 28850 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.064517 29059 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.064653 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.064944 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.065591 22592 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:17.065846 28850 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:17.065322 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.065927 22593 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:17.067046 29059 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:17.067811 29059 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:17.068267 29059 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:17.075798 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.076630 22588 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:17.083945 21851 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:17.085638 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1554) took 1515 ms. Trace:
I20260709 13:08:17.100517 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.102489 22587 rpcz_store.cc:276] 0709 13:08:15.570224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.570368 (+   144us) service_pool.cc:224] Handling call
0709 13:08:15.570683 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:16.556328 (+985645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.085384 (+529056us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b540060d6d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:17.085577 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590262144}
I20260709 13:08:17.103106 21974 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:17.087339 29124 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.560s	user 0.002s	sys 0.014s
I20260709 13:08:17.088868 29148 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:17.089717 21852 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:17.086987 29158 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.098165 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.101732 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3286) took 2142 ms. Trace:
I20260709 13:08:17.101701 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.104183 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:17.089768 29119 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.799s	user 0.010s	sys 0.007s
I20260709 13:08:17.105861 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.075943 29048 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.110549 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.118093 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.107103 29041 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:17.075344 28886 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.108780 29118 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:17.117359 29175 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:08:17.118660 29019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.793s	user 0.012s	sys 0.030s
I20260709 13:08:17.112811 22002 rpcz_store.cc:276] 0709 13:08:14.958872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.085648 (+1126776us) service_pool.cc:224] Handling call
0709 13:08:16.086438 (+   790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:16.086736 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:16.145422 (+ 58686us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.991623 (+846201us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b1800412720. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:08:17.101411 (+109788us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:17.101476 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:17.101516 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.101573 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.101676 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2599,"prepare.queue_time_us":51595,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6985,"raft.queue_time_us":3091,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8343,"spinlock_wait_cycles":95675136,"thread_start_us":106947,"threads_started":1}
I20260709 13:08:17.117378 28896 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:17.119822 29130 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:17.130929 28896 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.131512 28896 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.131705 29164 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.133244 29050 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.133605 28886 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.120177 29048 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.134465 22001 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:17.122907 29041 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.135470 29041 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.139345 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2751) took 1840 ms. Trace:
I20260709 13:08:17.127393 29175 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.141639 29175 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:17.142926 29049 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.134140 29130 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:17.145833 29130 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.109694 29063 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.149809 29040 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.150396 29010 raft_consensus.cc:3060] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:17.151028 29063 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.152478 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.153682 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.123446 29060 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:17.154657 21853 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:08:17.154872 29060 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:17.153792 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.155835 29060 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.156234 29059 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.156661 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.158075 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:17.145828 29122 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.685s	user 0.000s	sys 0.012s
I20260709 13:08:17.126138 29118 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.161347 29118 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:17.155480 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.170526 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.162072 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:17.162532 22006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 1422) took 2453 ms (client timeout 1779 ms). Trace:
W20260709 13:08:17.165388 21854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41266 (request call id 3070) took 2873 ms (client timeout 2098 ms). Trace:
W20260709 13:08:17.173089 21854 rpcz_store.cc:269] 0709 13:08:14.292009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.607741 (+315732us) service_pool.cc:224] Handling call
0709 13:08:17.165191 (+2557450us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b5400017054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:17.165320 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547546496}
I20260709 13:08:17.173370 21972 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:17.168195 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.139864 21855 rpcz_store.cc:276] 0709 13:08:15.298480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.740848 (+442368us) service_pool.cc:224] Handling call
0709 13:08:15.741252 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:16.041602 (+300350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.041634 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:16.041683 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:16.041724 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.138988 (+1097264us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54003d2a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:17.139056 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.139261 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67901,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":27073,"spinlock_wait_cycles":681867648}
I20260709 13:08:17.175563 21859 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:08:17.172115 22006 rpcz_store.cc:269] 0709 13:08:14.708966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.753465 (+ 44499us) service_pool.cc:224] Handling call
0709 13:08:17.162324 (+2408859us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540038a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:17.162436 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696349952}
I20260709 13:08:17.180032 22587 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:08:17.128870 29019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.841s	user 0.013s	sys 0.031s
I20260709 13:08:17.180922 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.182053 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.180045 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.167299 28896 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.182423 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:17.182796 22592 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:17.184101 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:17.184784 21853 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:17.182288 22593 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:17.193341 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.195796 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:17.196524 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.196763 21855 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:17.198213 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.200918 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.201437 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.201692 22004 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:08:17.202291 22002 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:17.203940 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.204109 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.204618 22587 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:17.176198 29063 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.205560 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:17.207782 22004 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:17.208298 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.208822 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.212893 21855 raft_consensus.cc:2468] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:17.217993 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.195474 28989 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:17.208910 21854 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:17.236972 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.238212 22592 raft_consensus.cc:2410] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:08:17.216492 29063 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.206014 29158 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:17.243110 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.243923 22000 raft_consensus.cc:2410] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:08:17.204340 28754 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.245252 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.249918 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.250963 22592 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:17.253113 22004 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
W20260709 13:08:17.253808 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.140s (SENT)
I20260709 13:08:17.260548 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.265856 22005 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:17.268673 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.269543 22003 raft_consensus.cc:2393] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:17.271237 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.273757 21853 raft_consensus.cc:2468] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:17.270766 29164 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:17.274755 29164 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.275110 29164 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:17.275701 29039 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.275061 28984 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:17.277297 28984 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.277788 28984 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:17.279923 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.284054 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2755) took 1241 ms. Trace:
I20260709 13:08:17.284471 21857 rpcz_store.cc:276] 0709 13:08:16.042199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.061530 (+ 19331us) service_pool.cc:224] Handling call
0709 13:08:17.283842 (+1222312us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540008b1d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:17.283995 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104818944}
I20260709 13:08:17.290417 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3225) took 2709 ms. Trace:
I20260709 13:08:17.291349 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.291843 22003 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 15 index: 16. (index mismatch)
I20260709 13:08:17.240664 29041 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.256513 29052 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.297986 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.300006 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.272089 22009 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:17.303674 21972 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.290868 21934 rpcz_store.cc:276] 0709 13:08:14.581143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.645929 (+ 64786us) service_pool.cc:224] Handling call
0709 13:08:14.646294 (+   365us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:08:15.164091 (+517797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.164121 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:08:15.186182 (+ 22061us) log.cc:844] Serialized 78 byte log entry
0709 13:08:15.329290 (+143108us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:15.536663 (+207373us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.544947 (+  8284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.023853 (+1478906us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b1800486160. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:17.290033 (+266180us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b54003552d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:17.290087 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.290135 (+    48us) raft_consensus.cc:1685] finished
0709 13:08:17.290150 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.290357 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":544671,"prepare.queue_time_us":266777,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":449276,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":22526,"spinlock_wait_cycles":312654336,"thread_start_us":267530,"threads_started":2,"wal-append.queue_time_us":392143}
I20260709 13:08:17.352658 28001 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:17.293900 29190 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:08:17.353533 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49961.8us
I20260709 13:08:17.361377 22592 raft_consensus.cc:2468] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:17.359792 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.348670 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.362337 22003 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
W20260709 13:08:17.362470 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.364040 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:17.301316 29154 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:17.367300 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3113) took 2692 ms. Trace:
I20260709 13:08:17.368160 21853 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:17.369445 22000 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:17.369407 21974 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.367693 22007 rpcz_store.cc:276] 0709 13:08:14.667977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.669071 (+  1094us) service_pool.cc:224] Handling call
0709 13:08:15.004565 (+335494us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:15.004959 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:15.005363 (+   404us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:15.030703 (+ 25340us) log.cc:844] Serialized 52 byte log entry
0709 13:08:15.212168 (+181465us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:15.212235 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.212311 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.145143 (+1932832us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:17.161610 (+ 16467us) raft_consensus.cc:1685] finished
0709 13:08:17.161628 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.359912 (+198284us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b540045b5d0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:17.360041 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26132,"raft.queue_time_us":229,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9203,"spinlock_wait_cycles":1123051136,"thread_start_us":122597,"threads_started":1,"wal-append.queue_time_us":274308}
I20260709 13:08:17.372062 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.373150 21854 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 24: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:17.376611 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.307687 28989 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:17.379905 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:17.381484 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 113840us
I20260709 13:08:17.383973 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.385059 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:17.261783 28754 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:17.393432 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 125621us
I20260709 13:08:17.366750 29158 raft_consensus.cc:2804] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:17.394420 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:17.403175 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.363288 29147 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:08:17.373447 29040 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:17.353848 29052 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.381747 29077 raft_consensus.cc:2804] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:17.417027 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.417563 22592 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:17.418467 22002 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
W20260709 13:08:17.427397 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.405573 29063 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.413899 29130 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:17.417212 29158 raft_consensus.cc:697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:17.454631 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:17.395609 28989 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.430101 28981 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:17.461598 28981 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:17.461961 29041 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.462793 28754 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.401171 28842 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:08:17.463523 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.464138 22588 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.464264 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1424) took 2717 ms. Trace:
I20260709 13:08:17.458197 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.434312 29077 raft_consensus.cc:697] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:17.460934 29192 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:08:17.464823 22008 rpcz_store.cc:276] 0709 13:08:14.747288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.054289 (+307001us) service_pool.cc:224] Handling call
0709 13:08:15.080614 (+ 26325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.082697 (+  2083us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:08:15.697559 (+614862us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.008637 (+311078us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:17.291941 (+1283304us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:08:17.299576 (+  7635us) log.cc:844] Serialized 26 byte log entry
0709 13:08:17.300196 (+   620us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:17.300244 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.300341 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.446392 (+146051us) raft_consensus.cc:1685] finished
0709 13:08:17.463996 (+ 17604us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.464191 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29026,"prepare.queue_time_us":553734,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":103436,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":33471,"spinlock_wait_cycles":7681280,"thread_start_us":850303,"threads_started":2,"wal-append.queue_time_us":855670}
I20260709 13:08:17.462831 29048 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:17.487100 29196 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:08:17.487262 28707 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.488639 21972 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.487813 28707 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.462129 28981 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.428606 29040 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.489758 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:17.516867 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.518626 21852 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:17.500459 29205 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:17.519503 29205 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:17.519945 29205 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:17.519886 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.516569 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3222) took 2956 ms. Trace:
I20260709 13:08:17.521274 29063 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:17.521575 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:17.491067 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 1.82253e+06us
I20260709 13:08:17.463635 29023 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:17.508028 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:17.488950 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 1.93092e+06us
I20260709 13:08:17.527894 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:17.489871 29041 raft_consensus.cc:2804] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:17.494484 29149 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:17.490026 28829 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:08:17.494683 28807 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:17.494599 28708 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.510394 28707 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.512173 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2688) took 1903 ms. Trace:
I20260709 13:08:17.545228 21931 rpcz_store.cc:276] 0709 13:08:15.608926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.697875 (+ 88949us) service_pool.cc:224] Handling call
0709 13:08:15.698311 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.492578 (+1794267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:17.492616 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:17.492668 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:17.492710 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.511914 (+ 19204us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54002e7cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:17.511983 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.512114 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1430854,"raft.queue_time_us":1431281,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"spinlock_wait_cycles":834760960}
I20260709 13:08:17.491516 29091 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:17.490089 28989 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.471294 29020 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:08:17.520854 21859 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:17.522969 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:17.489892 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 1.87653e+06us
W20260709 13:08:17.489733 29198 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:08:17.522332 21851 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.494400 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3283) took 2603 ms. Trace:
I20260709 13:08:17.494494 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.525821 21858 raft_consensus.cc:2468] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
W20260709 13:08:17.560767 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 66350.7us
I20260709 13:08:17.525599 29158 consensus_queue.cc:237] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.489288 28918 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:17.535210 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45521.6us
I20260709 13:08:17.576325 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:17.536293 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:17.521306 21932 rpcz_store.cc:276] 0709 13:08:14.560448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.561353 (+   905us) service_pool.cc:224] Handling call
0709 13:08:14.561670 (+   317us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:15.103705 (+542035us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:15.221840 (+118135us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.425276 (+203436us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:17.495313 (+2070037us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b2c0000e5c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:08:17.496057 (+   744us) log.cc:844] Serialized 52 byte log entry
0709 13:08:17.496235 (+   178us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:17.496281 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.496349 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.500971 (+  4622us) raft_consensus.cc:1685] finished
0709 13:08:17.516361 (+ 15390us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.516512 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":541547,"prepare.queue_time_us":413060,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":567904,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":11550,"spinlock_wait_cycles":27862400,"thread_start_us":298050,"threads_started":2,"wal-append.queue_time_us":1017791}
I20260709 13:08:17.579046 28807 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.542459 28869 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:17.548575 29041 raft_consensus.cc:697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:17.582100 29041 consensus_queue.cc:237] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.582734 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.583629 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:17.584163 29039 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:17.585810 21931 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:17.586957 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:17.583559 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:17.587350 28896 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:17.587781 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.589138 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:17.589367 29059 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.589680 21930 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:17.590735 21858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 1104) took 2733 ms (client timeout 1597 ms). Trace:
W20260709 13:08:17.591167 21858 rpcz_store.cc:269] 0709 13:08:14.856949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.860901 (+  3952us) service_pool.cc:224] Handling call
0709 13:08:17.590543 (+2729642us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b540010b654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:17.590661 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6109215616}
I20260709 13:08:17.591609 29095 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:17.544173 22007 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
W20260709 13:08:17.595906 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59680.5us
I20260709 13:08:17.596804 28001 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.597241 28001 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.546613 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:17.604794 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.522600 29043 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.561611 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:17.474395 21806 placement_policy_util.cc:248] tablet 4458afcc40954b359ea857cddbfdd681: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:08:17.494428 29085 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.566085 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:17.560462 22001 rpcz_store.cc:276] 0709 13:08:14.891233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.739275 (+848042us) service_pool.cc:224] Handling call
0709 13:08:15.739634 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.134834 (+1395200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.494175 (+359341us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54002710d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:17.494339 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3206973824}
I20260709 13:08:17.567830 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.568763 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.593825 21929 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:17.579339 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.607879 21932 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:17.607939 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.608633 21929 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:17.609483 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.610023 21929 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:17.510434 29052 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.605785 21860 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:17.602890 22008 raft_consensus.cc:2376] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22. Re-sending same reply.
I20260709 13:08:17.538151 29149 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.622835 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:17.558475 29193 log.cc:927] Time spent T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:08:17.620225 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 58708.5us
I20260709 13:08:17.620391 21933 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:17.626796 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:17.629164 29043 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.630024 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.630900 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.631111 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.631888 29038 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:17.571732 28886 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:17.632269 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:17.829872 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.632752 29149 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:17.623827 21932 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:17.637722 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.633419 28987 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:08:17.782892 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 216846us
I20260709 13:08:17.832454 22593 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:17.832855 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:17.833068 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:17.544863 29130 consensus_queue.cc:1059] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.090s
I20260709 13:08:17.785642 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.548765 28708 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:17.786742 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 151795us
W20260709 13:08:17.787177 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 155445us
I20260709 13:08:17.786788 22002 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:17.789247 22592 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:17.559553 28923 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.123s
I20260709 13:08:17.535327 29023 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:17.632078 29201 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:08:17.562166 28652 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.597963 28707 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:17.636694 21927 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:17.846500 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:17.846644 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:17.846504 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56313.3us
I20260709 13:08:17.846200 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:17.613726 29147 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:17.885769 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.887326 22002 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:17.628487 29059 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:17.638160 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.856101 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.828785 29054 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:17.857362 21851 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:17.852941 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.851287 21930 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:17.872875 21928 raft_consensus.cc:2376] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18. Re-sending same reply.
I20260709 13:08:17.874019 22592 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:08:17.873852 21824 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.541607 29024 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:08:17.888430 22588 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:17.845918 28874 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.890071 21931 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:17.900439 22591 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:17.846168 29097 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:17.897850 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.898423 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:17.899176 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.899655 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.909871 21928 raft_consensus.cc:2410] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:08:17.908978 21974 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.910311 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:17.601047 29077 consensus_queue.cc:237] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.647419 29085 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.903537 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.798975 29091 consensus_queue.cc:1059] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:08:17.871538 29095 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:17.903537 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.926378 21930 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:17.908017 21931 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:17.927711 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.897917 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:17.871039 28707 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.928665 28707 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:17.909608 22001 raft_consensus.cc:2410] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:08:17.913153 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.935297 21928 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:17.913033 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:17.915524 22008 raft_consensus.cc:2410] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
W20260709 13:08:17.917382 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 84415.6us
I20260709 13:08:17.918744 21935 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
W20260709 13:08:17.919895 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67028.8us
I20260709 13:08:17.863955 28900 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:08:17.918532 21824 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:17.938670 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.943158 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:17.942764 21860 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:17.909298 29020 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:17.885227 28652 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.948935 29020 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.949420 29020 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.949882 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.950533 21935 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:17.951367 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.952095 21935 raft_consensus.cc:2410] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:08:17.945456 21974 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:17.953400 29006 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:17.797669 29096 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:08:17.954864 29096 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:17.955185 29096 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:17.955976 29096 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:17.945780 29202 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.466s	user 0.001s	sys 0.000s
I20260709 13:08:17.909687 28874 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:17.958113 29160 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.904s	user 0.007s	sys 0.004s
I20260709 13:08:17.958648 29160 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:17.959149 28874 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.926158 29095 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:17.959954 29096 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:17.960462 29095 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:17.952836 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.965206 29040 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:17.965771 21935 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:17.967418 28708 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:17.967235 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:17.968366 29211 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:17.969310 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.969805 22003 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:17.970314 21935 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:17.971230 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.969424 21856 raft_consensus.cc:1275] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:17.971784 22001 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:17.946310 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:17.972424 21935 raft_consensus.cc:2410] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:08:17.972834 21928 raft_consensus.cc:2393] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 16: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:17.982699 28001 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:17.938548 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:18.002470 29020 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.008456 29020 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:17.916321 28886 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:17.923276 29038 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:08:18.011761 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 66387.4us
I20260709 13:08:17.924434 29063 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:08:18.023412 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 104959us
W20260709 13:08:18.017814 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46891.4us
I20260709 13:08:18.019222 21857 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:18.019349 28001 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:17.960856 29160 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.035583 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.042547 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:18.043285 22589 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:18.043416 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:17.958181 29023 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.044170 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:18.044778 22586 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:08:18.045202 22585 raft_consensus.cc:2368] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
W20260709 13:08:18.049734 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 111249us
I20260709 13:08:18.050796 21898 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:18.021122 29038 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.060248 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.061559 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.062109 21935 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:18.062865 28652 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.044545 29019 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:08:18.059087 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.038255 28754 consensus_queue.cc:1059] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:08:18.064956 28981 consensus_meta.cc:287] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.004s
I20260709 13:08:18.068739 28981 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.071659 28981 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.068141 21928 raft_consensus.cc:2376] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16. Re-sending same reply.
I20260709 13:08:18.072633 22000 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:18.073058 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.074263 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.075356 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.075920 21928 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:18.068189 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1123) took 3091 ms. Trace:
I20260709 13:08:18.077517 21856 rpcz_store.cc:276] 0709 13:08:14.977211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.132644 (+155433us) service_pool.cc:224] Handling call
0709 13:08:17.592118 (+2459474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.971844 (+379726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.067505 (+ 95661us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540010b654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:18.067977 (+   472us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b540010b650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:18.068107 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6522195968}
W20260709 13:08:18.078348 29209 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:08:18.065943 29095 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.081234 29095 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.076742 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.082285 21928 raft_consensus.cc:2368] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24.
I20260709 13:08:18.083158 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.083305 29149 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.084416 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.085204 21928 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:18.060272 28886 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.086090 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.017477 29006 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:18.050626 29217 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:18.059957 28708 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:17.914503 29097 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.034766 29040 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:18.011224 29074 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:08:18.089174 29218 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.023512 29063 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.111779 29063 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:18.103665 29040 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:18.038749 29152 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:18.113974 29040 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.114938 29152 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.115483 29152 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:18.117913 29176 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.118654 29176 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:18.120729 29118 consensus_meta.cc:287] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.939s	user 0.010s	sys 0.001s
I20260709 13:08:18.125159 22590 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:18.105700 29006 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.122949 29217 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:18.137668 29195 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.740s	user 0.005s	sys 0.000s
I20260709 13:08:18.137925 29217 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.060474 29038 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:18.124086 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.089735 21928 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:18.062891 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:18.122011 28984 consensus_meta.cc:287] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.844s	user 0.000s	sys 0.017s
I20260709 13:08:18.130149 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:18.152444 21974 leader_election.cc:400] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:08:18.152865 21974 leader_election.cc:403] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:08:18.058792 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:18.135565 29054 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:18.140698 29217 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.151109 21935 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:18.130743 22593 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
W20260709 13:08:18.052742 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 199231us
I20260709 13:08:18.164714 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.165483 21929 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:18.166560 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.170406 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.171118 21928 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:18.172066 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.172592 21928 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:18.172878 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:18.174791 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:18.176630 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 117928us
I20260709 13:08:18.177469 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:18.182847 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.184058 21898 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:18.131924 28708 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:18.187202 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.187528 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:18.191428 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.191502 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:18.157956 29054 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:18.203102 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.204787 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.209910 21926 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:18.170286 29160 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:18.170153 29217 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:18.189651 29095 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:18.188809 28754 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:18.205971 29024 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:18.141593 29118 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.213630 29019 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:18.226580 22009 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:18.213338 29013 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:18.106997 29218 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.164150 28984 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:18.240908 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.240440 21929 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:18.182236 29148 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:08:18.185252 29014 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:08:18.251400 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.204943 28708 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.218945 29160 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:18.263159 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 234984us
I20260709 13:08:18.218874 28807 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:18.224218 29149 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.238459 29019 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:18.224224 29096 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:08:18.237720 29013 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:18.270532 22006 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:18.266144 29175 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 1.124s	user 0.010s	sys 0.003s
I20260709 13:08:18.224272 29091 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:08:18.267066 28919 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.290445 28919 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.292646 28919 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.312085 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.305410 29085 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.293344 28900 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:18.289938 29214 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:08:18.310362 28708 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.262568 29176 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:18.302497 29218 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.272552 29217 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.299690 29118 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.303108 29175 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.334960 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:18.335788 22590 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:18.336331 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.306689 29050 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:18.267915 29160 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:18.308195 29059 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:18.308449 28984 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.272989 29019 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:18.275192 29013 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:18.311175 29063 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.298770 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.354369 21931 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
W20260709 13:08:18.277693 29164 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 1.002s	user 0.017s	sys 0.001s
I20260709 13:08:18.334232 29038 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.369611 29164 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.305058 28707 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.372061 22584 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:18.373852 28707 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.346683 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.333941 28900 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.375145 21934 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:18.377705 28900 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.351841 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.379786 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.381042 21933 raft_consensus.cc:2393] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:18.381964 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.382651 21933 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 20: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:18.383358 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.384351 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.384636 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.352929 22004 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 24: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:18.340551 29217 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.385572 29152 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.386865 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32904 (request call id 1433) took 1106 ms. Trace:
I20260709 13:08:18.387463 29217 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.387257 22004 rpcz_store.cc:276] 0709 13:08:17.294400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.349709 (+ 55309us) service_pool.cc:224] Handling call
0709 13:08:18.386692 (+1036983us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b540033ead4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:18.386809 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235124096}
I20260709 13:08:18.388298 29152 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.389376 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.390201 21933 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:18.391321 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.392545 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.304721 29054 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.393774 29054 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:18.393903 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.395409 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:18.400661 29205 consensus_meta.cc:287] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.880s	user 0.012s	sys 0.000s
I20260709 13:08:18.401343 29205 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.402971 21934 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:18.366503 29097 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:18.368053 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.368979 29175 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.374287 29038 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.385234 21932 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:18.403832 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:18.432394 22591 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:18.406090 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.435240 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.409901 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:18.437309 22592 raft_consensus.cc:2393] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:18.411162 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:18.439226 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:18.442525 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:18.403844 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:18.443642 21859 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:08:18.420238 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:18.443368 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:18.445591 22590 raft_consensus.cc:2468] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:18.445686 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.426128 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:18.446903 21932 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:18.447834 22000 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:18.421919 21933 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:18.448184 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.422659 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:18.448948 21932 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:18.452292 21974 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:18.455307 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:18.419695 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:18.455972 22592 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:18.456590 22586 raft_consensus.cc:1240] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:08:18.455595 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:18.457679 22591 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:18.458070 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.405061 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:18.459625 22593 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:18.466985 22592 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:18.456800 22556 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:18.449131 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:18.474507 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:18.484056 22593 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:18.485214 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.486232 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.487375 21857 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 21. Current term is 22. Ops: [16.13-21.15]
I20260709 13:08:18.487609 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.406313 29063 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:18.489950 21932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:18.490803 21932 raft_consensus.cc:606] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:18.491199 21932 raft_consensus.cc:614] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:18.412969 29097 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 13:08:18.492144 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48829.7us
I20260709 13:08:18.493551 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:18.403455 29205 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.404765 29164 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:18.503479 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.506263 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:18.524132 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:18.415045 29060 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.415181 29173 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:18.529309 29173 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.529858 29063 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:18.530408 29063 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:18.531526 29173 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.531180 29063 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.528442 29060 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.513924 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:18.425815 29175 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:18.473572 28707 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:18.455703 28807 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:18.526813 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.446429 29204 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.434011 29074 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.527608 21860 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:18.423799 29228 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:18.512315 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.544930 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3303) took 1091 ms. Trace:
I20260709 13:08:18.501821 29059 raft_consensus.cc:2804] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:18.545521 22008 rpcz_store.cc:276] 0709 13:08:17.453625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.937492 (+483867us) service_pool.cc:224] Handling call
0709 13:08:18.012651 (+ 75159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.048388 (+ 35737us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:18.275571 (+227183us) log.cc:844] Serialized 14 byte log entry
0709 13:08:18.544444 (+268873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:18.544508 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:18.544564 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.544656 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.544861 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10196,"prepare.queue_time_us":163272,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":21683,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":4041,"thread_start_us":263447,"threads_started":2}
I20260709 13:08:18.527642 22591 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:18.534142 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:18.534171 21859 raft_consensus.cc:1240] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:18.487051 29130 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:18.485567 29152 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:18.539681 22585 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:08:18.540499 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 187814us
W20260709 13:08:18.700033 21823 leader_election.cc:336] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.734s (SENT)
I20260709 13:08:18.700417 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:18.489770 29176 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:08:18.596375 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 45835.3us
I20260709 13:08:18.702028 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:18.524010 28850 consensus_queue.cc:1048] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:08:18.621611 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1439) took 1246 ms. Trace:
I20260709 13:08:18.703670 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3304) took 1247 ms. Trace:
I20260709 13:08:18.702864 22002 rpcz_store.cc:276] 0709 13:08:17.375194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.929025 (+553831us) service_pool.cc:224] Handling call
0709 13:08:17.929321 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.929524 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:08:18.011512 (+ 81988us) log.cc:844] Serialized 14 byte log entry
0709 13:08:18.621264 (+609752us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:18.621314 (+    50us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:18.621363 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.621432 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.621556 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":281748,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":95244,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":660455,"threads_started":3,"wal-append.queue_time_us":338340}
I20260709 13:08:18.594838 21934 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:18.704075 22005 rpcz_store.cc:276] 0709 13:08:17.456037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.965413 (+509376us) service_pool.cc:224] Handling call
0709 13:08:17.965730 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.966004 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:18.160294 (+194290us) log.cc:844] Serialized 14 byte log entry
0709 13:08:18.702975 (+542681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:18.703038 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:18.703122 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.703484 (+   362us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.703601 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10549,"prepare.queue_time_us":245459,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":60922,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":14709,"thread_start_us":714405,"threads_started":3,"wal-append.queue_time_us":376810}
I20260709 13:08:18.595881 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3239) took 1026 ms. Trace:
I20260709 13:08:18.539624 28807 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.617633 22007 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:18.566365 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:18.511564 28708 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:18.660216 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.577s (SENT) [suppressed 56 similar messages]
I20260709 13:08:18.662787 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 16 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:18.485332 29217 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:08:18.420789 29019 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.705722 21935 rpcz_store.cc:276] 0709 13:08:17.569264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.269200 (+699936us) service_pool.cc:224] Handling call
0709 13:08:18.280115 (+ 10915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.595446 (+315331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:18.595477 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:18.595523 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:18.595577 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.595657 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.595819 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2426,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":311791,"threads_started":1}
I20260709 13:08:18.711257 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:18.541142 29175 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.712540 22002 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:18.712617 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:18.623358 29059 raft_consensus.cc:697] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:18.713877 22008 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:18.720636 21857 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:18.721472 21859 raft_consensus.cc:1240] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:18.725188 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 159121us
I20260709 13:08:18.672803 28707 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:18.731094 21935 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:18.707839 28900 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.697788 29135 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:18.703922 29130 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:18.732844 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.735759 21860 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:18.736671 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:18.580148 29074 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.708400 28807 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:18.710659 28984 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:18.708434 28708 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:18.736944 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:18.744380 21857 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:18.747287 21856 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:18.580410 29228 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.566682 29204 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.753170 21935 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:08:18.754187 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3243) took 1174 ms. Trace:
I20260709 13:08:18.735313 29096 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:18.760867 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:18.761015 21929 rpcz_store.cc:276] 0709 13:08:17.579746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.191024 (+611278us) service_pool.cc:224] Handling call
0709 13:08:18.754014 (+562990us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b54003075d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:18.754130 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110574336}
I20260709 13:08:18.785720 22005 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:18.786482 22557 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:18.734642 29038 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:18.738895 29205 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:18.741523 29130 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:18.793637 29059 consensus_queue.cc:237] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:18.794504 22618 heartbeater.cc:646] Failed to heartbeat to 127.20.66.190:46167 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.20.66.190:46167 timed out after 15.000s (SENT)
I20260709 13:08:18.748549 28708 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:18.806728 21935 raft_consensus.cc:1240] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:18.812731 22008 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:18.816720 21851 raft_consensus.cc:2393] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 23: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:18.763195 29096 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:18.814143 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.743809 29135 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.824869 22008 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 15.
I20260709 13:08:18.785890 29148 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:18.744827 28900 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.834661 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:18.835758 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:18.837694 29043 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:18.830483 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 170298us
I20260709 13:08:18.842200 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3132) took 1269 ms. Trace:
I20260709 13:08:18.842689 22007 rpcz_store.cc:276] 0709 13:08:17.572904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.047864 (+474960us) service_pool.cc:224] Handling call
0709 13:08:18.127863 (+ 79999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.707065 (+579202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.841295 (+134230us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b540040c254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:18.841992 (+   697us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b540040c250. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:18.842142 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308714624}
I20260709 13:08:18.843326 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.843927 22008 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:18.843999 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.844853 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.849534 29096 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.850073 29175 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:18.850834 29096 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:18.851529 29060 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:18.851729 28900 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:18.852895 29246 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:08:18.853891 21858 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:08:18.850025 29236 log.cc:927] Time spent T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:08:18.827020 28707 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:18.858583 29239 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:08:18.837608 29228 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:18.826258 29248 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:18.822227 29205 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:18.867178 29248 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.832181 29013 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.884084 22005 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:18.834299 29148 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:18.885933 22007 raft_consensus.cc:2410] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 13:08:18.893724 21823 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.943s (SENT)
W20260709 13:08:18.900193 21823 leader_election.cc:336] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.004s (SENT)
I20260709 13:08:18.900776 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:18.902417 28886 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:18.906965 29074 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:18.908597 29054 consensus_meta.cc:287] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.002s
I20260709 13:08:18.933956 21857 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:18.917326 29238 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:08:18.926999 28754 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.000s
I20260709 13:08:18.950064 29019 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:18.948776 28754 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:08:18.959138 29204 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:18.977346 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.949703 29238 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:18.940562 29205 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:18.984068 29244 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:08:18.979780 29135 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:18.988713 29245 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:08:18.989588 29148 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.993974 29013 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:18.994369 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.995640 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:18.996235 22008 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:18.996434 29248 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.001278 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2695) took 1154 ms. Trace:
I20260709 13:08:19.002595 21932 rpcz_store.cc:276] 0709 13:08:17.846428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.513515 (+667087us) service_pool.cc:224] Handling call
0709 13:08:18.537985 (+ 24470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:18.569760 (+ 31775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:18.569821 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:18.591115 (+ 21294us) log.cc:844] Serialized 52 byte log entry
0709 13:08:18.889299 (+298184us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:18.958385 (+ 69086us) log.cc:844] Serialized 14 byte log entry
0709 13:08:18.961423 (+  3038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.961549 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:19.000988 (+ 39439us) raft_consensus.cc:1685] finished
0709 13:08:19.001036 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.001202 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86789,"prepare.queue_time_us":59059,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":4790,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":28115,"spinlock_wait_cycles":1242496,"thread_start_us":208409,"threads_started":2,"wal-append.queue_time_us":254170}
W20260709 13:08:19.016299 29160 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.664s	user 0.006s	sys 0.002s
I20260709 13:08:19.018909 21928 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:19.024184 22005 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:08:19.033231 29238 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.025851 28807 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.035816 28807 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.041815 28708 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.016825 29160 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.051978 21934 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:19.052937 28919 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:19.053558 28919 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.054989 28919 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.056344 28919 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:19.059657 21859 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 23. Current term is 24. Ops: [19.10-23.12]
I20260709 13:08:19.059960 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.060663 21853 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:08:19.059660 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.061784 21857 raft_consensus.cc:2368] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23.
I20260709 13:08:19.061985 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.070389 29048 raft_consensus.cc:3055] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:19.070875 29048 raft_consensus.cc:740] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:19.071753 29048 consensus_queue.cc:260] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.074353 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3128) took 1533 ms. Trace:
I20260709 13:08:19.074350 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.074908 22009 rpcz_store.cc:276] 0709 13:08:17.541234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.966751 (+425517us) service_pool.cc:224] Handling call
0709 13:08:17.967037 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.237904 (+270867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.074133 (+836229us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400169254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.074298 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591320832}
I20260709 13:08:19.075511 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.077877 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.079231 22008 raft_consensus.cc:1240] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:19.081935 22591 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:08:19.083046 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:19.083673 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.085283 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.083940 22587 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:19.087719 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.072755 29048 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:19.089879 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.091079 21857 raft_consensus.cc:2410] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:19.092612 29160 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.094493 21898 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:19.095992 29255 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.120s	user 0.002s	sys 0.000s
I20260709 13:08:19.096597 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.100032 29024 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.100263 21855 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:19.104233 22587 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:19.105922 21900 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:19.108239 29024 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:19.045656 29130 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.108915 29024 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.123476 29024 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.110540 28918 consensus_queue.cc:1059] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:08:19.126861 28918 raft_consensus.cc:3055] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:19.127799 28918 raft_consensus.cc:740] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:19.130005 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.111502 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:19.121413 21823 leader_election.cc:336] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.943s (SENT)
I20260709 13:08:19.132054 21898 leader_election.cc:400] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23.
I20260709 13:08:19.132002 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:19.132611 21898 leader_election.cc:403] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:19.121654 22589 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:19.136262 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:19.137112 22587 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.136181 29096 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.137812 29147 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:19.144742 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.147195 22587 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:19.147351 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.148370 21900 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:19.119038 29217 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:19.134093 28807 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:19.151013 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.110903 29054 consensus_queue.cc:1059] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:08:19.151209 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.113834 29130 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:19.152132 21930 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:19.111822 28708 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.148610 28919 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:08:19.155215 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.155967 21935 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:08:19.148787 28869 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:08:19.158540 21935 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:19.151795 29240 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:19.167018 29054 raft_consensus.cc:3055] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:19.167520 29054 raft_consensus.cc:740] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.169289 29240 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.172075 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.173579 21929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:19.174404 21929 raft_consensus.cc:606] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.174721 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:19.153267 28807 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.177320 21824 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:19.179869 29205 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:19.180954 28869 consensus_queue.cc:1243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:19.181519 28708 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:19.180459 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:19.183221 28820 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:19.177093 29032 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:19.149816 28918 consensus_queue.cc:260] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.182013 28708 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.185882 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2689) took 3574 ms. Trace:
I20260709 13:08:19.183272 22001 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:19.186101 28708 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.186282 21926 rpcz_store.cc:276] 0709 13:08:15.611414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.709959 (+ 98545us) service_pool.cc:224] Handling call
0709 13:08:15.710259 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.216280 (+2506021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.185663 (+969383us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b540017f554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.185827 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5747883264}
I20260709 13:08:19.186964 29096 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23.
I20260709 13:08:19.187314 28708 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.187822 22001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3129) took 1640 ms. Trace:
I20260709 13:08:19.188179 28918 raft_consensus.cc:3060] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.188206 22001 rpcz_store.cc:276] 0709 13:08:17.547935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.013560 (+465625us) service_pool.cc:224] Handling call
0709 13:08:18.013909 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.186959 (+1173050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.187607 (+   648us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002152d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.187765 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2688805760}
I20260709 13:08:19.189635 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1556) took 1564 ms. Trace:
I20260709 13:08:19.190045 22584 rpcz_store.cc:276] 0709 13:08:17.625653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.625834 (+   181us) service_pool.cc:224] Handling call
0709 13:08:17.626270 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.372603 (+746333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.189358 (+816755us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540076fcd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.189565 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714442496}
I20260709 13:08:19.191711 29205 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:19.191993 28820 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:19.190739 29240 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.193162 28820 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.179068 28807 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:19.199811 29242 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
I20260709 13:08:19.168577 29054 consensus_queue.cc:260] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.206728 22587 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:08:19.210367 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:19.213531 28829 consensus_queue.cc:1059] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:19.205232 29152 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:19.214587 28829 raft_consensus.cc:3055] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:19.214977 29152 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.215587 29152 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.215061 28829 raft_consensus.cc:740] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.219218 21852 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:08:19.221416 21899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:19.231504 21823 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 2.241s (SENT)
I20260709 13:08:19.192683 29205 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.232349 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:19.251195 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.252040 22002 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:19.256167 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:19.257638 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47357.5us
I20260709 13:08:19.261430 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.262506 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.264093 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.266103 22002 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:08:19.267760 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.269484 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.270598 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.272501 22005 raft_consensus.cc:1240] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:19.270179 22000 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:19.272703 22004 raft_consensus.cc:1240] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:08:19.214990 29054 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.288554 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.247031 29135 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:19.263103 29096 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:19.263868 28984 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:19.349225 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.271199 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.273813 29148 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:19.379815 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2722) took 1414 ms. Trace:
I20260709 13:08:19.389819 21934 rpcz_store.cc:276] 0709 13:08:17.965366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.737105 (+771739us) service_pool.cc:224] Handling call
0709 13:08:18.751612 (+ 14507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.052423 (+300811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.379541 (+327118us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b5400206f54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.379751 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690356992}
I20260709 13:08:19.380534 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3290) took 2181 ms. Trace:
I20260709 13:08:19.382417 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1276) took 1053 ms. Trace:
I20260709 13:08:19.273600 28923 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.392144 21928 rpcz_store.cc:276] 0709 13:08:18.329547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.336091 (+  6544us) service_pool.cc:224] Handling call
0709 13:08:19.382144 (+1046053us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b540017e654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:19.382335 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346809728}
W20260709 13:08:19.387055 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:19.387737 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 116597us
I20260709 13:08:19.390893 21932 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:19.392829 22006 rpcz_store.cc:276] 0709 13:08:17.199357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.199610 (+   253us) service_pool.cc:224] Handling call
0709 13:08:17.199987 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.272529 (+1072542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.351432 (+1078903us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001982d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.380477 (+ 29045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2449535360}
I20260709 13:08:19.396397 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:19.398720 21972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.126s	user 0.037s	sys 0.019s
W20260709 13:08:19.399164 21972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.037s	sys 0.019s
W20260709 13:08:19.399493 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 1.12658e+06us
I20260709 13:08:19.400338 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.295888 29153 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:08:19.302982 29135 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.408116 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.408154 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.262172 28829 consensus_queue.cc:260] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.411877 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:19.414155 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.411571 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.414740 22006 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.383837 29240 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:19.389503 29096 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:19.430883 28989 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:19.420461 29158 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:19.431591 28989 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.432250 29158 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:19.432837 29158 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.434206 28989 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.421833 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:19.409997 29006 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:19.426702 29160 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:19.420861 29077 raft_consensus.cc:3055] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:19.410485 28919 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.429450 28829 raft_consensus.cc:3060] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.445966 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.447934 22006 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:19.437920 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.442836 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.442925 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.443667 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.453373 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.453953 21931 raft_consensus.cc:2376] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20. Re-sending same reply.
I20260709 13:08:19.445262 28001 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:19.455013 28001 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:19.413015 29135 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.455468 28001 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.446571 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.458575 22009 raft_consensus.cc:2368] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23.
I20260709 13:08:19.385972 28984 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.460223 22006 pending_rounds.cc:85] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 12
I20260709 13:08:19.461710 22006 pending_rounds.cc:107] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:08:19.462981 22006 raft_consensus.cc:2889] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:19.439365 29217 raft_consensus.cc:3055] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:08:19.475929 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 178208us
I20260709 13:08:19.484508 28001 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.471413 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.443909 29077 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:19.468111 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.450912 29039 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.493634 29039 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.444777 29097 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:19.496681 29039 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:19.485399 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.498610 29006 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:19.499079 29006 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:19.499470 29006 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:19.501392 29218 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:19.473822 29217 raft_consensus.cc:740] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.454952 29238 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.450584 28820 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:19.477145 28918 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:19.440886 29240 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:19.515883 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47843.4us
I20260709 13:08:19.525579 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.527837 29238 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:19.528437 29238 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:19.529727 29238 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.448844 29054 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:08:19.531934 29054 consensus_queue.cc:1243] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 24.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:19.534096 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:19.535526 29160 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:19.513985 28844 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:19.535982 29160 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:19.466365 29096 consensus_queue.cc:1059] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:08:19.536422 29160 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.537091 29096 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:19.537495 29096 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:19.538375 29096 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.539199 21929 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:19.540249 29096 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:19.542297 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.544100 28984 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:19.525794 28820 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.546224 21972 leader_election.cc:400] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24.
I20260709 13:08:19.562412 21972 leader_election.cc:403] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:08:19.564141 28820 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:19.489238 29148 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:19.547770 29160 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.556484 29147 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:19.564966 29039 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24.
I20260709 13:08:19.540814 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.579101 21929 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.567433 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.555676 29158 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:19.584168 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 141060us
W20260709 13:08:19.586158 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.736s (SENT)
I20260709 13:08:19.565536 29148 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.576349 28984 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:19.590624 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.600072 21855 raft_consensus.cc:1240] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:19.602981 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.606812 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.591562 28820 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.592715 29039 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:19.610987 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.613891 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.619012 21927 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:19.618194 21926 raft_consensus.cc:2376] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22. Re-sending same reply.
I20260709 13:08:19.620888 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.621066 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.621595 21926 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:19.621939 21927 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:19.622637 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.623566 21926 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:19.623980 29038 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.624516 29038 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.625201 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.635367 21929 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:19.661525 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.601754 28984 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.625401 29039 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:19.587749 29077 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.608129 29152 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:19.667662 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 100381us
I20260709 13:08:19.669858 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:19.669024 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 882562us
I20260709 13:08:19.659624 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:19.673951 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 87807.2us
I20260709 13:08:19.660940 29176 raft_consensus.cc:697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:19.678067 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.643605 28807 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.668143 29039 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:19.625595 28820 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.677263 28917 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:19.687141 28917 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:19.671423 29164 raft_consensus.cc:2764] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 13:08:19.677778 29077 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:19.689601 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:19.700178 21851 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:19.699965 28917 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:08:19.703120 29272 log.cc:927] Time spent T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:08:19.705528 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:19.708556 28896 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:19.710110 28896 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:19.710629 28896 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:19.711571 28896 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.691896 28850 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:08:19.715140 28850 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:19.690028 29048 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:08:19.690254 29173 consensus_queue.cc:1059] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 15.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.034s
W20260709 13:08:19.725086 29273 log.cc:927] Time spent T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:08:19.725013 29173 raft_consensus.cc:3055] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:19.725703 29173 raft_consensus.cc:740] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:19.725780 29038 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.726604 29173 consensus_queue.cc:260] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 15.16, Last appended by leader: 16, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.727625 28917 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:19.728024 29050 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:19.728158 28917 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:19.727509 29135 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.729282 29130 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:19.729163 28917 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.730193 29158 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:19.730386 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.730544 28917 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:19.729789 29130 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:19.732491 28984 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.732628 29130 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:19.735266 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.735476 21929 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:19.740087 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.728475 29173 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:19.694167 29160 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.734382 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:19.762931 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.750527 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.750944 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1561) took 1767 ms. Trace:
I20260709 13:08:19.735298 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:19.696142 28708 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.008s
W20260709 13:08:19.768981 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 60847.2us
I20260709 13:08:19.770632 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:19.779585 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 44392.7us
I20260709 13:08:19.776180 21899 leader_election.cc:400] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521) with higher term. Message: Invalid argument: T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for earlier term 22. Current term is 23.
W20260709 13:08:19.729707 29060 consensus_queue.cc:1175] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" responder_term: 15 status { last_received { term: 15 index: 16 } last_committed_idx: 15 last_received_current_leader { term: 15 index: 16 } }
I20260709 13:08:19.779671 21934 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:19.781944 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.782724 21934 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
W20260709 13:08:19.775538 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 105763us
I20260709 13:08:19.783493 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.783954 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.787087 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.787680 21934 raft_consensus.cc:2376] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23. Re-sending same reply.
I20260709 13:08:19.788431 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.728513 29050 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:19.703670 29148 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:19.786345 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.749022 29164 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.748617 28918 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:19.804157 21927 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:19.790875 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.805272 21935 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:19.789325 29050 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:19.748325 28829 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:19.794210 29286 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:08:19.794343 29095 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.778826 29182 raft_consensus.cc:2764] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:08:19.798188 29135 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.760525 29240 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:19.801695 29284 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:19.805703 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.811190 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:19.815945 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.863665 21927 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:19.864032 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.867022 22001 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:08:19.817427 21934 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:19.751055 29276 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:08:19.833039 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.869019 21859 raft_consensus.cc:2368] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:08:19.769658 29006 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:19.841524 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.842376 21929 raft_consensus.cc:2393] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:19.776573 29038 consensus_queue.cc:1059] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:08:19.817806 29148 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:19.816941 29118 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:19.803967 29147 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:19.790030 29176 consensus_queue.cc:237] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.852154 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.890020 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:08:19.856564 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.801833 29152 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:19.862149 22008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3314) took 1434 ms. Trace:
I20260709 13:08:19.870307 21859 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:19.786222 29024 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.893775 21927 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:19.877866 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.878340 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.902380 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.835368 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
W20260709 13:08:19.878643 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 67580.8us
I20260709 13:08:19.834957 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.903517 21823 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.881853 22002 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:19.887439 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.905256 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.875135 22005 raft_consensus.cc:1217] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Deduplicated request from leader. Original: 16.9->[19.10-23.12]   Dedup: 21.11->[23.12-23.12]
I20260709 13:08:19.906390 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.891021 22008 rpcz_store.cc:276] 0709 13:08:18.427683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.996892 (+569209us) service_pool.cc:224] Handling call
0709 13:08:18.999038 (+  2146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.079710 (+ 80672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.861889 (+782179us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b54001e1ed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.862083 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179822464}
I20260709 13:08:19.907176 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.891283 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:19.891300 22556 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Timed out: RequestConsensusVote RPC to 127.20.66.130:45191 timed out after 1.779s (SENT)
I20260709 13:08:19.889297 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.907864 22001 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.897766 21854 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:19.909193 22005 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 23. Current term is 24. Ops: [19.10-23.12]
I20260709 13:08:19.857832 29160 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:08:19.910501 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.910980 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.911394 21853 raft_consensus.cc:2393] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:19.912077 21859 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:19.912948 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.908097 21935 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:19.916615 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.918133 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:19.918744 22002 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:19.883581 29135 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:19.917768 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.920960 22008 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.916816 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:19.921568 21854 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:08:19.919605 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.920452 22006 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:19.921365 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.881713 29148 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:19.925817 21853 raft_consensus.cc:3055] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:19.927024 21853 raft_consensus.cc:740] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:19.929391 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.916692 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.930047 21854 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:19.918442 29095 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:08:19.923480 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.923148 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:19.919605 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.933562 22005 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:19.934754 21927 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:19.935001 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.933881 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.935616 21935 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:19.937145 22006 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:19.938773 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.939733 21857 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
W20260709 13:08:19.935799 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 201491us
I20260709 13:08:19.940104 21858 raft_consensus.cc:2393] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:19.923990 21859 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 23. Current term is 24. Ops: [19.10-23.12]
I20260709 13:08:19.943948 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:19.929256 21853 consensus_queue.cc:260] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.945675 21853 raft_consensus.cc:3060] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:19.938068 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:19.943236 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:19.950344 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:19.951311 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:19.925206 29281 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:19.962496 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:19.960824 21935 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.895742 29118 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:19.928208 29148 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.949565 28850 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:19.944494 29095 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:19.949357 28900 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:19.928010 29017 consensus_queue.cc:1059] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:08:19.928267 29096 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:08:19.923631 29182 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:19.966374 29135 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:20.032078 29135 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:20.032565 29135 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.953177 29240 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:08:20.033190 29130 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.011031 29006 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:20.013228 28807 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:19.983606 28820 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.052469 29130 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.053943 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.055083 21930 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:08:20.056028 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:20.056725 21930 raft_consensus.cc:2410] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 13:08:20.013743 29277 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.437s	user 0.003s	sys 0.000s
I20260709 13:08:19.992013 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3306) took 1567 ms. Trace:
I20260709 13:08:20.033610 29240 consensus_queue.cc:1243] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 24.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:20.007505 29205 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:20.034636 29152 raft_consensus.cc:3055] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:20.035697 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:19.991524 29048 raft_consensus.cc:2764] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:08:20.008093 28900 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:20.030972 29281 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:19.977865 21927 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:20.060003 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.066524 22001 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:20.068425 21935 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:20.103334 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.106551 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:20.069702 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.071102 21853 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:08:19.976449 22008 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:20.074973 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:20.117524 21930 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.079026 21854 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:20.065728 22003 rpcz_store.cc:276] 0709 13:08:18.424796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.426025 (+  1229us) service_pool.cc:224] Handling call
0709 13:08:18.434427 (+  8402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.737711 (+303284us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:18.755312 (+ 17601us) log.cc:844] Serialized 14 byte log entry
0709 13:08:18.890464 (+135152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:18.890532 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:18.890599 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.991779 (+1101180us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400167954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:19.991833 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.991957 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1063,"prepare.queue_time_us":17445,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":1207,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11111,"spinlock_wait_cycles":696968576,"thread_start_us":133510,"threads_started":1,"wal-append.queue_time_us":187195,"wal-append.run_cpu_time_us":1490,"wal-append.run_wall_time_us":1056572}
I20260709 13:08:19.987530 29118 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:20.068652 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 124840us
I20260709 13:08:20.119661 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.109880 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.108791 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:20.103781 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 140594us
I20260709 13:08:20.120507 21927 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:20.120666 21935 raft_consensus.cc:2410] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:20.121483 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:20.080837 29205 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:20.123811 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 190013us
I20260709 13:08:20.126783 21823 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.127070 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.119678 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:20.122282 21935 raft_consensus.cc:2393] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:20.122411 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 198779us
I20260709 13:08:20.117379 21853 pending_rounds.cc:85] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting all ops after (but not including) 13
I20260709 13:08:20.130231 21853 pending_rounds.cc:107] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:08:20.130514 21853 raft_consensus.cc:2889] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:20.129667 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.132249 21935 raft_consensus.cc:2410] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for term 23 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:20.116505 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3256) took 2529 ms. Trace:
I20260709 13:08:19.992154 29039 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:20.138782 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 189334us
I20260709 13:08:20.133921 21935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:20.000737 29279 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.115s
I20260709 13:08:20.104387 28900 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:20.146298 21972 leader_election.cc:400] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Vote denied by peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927) with higher term. Message: Invalid argument: T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20.
I20260709 13:08:20.112597 29054 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.082432 29148 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.117379 21934 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:20.192649 21934 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:20.108850 29147 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:20.082029 29152 raft_consensus.cc:740] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:20.136061 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:20.113092 29096 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:20.196887 29158 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:08:20.201603 29096 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.132745 21933 rpcz_store.cc:276] 0709 13:08:17.586669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.502149 (+915480us) service_pool.cc:224] Handling call
0709 13:08:18.502562 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.503001 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:18.528668 (+ 25667us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.433954 (+905286us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b18004e2ec0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:19.858152 (+424198us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18004e2ec0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2557 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:19.951836 (+ 93684us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b18004e2ec0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:08:20.096792 (+144956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:20.096878 (+    86us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:20.115428 (+ 18550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.116040 (+   612us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.116419 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13117,"prepare.queue_time_us":25917,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":204,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":23832,"spinlock_wait_cycles":33208192,"thread_start_us":144588,"threads_started":1}
I20260709 13:08:20.136000 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.115597 29024 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:20.084810 29085 consensus_queue.cc:1059] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:08:20.190208 28989 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:08:20.211499 29024 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.217193 28001 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.221315 21856 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:20.150247 21930 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:20.141182 21935 raft_consensus.cc:606] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:20.225407 21935 raft_consensus.cc:614] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:20.228550 21935 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:08:20.195086 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:20.190853 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:20.103672 29050 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.138728 28919 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:20.237421 22002 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:20.134016 29048 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:20.128345 28807 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:20.200119 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 64153.8us
I20260709 13:08:20.129570 29205 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:20.210191 29096 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.122453 29164 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:20.232079 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:20.249045 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 58248.1us
I20260709 13:08:20.125840 29010 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:08:20.251641 28001 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:20.251874 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.252185 28001 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:20.146910 29118 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:08:20.255694 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.242674 22005 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:20.235194 21933 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:20.243515 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.147655 29173 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:20.194298 29284 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:08:20.252668 28001 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:20.222110 29240 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.146166 29077 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.076s
I20260709 13:08:20.251050 28919 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:20.256371 22586 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:20.271917 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.263235 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.139863 29130 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.102s
I20260709 13:08:20.241228 29176 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:20.267481 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.253818 28807 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.265473 22587 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:20.275290 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:20.253198 29298 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.283601 28001 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.148932 28820 consensus_queue.cc:1059] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:08:20.206130 29054 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.266796 29147 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:20.265339 29173 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:20.283303 29176 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.289647 29232 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:20.282330 29302 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:20.221598 29281 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.275257 29240 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.297904 29147 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:20.464897 28001 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:20.465461 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 181063us
I20260709 13:08:20.470664 28820 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.470402 29147 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:20.389791 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 262881us
I20260709 13:08:20.467900 28919 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.476145 29017 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:20.391270 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 245393us
I20260709 13:08:20.401957 21855 raft_consensus.cc:1240] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:20.401587 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.487046 21853 raft_consensus.cc:2393] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:20.487887 29039 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:20.275476 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:20.258879 28917 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 13:08:20.304672 29173 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:20.310423 29204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.004s	sys 0.031s
I20260709 13:08:20.308274 29176 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:20.522059 29204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.004s	sys 0.032s
I20260709 13:08:20.335606 29240 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.528127 29148 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:20.330230 29282 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.641s	user 0.005s	sys 0.000s
I20260709 13:08:20.530473 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:20.534415 22591 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:20.535279 22592 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:08:20.284759 29297 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:08:20.536116 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.538867 22590 raft_consensus.cc:1240] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:20.541992 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.294507 29298 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.555588 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:20.319270 29300 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:08:20.458187 29288 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.290935 29152 consensus_queue.cc:260] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.557438 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.557628 29298 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.558555 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:20.559388 29152 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:20.559872 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:20.472797 22556 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:20.562366 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89893.8us
I20260709 13:08:20.562634 22590 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:20.490402 28807 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.557631 29288 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.490324 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:20.483744 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.491652 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.414566 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:20.491717 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.514094 22587 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:20.510972 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:20.542006 29309 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.067s	user 0.002s	sys 0.001s
W20260709 13:08:20.539616 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 65111.7us
I20260709 13:08:20.567025 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1445) took 2150 ms. Trace:
I20260709 13:08:20.472563 28820 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.581188 29298 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:20.572249 22585 rpcz_store.cc:276] 0709 13:08:18.416744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.416880 (+   136us) service_pool.cc:224] Handling call
0709 13:08:18.417402 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.698611 (+281209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.566460 (+1867849us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54007300d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:20.566841 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280419456}
I20260709 13:08:20.596565 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.600987 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:20.588691 28707 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:20.608791 28707 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.609185 28829 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.612627 28829 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.613210 21857 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:20.613366 21928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b42af043129e482498af1e2c3882a042"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:48000
I20260709 13:08:20.613976 22590 raft_consensus.cc:1275] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:20.614313 21928 raft_consensus.cc:606] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:20.614944 21928 raft_consensus.cc:614] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:20.617658 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:20.619015 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.632750 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:20.635617 29288 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.636425 28829 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.608657 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:20.635511 28807 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:20.657980 29147 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.658111 28807 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.658730 28807 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.661798 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:20.665566 29054 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.667218 22584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce"
dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
mode: GRACEFUL
new_leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
 from {username='slave'} at 127.0.0.1:35104
I20260709 13:08:20.667919 22584 raft_consensus.cc:606] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Received request to transfer leadership to TS 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:20.668275 22587 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:20.672652 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.685750 29148 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:20.686574 29148 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:20.692777 29017 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:20.693406 29017 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.693842 29017 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:20.694823 29085 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:20.694809 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:20.695729 29085 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.698206 29298 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:20.699153 29298 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.699895 29217 consensus_queue.cc:260] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.702003 29217 raft_consensus.cc:3060] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:20.721563 21927 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:20.707118 28820 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:20.708668 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59078.1us
I20260709 13:08:20.722604 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3288) took 1008 ms. Trace:
I20260709 13:08:20.723143 21927 rpcz_store.cc:276] 0709 13:08:19.714645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.647536 (+932891us) service_pool.cc:224] Handling call
0709 13:08:20.720893 (+ 73357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.722181 (+  1288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.722454 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:20.723682 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.709620 21857 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:20.726199 28820 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.726987 29085 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.716681 29298 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.716274 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:20.719799 29295 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.708s	user 0.000s	sys 0.002s
I20260709 13:08:20.720520 29276 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:20.730777 29297 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:20.713958 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3268) took 1221 ms. Trace:
I20260709 13:08:20.733240 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.732380 21931 rpcz_store.cc:276] 0709 13:08:19.492202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.220241 (+728039us) service_pool.cc:224] Handling call
0709 13:08:20.220582 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.220815 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:20.417915 (+197100us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.680896 (+262981us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.683471 (+  2575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:20.713358 (+ 29887us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:20.713487 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.713606 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.713773 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3502,"prepare.queue_time_us":150134,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":64,"raft.queue_time_us":28045,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":2423,"spinlock_wait_cycles":2688384,"thread_start_us":380799,"threads_started":2,"wal-append.queue_time_us":310110}
I20260709 13:08:20.735214 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.736181 21931 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:08:20.729485 29041 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:20.734362 21927 raft_consensus.cc:2393] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 22: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:20.738489 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:20.721922 29296 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.528s	user 0.000s	sys 0.002s
I20260709 13:08:20.743587 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:20.746622 28900 consensus_meta.cc:287] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.555s	user 0.000s	sys 0.025s
I20260709 13:08:20.688251 29148 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:20.761731 29135 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.729s	user 0.025s	sys 0.006s
I20260709 13:08:20.761533 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.763708 29204 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:20.764354 29275 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:20.764505 29204 consensus_queue.cc:1059] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:20.765635 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:20.772151 28900 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.773361 29294 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.774024 29294 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.774824 28900 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.757853 29085 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:20.771148 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.783067 29118 consensus_queue.cc:1059] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:08:20.785429 29085 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.796090 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.802974 29218 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:20.787151 29294 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:20.810753 29085 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:20.825716 21927 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:08:20.826622 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:20.781226 29135 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.824268 29298 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:20.842186 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3285) took 1188 ms. Trace:
I20260709 13:08:20.842995 22557 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.841346 21860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3111) took 1722 ms. Trace:
I20260709 13:08:20.844758 21935 rpcz_store.cc:276] 0709 13:08:19.653494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.238660 (+585166us) service_pool.cc:224] Handling call
0709 13:08:20.246825 (+  8165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.247479 (+   654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:20.247519 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.248148 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.525879 (+277731us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:20.526085 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.527439 (+  1354us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:20.840734 (+313295us) raft_consensus.cc:1685] finished
0709 13:08:20.840753 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.842004 (+  1251us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":12641,"threads_started":1,"wal-append.queue_time_us":482788}
I20260709 13:08:20.849112 21935 raft_consensus.cc:1240] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:08:20.853268 28820 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:20.854820 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2728) took 1015 ms. Trace:
I20260709 13:08:20.855413 21935 rpcz_store.cc:276] 0709 13:08:19.834615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.848062 (+1013447us) service_pool.cc:224] Handling call
0709 13:08:20.848671 (+   609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.849546 (+   875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.849729 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:20.860682 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1290) took 1716 ms. Trace:
I20260709 13:08:20.882283 29017 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.888942 22585 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:08:20.889200 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 107207us
I20260709 13:08:20.860970 29173 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.890015 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.890116 22587 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:08:20.890658 22593 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:20.892027 22584 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:20.893327 29173 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:20.893610 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 112454us
I20260709 13:08:20.894001 29276 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:20.894485 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.894718 29276 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.895475 29182 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:20.896031 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.896337 29182 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:20.896986 29298 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:20.897784 29298 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:20.900472 29024 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:20.900665 29325 log.cc:927] Time spent T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 13:08:20.902119 29320 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:08:20.902060 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.929440 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.930289 21859 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:19.764971 22588 rpcz_store.cc:276] 0709 13:08:17.983663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.983980 (+   317us) service_pool.cc:224] Handling call
0709 13:08:17.984388 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.984649 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:08:18.040608 (+ 55959us) log.cc:844] Serialized 14 byte log entry
0709 13:08:18.408931 (+368323us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b18003be600. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:08:18.647756 (+238825us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.121701 (+473945us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b18003be600. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:19.644879 (+523178us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18003be600. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287 00007f3b707ea208
0709 13:08:19.701139 (+ 56260us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.749232 (+ 48093us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.750200 (+   968us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.750512 (+   312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:19.750570 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:19.750640 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.750732 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.750865 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1096383,"prepare.queue_time_us":55093,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":5885,"raft.queue_time_us":1148203,"raft.run_cpu_time_us":645,"raft.run_wall_time_us":30390,"spinlock_wait_cycles":204071808,"thread_start_us":235211,"threads_started":2,"wal-append.queue_time_us":824346}
I20260709 13:08:20.931595 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.932154 29097 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.932407 28923 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:20.903728 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.932685 29097 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.933059 28923 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.893498 21926 raft_consensus.cc:2468] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:20.935040 29097 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.935372 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3275) took 1377 ms. Trace:
I20260709 13:08:20.935945 28923 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.935794 21926 rpcz_store.cc:276] 0709 13:08:19.558556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.660892 (+102336us) service_pool.cc:224] Handling call
0709 13:08:20.935185 (+1274293us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002e7cd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:20.935305 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2830077184}
I20260709 13:08:20.940526 21927 raft_consensus.cc:1240] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:20.941337 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2725) took 1108 ms. Trace:
I20260709 13:08:20.941776 21927 rpcz_store.cc:276] 0709 13:08:19.833213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.841700 (+1008487us) service_pool.cc:224] Handling call
0709 13:08:20.940151 (+ 98451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.941056 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.941277 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:20.946370 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:20.861289 21932 rpcz_store.cc:276] 0709 13:08:19.143822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.143944 (+   122us) service_pool.cc:224] Handling call
0709 13:08:19.382923 (+238979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.395702 (+ 12779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.860155 (+1464453us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b540017e650. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:20.860440 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529838976}
I20260709 13:08:20.908217 29276 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.909986 21851 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:20.949168 21932 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:20.949666 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:20.950762 21851 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:20.910465 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3266) took 1418 ms. Trace:
I20260709 13:08:20.911386 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3264) took 1420 ms. Trace:
I20260709 13:08:20.953179 21933 rpcz_store.cc:276] 0709 13:08:19.491766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.207485 (+715719us) service_pool.cc:224] Handling call
0709 13:08:20.234515 (+ 27030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.260044 (+ 25529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.910098 (+650054us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5400354dd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:20.910291 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60906880}
I20260709 13:08:20.911490 21857 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:08:20.955255 21933 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:20.953289 21934 rpcz_store.cc:276] 0709 13:08:19.491033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.206151 (+715118us) service_pool.cc:224] Handling call
0709 13:08:20.206710 (+   559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.450601 (+243891us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:20.579870 (+129269us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.622045 (+ 42175us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.641052 (+ 19007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:20.641104 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:20.641154 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.911031 (+269877us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540017f054. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:20.911071 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.911202 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18569,"prepare.queue_time_us":35313,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":103,"raft.queue_time_us":50073,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":7065,"spinlock_wait_cycles":553050368,"thread_start_us":139126,"threads_started":2,"wal-append.queue_time_us":53844}
I20260709 13:08:20.911942 21853 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: [20.13-22.14]
I20260709 13:08:20.910728 29017 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.861718 21935 raft_consensus.cc:1275] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:08:20.864742 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.957082 21858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41262 (request call id 2784) took 1124 ms. Trace:
I20260709 13:08:20.957600 21858 rpcz_store.cc:276] 0709 13:08:19.832937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.834711 (+  1774us) service_pool.cc:224] Handling call
0709 13:08:20.956606 (+1121895us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b540008acd4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:20.956880 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82950016}
W20260709 13:08:20.958098 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 172450us
I20260709 13:08:20.958705 29135 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.957617 29305 raft_consensus.cc:3055] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:08:20.914340 28984 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:08:20.869174 22009 raft_consensus.cc:3055] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:20.915657 29050 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:20.913690 28707 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:20.918289 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3290) took 1183 ms. Trace:
I20260709 13:08:20.918936 29182 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:20.919621 21855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2782) took 1087 ms. Trace:
I20260709 13:08:20.873675 21824 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:20.877719 29275 raft_consensus.cc:3055] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:08:20.878692 29205 consensus_meta.cc:287] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.011s
I20260709 13:08:20.922011 22000 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: [20.13-22.14]
I20260709 13:08:20.845829 21860 rpcz_store.cc:276] 0709 13:08:19.058137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.058285 (+   148us) service_pool.cc:224] Handling call
0709 13:08:19.068807 (+ 10522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:19.069129 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:19.146509 (+ 77380us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.341069 (+194560us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.377965 (+ 36896us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.378235 (+   270us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.378323 (+    88us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:19.395668 (+ 17345us) log.cc:844] Serialized 26 byte log entry
0709 13:08:19.395832 (+   164us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:20.516861 (+1121029us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b180045d360. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d65cc0a 00007f3b76d9c219 00007f3b6d6e4a40 00007f3b6d6df4e4 00007f3b6d6df455 00007f3b741c73c7 00007f3b707d99a7 00007f3b707e90bf 00007f3b707e9038
0709 13:08:20.576169 (+ 59308us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.771582 (+195413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.778412 (+  6830us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:20.780756 (+  2344us) raft_consensus.cc:1685] finished
0709 13:08:20.780778 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.780939 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1210698,"prepare.queue_time_us":3994194,"prepare.run_cpu_time_us":217,"prepare.run_wall_time_us":271,"raft.queue_time_us":1552,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":24717,"spinlock_wait_cycles":7096192,"thread_start_us":291098,"threads_started":3,"wal-append.queue_time_us":503016,"wal-append.run_cpu_time_us":4350,"wal-append.run_wall_time_us":1047459}
W20260709 13:08:20.870213 21852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 1149) took 2137 ms (client timeout 1786 ms). Trace:
I20260709 13:08:20.869076 28807 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:20.925086 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:20.987273 21934 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:20.949783 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2731) took 1114 ms. Trace:
I20260709 13:08:20.957295 21935 pending_rounds.cc:85] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:08:20.865209 29147 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:08:20.912518 21928 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:20.980633 21858 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:20.981230 22009 raft_consensus.cc:740] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:20.982096 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:20.960592 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:20.969935 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 73935.8us
I20260709 13:08:20.985693 21851 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:20.953467 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3284) took 1222 ms. Trace:
I20260709 13:08:20.953095 21931 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:08:20.951903 29085 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.984715 21929 rpcz_store.cc:276] 0709 13:08:19.734880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.738859 (+  3979us) service_pool.cc:224] Handling call
0709 13:08:20.917803 (+1178944us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b54002e9ad4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:20.918064 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180471040}
W20260709 13:08:20.986382 21852 rpcz_store.cc:269] 0709 13:08:18.732489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.732586 (+    97us) service_pool.cc:224] Handling call
0709 13:08:20.869808 (+2137222us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:20.870009 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105236736}
W20260709 13:08:20.925837 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.988076 21932 rpcz_store.cc:276] 0709 13:08:19.835662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.948463 (+1112801us) service_pool.cc:224] Handling call
0709 13:08:20.948858 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.949542 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.949730 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:20.985267 21855 rpcz_store.cc:276] 0709 13:08:19.832312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.834153 (+  1841us) service_pool.cc:224] Handling call
0709 13:08:19.834470 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.481762 (+647292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.919272 (+437510us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b540028c154. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:20.919550 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302189440}
I20260709 13:08:21.016515 29205 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.017388 28923 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:21.017877 28923 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.018213 28923 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.019203 29205 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.020798 22009 consensus_queue.cc:260] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:21.021860 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 61340us
I20260709 13:08:21.022730 22009 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:21.032097 22005 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:08:21.039770 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:20.959616 29305 raft_consensus.cc:740] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:20.988230 21935 pending_rounds.cc:107] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 20.12
W20260709 13:08:21.042549 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 169180us
I20260709 13:08:21.014276 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3289) took 1279 ms. Trace:
I20260709 13:08:21.045859 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2736) took 1148 ms. Trace:
W20260709 13:08:21.054643 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.055192 21935 raft_consensus.cc:2889] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:21.056586 21934 rpcz_store.cc:276] 0709 13:08:19.861242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.956050 (+1094808us) service_pool.cc:224] Handling call
0709 13:08:20.956395 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.987728 (+ 31333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.009542 (+ 21814us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.055567 21928 rpcz_store.cc:276] 0709 13:08:19.734588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.895755 (+1161167us) service_pool.cc:224] Handling call
0709 13:08:20.896149 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.009259 (+113110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.014205 (+  4946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.043502 29204 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:21.087428 28923 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:20.987646 29275 raft_consensus.cc:740] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:21.085079 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:21.095986 21852 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:21.088512 29284 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.015s	sys 0.015s
I20260709 13:08:21.043565 28807 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.044497 29291 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:21.099112 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2729) took 1264 ms. Trace:
I20260709 13:08:21.010604 21930 rpcz_store.cc:276] 0709 13:08:19.609314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.231202 (+621888us) service_pool.cc:224] Handling call
0709 13:08:20.231786 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.232070 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:20.232104 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.232707 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.417964 (+185257us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:20.811634 (+393670us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.819649 (+  8015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.824489 (+  4840us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:20.824538 (+    49us) raft_consensus.cc:1685] finished
0709 13:08:20.824549 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.831363 (+  6814us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67638,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":77,"raft.queue_time_us":7306,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2392,"thread_start_us":573160,"threads_started":2,"wal-append.queue_time_us":103874}
I20260709 13:08:21.106753 21931 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:08:20.883266 29176 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:21.108984 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 216355us
I20260709 13:08:21.134459 22593 raft_consensus.cc:1240] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:21.121426 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.118294 21935 rpcz_store.cc:276] 0709 13:08:19.834977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.860587 (+1025610us) service_pool.cc:224] Handling call
0709 13:08:20.861107 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.098775 (+237668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.099038 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.135942 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.145757 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2745) took 1271 ms. Trace:
W20260709 13:08:21.109668 29284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.015s	sys 0.016s
I20260709 13:08:21.148435 21931 rpcz_store.cc:276] 0709 13:08:19.874615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.084376 (+1209761us) service_pool.cc:224] Handling call
0709 13:08:21.103682 (+ 19306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.135539 (+ 31857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.145691 (+ 10152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.148947 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.107383 29052 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:21.150810 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 57207.9us
I20260709 13:08:21.103055 29024 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.153831 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.147382 22587 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:21.156010 22590 raft_consensus.cc:2410] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:08:21.105685 29204 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:21.044360 28984 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:08:21.136358 28919 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.162971 21935 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:21.165761 21926 raft_consensus.cc:1275] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 13:08:21.164860 29240 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.166935 29077 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:08:21.167441 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 99112.1us
I20260709 13:08:21.167186 29240 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.167219 29275 consensus_queue.cc:260] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.171231 29275 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:21.173519 29153 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.174151 29153 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.176535 29153 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.177384 28919 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.177989 22586 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:21.179023 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.179904 28919 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.180958 22593 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:08:21.181159 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:21.181764 22589 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:21.182075 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:21.184007 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:21.166599 21901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.184607 22585 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:21.184980 28919 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:08:21.185459 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 225916us
I20260709 13:08:21.188989 29240 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.189385 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:21.190213 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.191408 29299 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:21.192250 21854 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:21.193223 21855 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:21.192452 22592 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 2 to 12, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:21.194235 21853 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:21.194085 29182 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:21.194495 29085 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.195864 21857 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:08:21.200340 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.201076 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:21.166167 21926 pending_rounds.cc:85] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 11
I20260709 13:08:21.223241 21926 pending_rounds.cc:107] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:08:21.223505 21926 raft_consensus.cc:2889] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:21.203508 22584 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:21.107620 29097 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:21.190166 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:21.197690 21929 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:21.206130 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:21.204911 21856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41262 (request call id 2781) took 1275 ms. Trace:
I20260709 13:08:21.209856 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:21.209976 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.210659 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:21.165087 29024 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.229194 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.229993 22584 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 16.
I20260709 13:08:21.231076 29284 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:21.231220 29182 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:21.231648 29182 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:21.211398 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2746) took 1336 ms. Trace:
I20260709 13:08:21.231712 29284 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.234493 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.232430 21932 rpcz_store.cc:276] 0709 13:08:19.874915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.015524 (+1140609us) service_pool.cc:224] Handling call
0709 13:08:21.016090 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.016433 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.016470 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.017055 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:08:21.094322 (+ 77267us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:21.094366 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.094442 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.211204 (+116762us) raft_consensus.cc:1685] finished
0709 13:08:21.211224 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.211347 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":76894,"threads_started":1,"wal-append.queue_time_us":184113}
I20260709 13:08:21.235350 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.236449 22004 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:21.236853 22004 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:21.236958 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.237784 22004 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.213099 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.238823 22585 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 15.
I20260709 13:08:21.238979 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.239799 22002 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:21.233428 29052 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:08:21.233776 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.241279 29049 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.241796 22001 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 17: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:21.232605 29182 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.242000 29049 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.243119 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.212886 29160 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:21.244091 22001 raft_consensus.cc:1240] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:21.244138 29160 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:21.244441 29049 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:21.197439 28807 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:08:21.233453 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.245218 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2740) took 1382 ms. Trace:
I20260709 13:08:21.236490 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.218036 21854 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:21.218263 28820 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:21.218904 29204 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.219584 29205 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:21.227941 21858 raft_consensus.cc:2393] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 23: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:21.225587 21856 rpcz_store.cc:276] 0709 13:08:19.831961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.834056 (+  2095us) service_pool.cc:224] Handling call
0709 13:08:19.869797 (+ 35741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.221818 (+352021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.084813 (+862995us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54003d3954. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:21.107378 (+ 22565us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807970176}
I20260709 13:08:21.228415 21852 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:21.238150 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.162258 29148 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:08:21.210692 29305 consensus_queue.cc:260] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.246718 22589 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:21.242683 22004 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:21.247674 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 47545.6us
I20260709 13:08:21.245718 21929 rpcz_store.cc:276] 0709 13:08:19.863125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.013096 (+1149971us) service_pool.cc:224] Handling call
0709 13:08:21.151234 (+138138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.224801 (+ 73567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.245154 (+ 20353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.248596 21933 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:21.249944 22586 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:21.236972 21932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:58544 (request call id 1) took 1522 ms. Trace:
I20260709 13:08:21.266042 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.267110 21932 rpcz_store.cc:276] 0709 13:08:19.715003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.235839 (+1520836us) service_pool.cc:224] Handling call
0709 13:08:21.236884 (+  1045us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.234642 22008 raft_consensus.cc:1217] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Deduplicated request from leader. Original: 16.9->[19.10-23.12]   Dedup: 21.11->[23.12-23.12]
I20260709 13:08:21.254511 21825 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.261897 22588 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:21.255364 22005 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:21.251659 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.267231 21933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2735) took 1406 ms. Trace:
I20260709 13:08:21.263998 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.269670 21933 rpcz_store.cc:276] 0709 13:08:19.860847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.954701 (+1093854us) service_pool.cc:224] Handling call
0709 13:08:20.955091 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.266856 (+311765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.267162 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.224041 29054 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.272847 22008 raft_consensus.cc:1240] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 23. Current term is 24. Ops: [19.10-23.12]
I20260709 13:08:21.250454 28707 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.278234 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.281065 22003 raft_consensus.cc:2410] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 13:08:21.282825 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.285512 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.306282 22005 raft_consensus.cc:1275] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:21.310698 22557 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.266878 29299 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.313110 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.317375 22007 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:21.283105 29130 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:21.318157 29130 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:21.317981 29173 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:21.319350 29130 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.315650 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.321363 21852 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:21.313275 21926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2733) took 1476 ms. Trace:
I20260709 13:08:21.323897 21926 rpcz_store.cc:276] 0709 13:08:19.836395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.951525 (+1115130us) service_pool.cc:224] Handling call
0709 13:08:20.951994 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.250068 (+298074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.306773 (+ 56705us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b540017fa54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:21.313188 (+  6415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490812288}
I20260709 13:08:21.313381 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:21.329483 21854 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:21.315618 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 114685us
W20260709 13:08:21.286082 28652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.048s
I20260709 13:08:21.330700 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.283115 29147 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:21.279883 29085 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.290552 28707 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.316370 29305 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:21.362454 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.353434 22003 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:21.359344 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2730) took 1524 ms. Trace:
W20260709 13:08:21.352335 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:21.359421 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 149661us
W20260709 13:08:21.337599 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52162.5us
I20260709 13:08:21.331986 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.334249 21853 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:21.325901 29153 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:21.320921 29130 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:21.368180 22000 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: []
W20260709 13:08:21.334882 28652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.013s	sys 0.058s
I20260709 13:08:21.374984 22005 raft_consensus.cc:1240] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:21.368566 21927 rpcz_store.cc:276] 0709 13:08:19.835297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.942654 (+1107357us) service_pool.cc:224] Handling call
0709 13:08:20.943139 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.943634 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:20.943667 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.944212 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:08:21.088296 (+144084us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:21.134890 (+ 46594us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.135106 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.135173 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.194103 (+ 58930us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b180036d4e0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:21.358987 (+164884us) raft_consensus.cc:1685] finished
0709 13:08:21.359107 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.359271 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5164,"prepare.queue_time_us":45902,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":3655,"raft.queue_time_us":21068,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":4764,"spinlock_wait_cycles":8409856,"thread_start_us":143652,"threads_started":1,"wal-append.queue_time_us":317514}
I20260709 13:08:21.335258 29182 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.349241 28820 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:21.290354 29054 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:21.402114 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 128889us
I20260709 13:08:21.401762 29130 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:08:21.402688 29054 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.403398 21974 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.405288 29049 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:21.406806 29049 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:21.408695 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.410179 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.410751 28707 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.411626 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.412267 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.412467 22005 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 15.
I20260709 13:08:21.384512 29085 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.392338 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:21.412979 28850 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:21.413460 28850 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.413815 22000 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:21.414459 29052 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:21.371358 29153 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.414836 29052 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:21.415135 29153 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:21.416002 29052 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.360308 29017 consensus_queue.cc:1059] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:08:21.384858 29147 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:21.422053 22002 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.422502 29147 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.391637 22589 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:21.412637 22003 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:08:21.413818 28850 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:21.414848 29085 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.431658 29085 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.432098 28886 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:21.432569 29085 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.432890 28886 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:21.434060 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 239646us
I20260709 13:08:21.435791 28886 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.434193 29024 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.367383 28923 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.464470 22593 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:08:21.465113 22001 raft_consensus.cc:1240] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:21.435792 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.470206 21854 raft_consensus.cc:2468] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:21.479038 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.481400 29240 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:21.482470 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70652.5us
I20260709 13:08:21.481771 21757 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 5 to 12, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.482822 29240 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:21.337598 29097 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.349017 29299 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.493095 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.315313 29160 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.374306 29289 consensus_queue.cc:1059] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.102s
I20260709 13:08:21.484676 29077 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:21.497112 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.497910 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:21.517506 21857 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.468255 28820 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:21.499325 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.515966 21974 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.500247 29173 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:21.500563 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.500622 29281 consensus_queue.cc:1059] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:21.502074 29032 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.502271 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:21.504667 21932 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [17.16-20.17]
I20260709 13:08:21.548370 22009 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:21.508409 29240 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:21.509598 21898 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.511181 29097 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.498607 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.519311 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:21.521487 22002 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:21.523797 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:21.531280 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 64446.7us
I20260709 13:08:21.550722 21933 raft_consensus.cc:2393] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:21.523270 21860 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:21.552871 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.559875 22008 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:21.563934 21853 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:08:21.514271 29284 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.517284 29176 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:21.566991 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3154) took 1696 ms. Trace:
I20260709 13:08:21.579144 22009 rpcz_store.cc:276] 0709 13:08:19.866077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.868278 (+  2201us) service_pool.cc:224] Handling call
0709 13:08:19.868616 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.868863 (+1000247us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b54002d5620. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70746dc0 00007f3b707d11c0 00007f3b707d2231 00007f3b707ced8f 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c
0709 13:08:21.548793 (+679930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.562620 (+ 13827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193513344}
W20260709 13:08:21.578565 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 410314us
I20260709 13:08:21.560266 29135 raft_consensus.cc:2804] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:21.583977 21898 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.563294 29217 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:21.567021 29049 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:21.577564 29160 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:21.568555 28652 raft_consensus.cc:3055] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:21.566910 29024 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.596397 29240 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:21.611593 21926 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.614637 21927 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:08:21.619977 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:21.566516 29077 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.623956 29077 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.625069 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:21.626300 29344 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:08:21.629319 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:21.630535 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 263186us
I20260709 13:08:21.631387 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:21.633083 29238 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:08:21.635550 29238 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:21.636004 29238 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:21.636462 29238 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.639878 29135 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:21.640638 29135 raft_consensus.cc:3060] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:21.651747 29339 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.002s
I20260709 13:08:21.645865 29182 raft_consensus.cc:3055] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:21.653980 29182 raft_consensus.cc:740] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:21.655268 29182 consensus_queue.cc:260] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.647234 29085 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.647739 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:21.657739 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50288.4us
I20260709 13:08:21.658985 29050 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:21.659075 21974 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:21.659390 29050 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:21.659786 29050 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:21.659941 29333 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:21.661145 29153 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.662087 29333 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:21.662514 29333 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.662847 29333 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:21.665941 29291 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:21.667147 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1307) took 1006 ms. Trace:
I20260709 13:08:21.667531 21931 rpcz_store.cc:276] 0709 13:08:20.661212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.186082 (+524870us) service_pool.cc:224] Handling call
0709 13:08:21.186448 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.186957 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.186986 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.187535 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:08:21.466090 (+278555us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:21.466141 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.466234 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.666938 (+200704us) raft_consensus.cc:1685] finished
0709 13:08:21.666958 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.667089 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34974,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":25608,"thread_start_us":278150,"threads_started":1,"wal-append.queue_time_us":422463}
I20260709 13:08:21.668313 29050 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.608846 28652 raft_consensus.cc:740] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:21.669909 28652 consensus_queue.cc:260] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.670219 29024 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.670691 29050 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.671228 28652 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.672438 29024 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.578606 29297 consensus_queue.cc:1048] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:08:21.673254 29182 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:21.611255 29217 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:21.675594 29217 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.614745 29240 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.676765 29240 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.648849 29299 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.613909 28707 raft_consensus.cc:2804] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:08:21.678566 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.678992 28707 raft_consensus.cc:697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:21.679170 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.577373 28820 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.648506 29049 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:21.632856 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.590906 28850 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.669921 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:21.700246 22004 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:08:21.693897 21926 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:21.689127 29153 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:21.704733 29054 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:21.700768 22004 pending_rounds.cc:85] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 9
I20260709 13:08:21.705946 22004 pending_rounds.cc:107] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:08:21.706287 22004 raft_consensus.cc:2889] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:21.705298 29054 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.708391 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:21.712350 21853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.716125 29085 consensus_queue.cc:1059] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:08:21.711056 29077 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.724475 29085 raft_consensus.cc:3055] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:21.724380 29077 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.724853 29085 raft_consensus.cc:740] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:21.725821 29085 consensus_queue.cc:260] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.726599 29077 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:21.727196 29085 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:21.729326 21860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:21.730094 21749 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 194) took 40566 ms (client timeout 15000 ms). Trace:
I20260709 13:08:21.731168 29344 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:21.731711 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:21.731799 29344 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:21.734370 21757 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 9 to 13, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 13 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:21.737411 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 104760us
I20260709 13:08:21.736732 21757 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.738467 28807 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:21.738953 28807 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.739413 28807 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:21.740127 21757 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 10 to 13, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 13 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.741688 28850 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.721572 29054 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.745832 21900 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:21.745672 28707 consensus_queue.cc:237] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.746830 29341 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:21.746658 21757 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 10 to 12, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 12 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:21.741981 21934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2748) took 1331 ms. Trace:
W20260709 13:08:21.730619 21749 rpcz_store.cc:269] 0709 13:07:41.163613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.173182 (+  9569us) service_pool.cc:224] Handling call
0709 13:08:21.729784 (+40556602us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:14.647714 (+     0us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b6d6c50ac 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b7420416c
0709 13:08:14.968397 (+320683us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:14.969117 (+   720us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:14.969168 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:08:14.969185 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:14.996730 (+ 27545us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:14.996821 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:14.996853 (+    32us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:08:14.998715 (+  1862us) tablet.cc:701] Row locks acquired
0709 13:08:14.998766 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:08:14.998960 (+   194us) write_op.cc:270] Start()
0709 13:08:14.999204 (+   244us) write_op.cc:276] Timestamp: P: 1783602494998890 usec, L: 0
0709 13:08:14.999230 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:08:15.013335 (+ 14105us) log.cc:844] Serialized 8639 byte log entry
0709 13:08:16.963269 (+1949934us) op_driver.cc:464] REPLICATION: finished
0709 13:08:17.203237 (+239968us) write_op.cc:301] APPLY: starting
0709 13:08:17.207969 (+  4732us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:17.958557 (+750588us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:17.958611 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:18.355628 (+397017us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:18.704022 (+348394us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:08:18.711414 (+  7392us) write_op.cc:312] APPLY: finished
0709 13:08:18.861120 (+149706us) log.cc:844] Serialized 196 byte log entry
0709 13:08:18.926564 (+ 65444us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.947794 (+ 21230us) write_op.cc:454] Released schema lock
0709 13:08:18.958171 (+ 10377us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:18.982901 (+ 24730us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.982937 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":90129,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":164964,"apply.run_cpu_time_us":59977,"apply.run_wall_time_us":1797845,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96219,"num_ops":22,"prepare.queue_time_us":520686,"prepare.run_cpu_time_us":29560,"prepare.run_wall_time_us":356610,"raft.queue_time_us":377187,"raft.run_cpu_time_us":26513,"raft.run_wall_time_us":142099,"replication_time_us":1963810,"spinlock_wait_cycles":35877504,"thread_start_us":1197392,"threads_started":5,"wal-append.queue_time_us":1340664,"wal-append.run_cpu_time_us":39255,"wal-append.run_wall_time_us":2726232}]]}
I20260709 13:08:21.748819 21934 rpcz_store.cc:276] 0709 13:08:20.410770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.100524 (+689754us) service_pool.cc:224] Handling call
0709 13:08:21.149381 (+ 48857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.245793 (+ 96412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.245833 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.267442 (+ 21609us) log.cc:844] Serialized 26 byte log entry
0709 13:08:21.580615 (+313173us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:21.580664 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.580818 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.741760 (+160942us) raft_consensus.cc:1685] finished
0709 13:08:21.741783 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.741922 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12299,"raft.queue_time_us":34912,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":24469,"thread_start_us":263861,"threads_started":1,"wal-append.queue_time_us":458670}
I20260709 13:08:21.754247 29032 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:21.754339 21900 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:21.757160 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.759219 29333 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:21.723469 29284 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:21.738420 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.766878 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:21.798188 21931 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:21.767647 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:21.768783 21929 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: [16.10-18.11]
W20260709 13:08:21.769174 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 94e8056fad5248118e00e80f8a81538c: Illegal state: Replica 821b3831b8a348c0826bd3a20a857f00 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:21.771180 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:21.708768 21928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.026s	sys 0.016s
W20260709 13:08:21.786670 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:21.798025 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 409633us
W20260709 13:08:21.800745 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:21.801820 21928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.026s	sys 0.016s
I20260709 13:08:21.776345 21757 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:21.796753 21857 raft_consensus.cc:2468] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:21.811561 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.813922 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.815125 22009 raft_consensus.cc:2410] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:08:21.819521 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.820415 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.825798 22004 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:21.826522 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.816159 21932 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:21.773435 29341 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:21.810510 29284 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:21.818667 29024 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:21.857811 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:21.830382 29160 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:21.878928 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:21.885406 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.886711 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:21.887053 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.888305 22587 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:21.887959 22586 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:21.889799 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:21.895536 22588 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 16: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:21.899631 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:21.900435 22006 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.901450 21757 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:21.902065 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:21.904001 21757 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 7 to 11, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 11 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:21.864713 29284 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:21.922695 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:21.909276 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.833387 28850 consensus_queue.cc:1059] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:08:21.879369 29176 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.922585 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:21.952806 22006 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:21.881314 29341 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:21.851689 29305 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 13:08:21.965396 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 496823us
I20260709 13:08:21.949834 21757 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 10 to 12, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:21.957967 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 473099us
I20260709 13:08:21.972127 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:21.972898 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 234604us
I20260709 13:08:21.978963 22003 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 25: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:21.921243 29147 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:21.969430 29339 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:22.003861 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:21.953559 29298 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:21.955734 29052 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:21.923677 29077 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.089s
I20260709 13:08:22.030838 21757 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 8 to 12, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:22.042243 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:21.993103 29017 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.989749 29024 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.011454 29289 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.065567 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:22.052975 28652 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:22.066069 29289 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.022738 29130 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.072831 29130 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.048087 21757 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 8 to 11, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 11 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:22.056128 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:22.089126 28984 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:22.063746 29024 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:21.979614 29176 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.091284 21757 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:22.092545 29176 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:22.065303 29333 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:22.074496 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33888 (request call id 2747) took 2199 ms. Trace:
I20260709 13:08:22.070425 29013 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:22.094694 29013 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:22.094619 21928 rpcz_store.cc:276] 0709 13:08:19.875194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.094932 (+1219738us) service_pool.cc:224] Handling call
0709 13:08:21.151760 (+ 56828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.166581 (+ 14821us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:08:21.166931 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.074045 (+907114us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b1800362740. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287
0709 13:08:22.074130 (+    85us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:22.074176 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:22.074256 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.074327 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.074440 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23472,"prepare.queue_time_us":2391,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":242270,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":39845,"spinlock_wait_cycles":13777152,"thread_start_us":778885,"threads_started":2,"wal-append.queue_time_us":800966}
I20260709 13:08:22.095639 29013 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.096763 29218 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:22.097139 29013 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:22.079686 29346 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:08:22.105094 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.080232 29019 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.229s
I20260709 13:08:22.082361 29284 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:22.085229 29349 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 13:08:22.087029 29275 raft_consensus.cc:3055] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:22.068286 29289 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:22.012662 28923 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.030215 28919 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:22.020740 29204 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:22.103811 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 711535us
I20260709 13:08:22.079864 29173 consensus_queue.cc:1059] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:21.987601 29054 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:22.115707 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 74360.1us
I20260709 13:08:22.115801 21931 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:22.015480 29147 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.105865 29049 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:22.019623 29135 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.132790 29275 raft_consensus.cc:740] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:22.141212 21899 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.021518 29341 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.146180 22006 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:22.114933 29333 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:22.155900 28820 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.129536 29305 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.134125 29344 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.139896 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:22.189620 22004 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:22.101838 29017 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.140242 28919 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.194931 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1468) took 1014 ms. Trace:
I20260709 13:08:22.195185 28919 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:22.195408 22592 rpcz_store.cc:276] 0709 13:08:21.180203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.185402 (+  5199us) service_pool.cc:224] Handling call
0709 13:08:21.185705 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.192216 (+  6511us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:21.306202 (+113986us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.860305 (+554103us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.863314 (+  3009us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.899160 (+ 35846us) log.cc:844] Serialized 26 byte log entry
0709 13:08:21.899374 (+   214us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:21.899423 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.899505 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.194689 (+295184us) raft_consensus.cc:1685] finished
0709 13:08:22.194711 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.194865 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11151,"prepare.queue_time_us":23805,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4105,"raft.queue_time_us":11761,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":10577,"thread_start_us":446681,"threads_started":1,"wal-append.queue_time_us":715755}
W20260709 13:08:22.197109 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 178231us
I20260709 13:08:22.198719 21972 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:22.107867 28984 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 13:08:22.175383 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 421079us
W20260709 13:08:22.199616 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.167829 21898 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:22.177284 21926 raft_consensus.cc:1240] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 17. Current term is 19. Ops: []
I20260709 13:08:22.201042 29344 raft_consensus.cc:2804] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:22.160952 29049 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:22.201779 29344 raft_consensus.cc:697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:22.202469 29049 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.202800 29344 consensus_queue.cc:237] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.203892 29049 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:22.206130 29153 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.206720 29153 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.208308 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.208823 29297 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:22.209124 29017 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.209450 29297 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:22.179018 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.199721 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:22.211144 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.211231 21899 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.212528 21899 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.186919 29130 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.213445 29153 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.213601 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.214840 28807 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.218359 29341 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.219007 29341 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:22.219656 29341 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:22.220019 29341 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:22.220376 29341 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:22.145466 29024 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:08:22.220741 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.151598 29204 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.221472 29275 consensus_queue.cc:260] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.222100 29024 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:22.222885 29275 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:22.223658 29204 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:22.187194 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:22.188388 29305 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.234162 22584 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 16.
I20260709 13:08:22.145618 29097 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:08:22.201195 29284 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:22.227486 22004 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
W20260709 13:08:22.186223 29357 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:22.234104 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.245045 21824 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:22.211524 29374 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:22.240435 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53368.5us
I20260709 13:08:22.144526 28923 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.261288 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.263619 21855 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:22.263746 22004 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:22.262216 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.266381 21857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:22.266992 21857 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:22.265153 22008 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:22.166019 29173 consensus_queue.cc:1243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:22.273746 21932 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:08:22.274385 29362 log.cc:927] Time spent T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:08:22.275378 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1162) took 1362 ms. Trace:
W20260709 13:08:22.276046 29365 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 13:08:22.277097 29135 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.277441 21926 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:22.234603 28820 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.280577 29305 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.181088 29217 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:22.284318 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:22.252245 29085 consensus_meta.cc:287] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.525s	user 0.005s	sys 0.008s
I20260709 13:08:22.210421 29297 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.260569 29130 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:22.275796 21859 rpcz_store.cc:276] 0709 13:08:20.912528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.932573 (+ 20045us) service_pool.cc:224] Handling call
0709 13:08:20.932990 (+   417us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:08:21.085391 (+152401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.085440 (+    49us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:08:21.202918 (+117478us) log.cc:844] Serialized 104 byte log entry
0709 13:08:21.580333 (+377415us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:21.616470 (+ 36137us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.621248 (+  4778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.621334 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.275137 (+653803us) raft_consensus.cc:1685] finished
0709 13:08:22.275160 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.275313 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5716,"prepare.queue_time_us":35916,"prepare.run_cpu_time_us":185,"prepare.run_wall_time_us":186,"raft.queue_time_us":15110,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":35840,"thread_start_us":313275,"threads_started":1,"wal-append.queue_time_us":584074}
I20260709 13:08:22.304584 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:22.325443 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.326473 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:22.300794 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:22.323065 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:22.335090 21926 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:22.268141 29153 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:08:22.304080 29238 consensus_meta.cc:287] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.000s
I20260709 13:08:22.338629 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.281261 29054 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:22.329877 21931 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 2 to 12, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:22.346184 29130 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:22.346779 29363 consensus_queue.cc:1048] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:08:22.347193 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:22.329898 29240 consensus_meta.cc:287] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.000s
I20260709 13:08:22.349175 21934 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:22.354729 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:22.311576 29024 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:22.346019 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:22.345981 22004 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:22.355396 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:22.359988 21857 raft_consensus.cc:2468] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:22.334447 29305 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:22.347477 29371 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:22.336444 28807 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:22.368741 21932 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:22.374622 21898 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:22.262523 29204 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:08:22.323172 29173 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
W20260709 13:08:22.365463 29361 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.142s	user 0.000s	sys 0.004s
I20260709 13:08:22.389997 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:22.396468 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:22.397020 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.397557 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.402483 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.377612 29305 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.378969 29298 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:22.353664 29153 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:22.375967 29240 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.423954 29305 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:22.389500 29173 consensus_queue.cc:1243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:22.354636 29238 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.452945 22006 raft_consensus.cc:1240] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:22.453153 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.453706 29378 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:08:22.454181 29204 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:22.455209 29238 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:22.446084 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 71590us
I20260709 13:08:22.448503 29366 raft_consensus.cc:2804] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:22.456322 29366 raft_consensus.cc:697] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:22.444463 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:22.454854 29204 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.457093 29374 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.457111 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:22.458681 29204 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:22.409034 22586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.005s	sys 0.017s
W20260709 13:08:22.460382 22586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.005s	sys 0.018s
I20260709 13:08:22.461149 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:22.449224 29182 consensus_meta.cc:287] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.775s	user 0.010s	sys 0.003s
I20260709 13:08:22.462073 29173 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:22.462481 29182 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:22.462385 29341 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.462625 29173 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.462777 29204 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:22.463696 29204 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.463548 21929 raft_consensus.cc:1240] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:22.464083 29204 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:22.458715 21899 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.465942 29054 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:22.466218 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:22.467573 21859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:22.463039 29182 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.469234 29375 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:08:22.470084 29182 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:22.469269 29341 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:22.478191 22008 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:22.481498 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:22.409492 28820 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.376318 28919 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.505755 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:22.464767 29173 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.423964 29359 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:22.431896 29298 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.443405 29097 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:22.513535 29298 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:22.516671 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.517343 29363 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.517359 22008 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:22.518062 29363 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.518596 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.519287 22008 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:22.520237 29363 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.520484 22008 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:08:22.520852 29372 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.521406 29153 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.16, Last appended by leader: 16, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.521427 29372 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.524694 29372 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.372517 29364 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:22.510622 29341 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:08:22.525681 29369 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:08:22.528007 29373 log.cc:927] Time spent T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:22.529381 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:22.532330 22008 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:08:22.541587 22008 pending_rounds.cc:85] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 16
I20260709 13:08:22.541821 22008 pending_rounds.cc:107] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:08:22.542052 22008 raft_consensus.cc:2889] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:22.533510 29352 raft_consensus.cc:3055] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:22.533396 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:22.493942 29135 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.552668 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:22.510114 29359 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.557821 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.560554 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.561640 22001 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:22.562537 29382 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.546932 29352 raft_consensus.cc:740] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:22.563174 29382 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.564234 29352 consensus_queue.cc:260] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.566613 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:22.556274 28850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.003s	sys 0.011s
I20260709 13:08:22.549180 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.541095 29341 consensus_queue.cc:1243] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:22.507735 29217 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:22.570729 21929 raft_consensus.cc:1275] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
W20260709 13:08:22.507270 22557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.005s	sys 0.030s
I20260709 13:08:22.572373 29352 raft_consensus.cc:3060] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:22.573184 29355 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.360s	user 0.002s	sys 0.001s
W20260709 13:08:22.575073 28850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.003s	sys 0.011s
I20260709 13:08:22.574746 21855 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:22.561228 29359 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:22.594686 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:22.594950 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:22.593497 21898 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:22.583581 22557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.005s	sys 0.030s
I20260709 13:08:22.556912 29147 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.599159 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.600630 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:22.570334 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 371670us
I20260709 13:08:22.578593 21929 pending_rounds.cc:85] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 15
I20260709 13:08:22.542243 29364 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.595958 21928 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:08:22.602643 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:22.603060 21929 pending_rounds.cc:107] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:08:22.628710 21929 raft_consensus.cc:2889] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:22.599251 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:22.610603 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.617259 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.586606 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:22.626147 28919 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:22.632189 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:22.560699 29135 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.536759 29374 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.636595 21935 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:22.579008 29363 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:22.640134 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.565505 29097 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.645939 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.642594 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:22.640837 29052 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.004s
I20260709 13:08:22.536019 29130 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.674430 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:22.680572 22589 raft_consensus.cc:2393] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 25: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:08:22.684741 22005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.014s	sys 0.010s
I20260709 13:08:22.590494 29217 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.646951 29049 raft_consensus.cc:2764] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:08:22.647612 29147 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:22.670078 29284 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:22.639920 28807 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:22.718402 21926 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:22.749771 21823 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.014s	sys 0.025s
W20260709 13:08:22.750358 21823 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.015s	sys 0.025s
I20260709 13:08:22.674100 29384 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:22.713618 29382 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.647650 29298 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.716339 29052 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:22.763000 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.671958 29363 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:22.781365 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.735826 28807 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.750430 29240 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:22.750891 28850 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:22.753163 29344 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:22.646683 29366 consensus_queue.cc:237] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:22.807150 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 190025us
I20260709 13:08:22.812747 21926 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:22.813862 21855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:22.785470 29382 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:22.784596 29147 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:22.814193 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 139434us
W20260709 13:08:22.849808 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.874903 21856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.875680 21860 raft_consensus.cc:1240] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: [19.13-20.14]
I20260709 13:08:22.851656 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:22.886325 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:22.886950 21854 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:22.896003 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.788609 29052 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:22.897796 21934 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.005s
I20260709 13:08:22.898537 21934 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
I20260709 13:08:22.899614 21852 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:08:22.903661 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 261160us
I20260709 13:08:22.905812 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.907352 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.908002 21851 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:22.908113 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:22.820636 28807 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:22.883139 29240 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:22.933838 22592 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:08:22.925216 21851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.932843 21858 raft_consensus.cc:2410] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:08:22.932865 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:22.940742 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b"
I20260709 13:08:22.941404 21852 raft_consensus.cc:3060] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:22.900036 29085 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.940106 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:22.934464 22592 pending_rounds.cc:85] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 12
I20260709 13:08:22.950214 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:22.950598 22592 pending_rounds.cc:107] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:08:22.951090 22588 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:22.951540 22592 raft_consensus.cc:2889] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:22.950575 22590 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:22.876081 29382 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.961475 22592 pending_rounds.cc:107] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:08:22.964504 22592 raft_consensus.cc:2889] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:22.960459 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:22.989810 21852 raft_consensus.cc:2468] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25.
I20260709 13:08:22.992839 21854 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:22.981796 22001 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:22.960691 29173 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:23.005576 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:22.910883 29147 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:22.984582 29391 log.cc:927] Time spent T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 13:08:23.010264 21860 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [18.14-21.15]
I20260709 13:08:22.950938 29240 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:22.982371 29384 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:23.025691 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 127705us
I20260709 13:08:23.040423 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.040061 29384 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:22.982739 29364 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.043390 29019 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:08:22.985682 29341 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:23.026623 29399 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:08:22.964185 29085 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:23.028007 29392 log.cc:927] Time spent T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:22.996316 29423 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:23.028795 29396 log.cc:927] Time spent T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:08:23.033646 29398 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 13:08:23.034545 29390 log.cc:927] Time spent T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:08:23.035006 29240 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.038987 29333 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:23.026530 29395 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:23.053359 21856 raft_consensus.cc:2468] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:08:23.051456 29393 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
I20260709 13:08:23.005798 29374 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.016791 29097 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.126802 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:23.051728 29217 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.134684 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3175) took 1361 ms. Trace:
W20260709 13:08:23.101879 29305 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 13:08:23.135460 22005 rpcz_store.cc:276] 0709 13:08:21.773350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.774444 (+  1094us) service_pool.cc:224] Handling call
0709 13:08:21.796751 (+ 22307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.796982 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:08:22.180305 (+383323us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.134012 (+953707us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.134102 (+    90us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:23.134214 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.134342 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.134571 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46754,"prepare.queue_time_us":82191,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":32,"raft.queue_time_us":224077,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":18382,"thread_start_us":1241388,"threads_started":3,"wal-append.queue_time_us":660670}
I20260709 13:08:23.157063 21934 raft_consensus.cc:1217] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Deduplicated request from leader. Original: 16.13->[18.14-21.15]   Dedup: 18.14->[21.15-21.15]
I20260709 13:08:23.157639 21934 raft_consensus.cc:1240] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [18.14-21.15]
I20260709 13:08:23.054536 29341 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:23.102756 29204 consensus_meta.cc:287] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.000s
W20260709 13:08:23.150851 29399 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:08:23.171496 21898 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:23.144332 29173 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:23.182116 29240 raft_consensus.cc:2804] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:23.149415 29160 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:08:23.170823 21900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.004s	sys 0.009s
I20260709 13:08:23.185436 21854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
W20260709 13:08:23.191618 21900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.004s	sys 0.010s
I20260709 13:08:23.192029 21854 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:23.192205 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 605627us
I20260709 13:08:23.186165 21852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:23.193177 21852 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:23.186731 21860 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:23.194061 21900 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.197463 21858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8f3b35a35257416db3eaaf364cf6b52b" is_pre_election: true
I20260709 13:08:23.198139 21858 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:23.186285 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
I20260709 13:08:23.212466 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.178731 29013 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:23.189072 29173 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.154780 29305 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:23.130106 29423 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.194510 29289 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.198200 29052 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.199242 29240 raft_consensus.cc:697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:23.183597 29374 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:08:23.249547 29289 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.216930 29013 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:23.251967 29289 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.251765 29426 consensus_queue.cc:1048] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:23.252926 29013 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:23.253980 29013 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:23.254660 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3292) took 1737 ms. Trace:
I20260709 13:08:23.255049 21930 rpcz_store.cc:276] 0709 13:08:21.517054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.522808 (+  5754us) service_pool.cc:224] Handling call
0709 13:08:21.578065 (+ 55257us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:21.733425 (+155360us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4800241ed8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b756f8aa1 00007f3b756efb37 00007f3b756ef9ca 00007f3b757e726f 00007f3b757e6f37 00007f3b757fb55c 00007f3b757fb508 00007f3b757fb488 00007f3b757fb443 00007f3b757fa6ec 00007f3b74724a96
0709 13:08:22.161814 (+428389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.162152 (+   338us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:22.174921 (+ 12769us) log.cc:844] Serialized 52 byte log entry
0709 13:08:22.588924 (+414003us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:22.605017 (+ 16093us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.630730 (+ 25713us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.764194 (+133464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.764288 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.254433 (+490145us) raft_consensus.cc:1685] finished
0709 13:08:23.254455 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.254601 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25308,"prepare.queue_time_us":16176,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":1961,"raft.queue_time_us":132622,"raft.run_cpu_time_us":1173,"raft.run_wall_time_us":60863,"spinlock_wait_cycles":28135936,"thread_start_us":910616,"threads_started":3,"wal-append.queue_time_us":639731}
I20260709 13:08:23.210345 28923 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.179065 29204 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:23.218626 29352 consensus_meta.cc:287] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.003s
I20260709 13:08:23.236830 29408 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.278702 21857 raft_consensus.cc:1240] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:23.234169 29333 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:08:23.238641 29384 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.243752 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3367) took 1077 ms. Trace:
I20260709 13:08:23.245039 29410 consensus_queue.cc:1059] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:08:23.214556 29431 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:23.226611 29077 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:08:23.226830 29173 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:23.246708 29409 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:23.248396 29341 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.219884 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:23.298691 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 78896us
W20260709 13:08:23.282660 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.292150 22002 rpcz_store.cc:276] 0709 13:08:22.165869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.166092 (+   223us) service_pool.cc:224] Handling call
0709 13:08:22.183022 (+ 16930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.224543 (+ 41521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.224574 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.225112 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.600492 (+375380us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:22.600542 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.600621 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.243496 (+642875us) raft_consensus.cc:1685] finished
0709 13:08:23.243521 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.243683 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9584,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":21774,"spinlock_wait_cycles":790400,"thread_start_us":363331,"threads_started":1,"wal-append.queue_time_us":934889}
W20260709 13:08:23.290907 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 278377us
W20260709 13:08:23.290762 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 246956us
I20260709 13:08:23.277117 29217 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:23.307744 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:23.293354 29333 consensus_queue.cc:1243] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:08:23.323253 21855 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:23.321543 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:23.322232 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:23.325639 22593 raft_consensus.cc:3055] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:23.326987 22593 raft_consensus.cc:740] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:23.303828 29374 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:08:23.321974 29359 consensus_meta.cc:287] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.702s	user 0.012s	sys 0.000s
I20260709 13:08:23.286479 29024 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:08:23.341464 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:23.349980 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.352167 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.353938 29415 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.354597 29415 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:23.355590 29182 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.885s	user 0.011s	sys 0.001s
I20260709 13:08:23.356093 29182 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.358403 29305 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.360219 29305 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:23.291535 29408 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.361261 22589 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:08:23.362480 22591 raft_consensus.cc:1275] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:23.343912 22593 consensus_queue.cc:260] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.335578 29431 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:23.327889 29147 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.365918 29420 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:23.366037 29431 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:23.366704 29420 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:23.367187 29423 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.276597 29240 consensus_queue.cc:237] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:23.369277 29352 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:23.369923 29352 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.375795 21899 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.346256 29024 consensus_queue.cc:1243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 8f3b35a35257416db3eaaf364cf6b52b log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:23.344389 29364 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:23.395133 22593 raft_consensus.cc:3060] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:23.378407 29374 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:23.349449 29359 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:23.428344 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 63531us
I20260709 13:08:23.380725 29054 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:23.397379 29420 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.351415 29049 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
W20260709 13:08:23.430244 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 88873.8us
I20260709 13:08:23.438519 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:23.448431 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:23.482388 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.485880 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:23.557637 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 71870.9us
I20260709 13:08:23.558652 21823 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:23.451529 29364 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:23.583890 22005 pending_rounds.cc:85] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 9
I20260709 13:08:23.589927 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:23.617192 22005 pending_rounds.cc:107] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:08:23.617640 22005 raft_consensus.cc:2889] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:23.414131 29297 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 13:08:23.476054 29147 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:23.646407 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 163137us
I20260709 13:08:23.646826 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:23.646324 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:23.646908 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:23.647737 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:23.647797 22007 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:23.653378 22006 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:23.670681 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:23.652666 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:23.672902 22001 raft_consensus.cc:3060] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:23.681550 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:23.683718 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:23.560009 29408 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.685793 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:23.686208 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 127655us
I20260709 13:08:23.683557 22591 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 16.
I20260709 13:08:23.690014 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:23.693747 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:23.648710 29204 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.694628 22585 raft_consensus.cc:2410] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:23.699115 21824 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:23.699314 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.689035 29054 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.681702 29364 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:23.651539 29437 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:23.672575 29415 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.700340 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:23.685155 29013 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:08:23.693943 29147 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:23.710079 29341 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:23.711768 29341 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.712191 29341 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:23.713091 29423 consensus_queue.cc:1048] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:23.710331 29437 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.696496 29182 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:23.703433 29017 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:23.681326 29077 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:08:23.709873 29411 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:23.707526 29364 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:23.728173 29364 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:23.710320 29204 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:23.708451 29054 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.703811 29305 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:08:23.721966 29017 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.703653 29366 consensus_queue.cc:1048] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:08:23.727221 29352 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:23.757839 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 47945.3us
I20260709 13:08:23.773738 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1316) took 1471 ms. Trace:
I20260709 13:08:23.737624 29204 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:23.798704 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 80005.6us
I20260709 13:08:23.786998 29017 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:23.724565 29438 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
W20260709 13:08:23.797991 29444 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:08:23.789495 21928 rpcz_store.cc:276] 0709 13:08:22.302369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.322989 (+ 20620us) service_pool.cc:224] Handling call
0709 13:08:22.344267 (+ 21278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.628326 (+284059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.773400 (+1145074us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540017e654. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:23.773659 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573862272}
I20260709 13:08:23.789382 29366 consensus_queue.cc:1243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:08:23.810582 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 107591us
I20260709 13:08:23.798856 29359 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:23.800076 29204 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:23.817665 29173 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.738046 29411 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:23.829072 22593 raft_consensus.cc:1275] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:08:23.830585 22593 pending_rounds.cc:85] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 13
I20260709 13:08:23.830902 22593 pending_rounds.cc:107] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:08:23.836490 22593 raft_consensus.cc:2889] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:23.831810 29410 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:23.850667 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.845479 29013 consensus_queue.cc:1059] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:08:23.836141 29437 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:23.822572 29440 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:08:23.836645 29077 consensus_queue.cc:1048] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:08:23.837980 29147 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:23.837716 29359 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:08:23.886400 22001 raft_consensus.cc:2468] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
W20260709 13:08:23.887128 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.882014 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.882644 29454 raft_consensus.cc:3055] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:08:23.949836 29435 log.cc:927] Time spent T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:08:23.951659 29341 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:23.954624 29441 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:08:23.945791 29453 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:08:23.961097 29443 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
W20260709 13:08:23.962409 29449 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:08:23.963707 29173 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:23.966230 29448 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:23.856817 29305 consensus_queue.cc:1059] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.073s
W20260709 13:08:23.962564 29455 log.cc:927] Time spent T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:08:24.000281 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.000778 29297 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:24.002255 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 120332us
I20260709 13:08:24.003578 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.003417 29431 consensus_queue.cc:1059] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:23.928332 29456 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:23.947022 29450 log.cc:927] Time spent T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:08:23.856880 29410 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.025838 22585 raft_consensus.cc:1217] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Deduplicated request from leader. Original: 16.13->[18.14-22.16]   Dedup: 18.14->[20.15-22.16]
I20260709 13:08:24.026546 22585 pending_rounds.cc:85] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 14
I20260709 13:08:24.026827 22585 pending_rounds.cc:107] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:08:24.027091 22585 raft_consensus.cc:2889] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:23.930359 29454 raft_consensus.cc:740] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:23.971174 28807 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 1.016s	user 0.008s	sys 0.005s
I20260709 13:08:23.866276 29024 consensus_queue.cc:1048] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
I20260709 13:08:23.947506 29371 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.004792 29431 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:24.101357 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 287525us
W20260709 13:08:24.103070 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 298880us
W20260709 13:08:24.107757 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.062695 29366 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.024308 29173 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.097708 29077 raft_consensus.cc:3055] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:24.092911 29240 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.114765 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:23.996608 29017 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.088980 29437 consensus_queue.cc:1059] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:08:24.119843 29077 raft_consensus.cc:740] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:24.137262 22592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1480) took 1225 ms. Trace:
I20260709 13:08:24.137835 22592 rpcz_store.cc:276] 0709 13:08:22.904591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.904747 (+   156us) service_pool.cc:224] Handling call
0709 13:08:22.905260 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.009606 (+104346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.129827 (+1120221us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b5400651b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:24.130080 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64798080}
I20260709 13:08:24.005528 29456 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.097980 28807 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.153137 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.118377 29297 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:08:24.132737 29431 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:24.118835 29366 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.120957 29173 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.112535 29371 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.113255 29147 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.162096 29437 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:24.157472 29411 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.130394 29240 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.196002 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:24.179554 29451 raft_consensus.cc:3055] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:24.197265 22592 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:24.183738 29341 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.184337 29299 raft_consensus.cc:3055] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:24.184355 29383 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:24.189693 29447 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:24.210258 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.204172 29299 raft_consensus.cc:740] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:24.199438 29451 raft_consensus.cc:740] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:24.180158 29077 consensus_queue.cc:260] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:24.238612 29436 log.cc:927] Time spent T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:08:24.238817 29410 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:24.239220 29442 log.cc:927] Time spent T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 13:08:24.238096 29451 consensus_queue.cc:260] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.241264 29451 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:24.246685 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.189554 29445 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.137s
I20260709 13:08:24.168846 29454 consensus_queue.cc:260] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.233439 29458 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:24.244108 29364 consensus_meta.cc:287] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.000s
I20260709 13:08:24.235422 29447 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.258589 29371 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.258765 29447 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.260391 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.260627 29173 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:24.261286 22586 raft_consensus.cc:2393] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 21: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:24.261348 29364 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.263831 29456 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.256953 29299 consensus_queue.cc:260] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:24.266290 21856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 1170) took 1451 ms (client timeout 1786 ms). Trace:
W20260709 13:08:24.266716 21856 rpcz_store.cc:269] 0709 13:08:22.815150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.815275 (+   125us) service_pool.cc:224] Handling call
0709 13:08:24.266113 (+1450838us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000104d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:24.266232 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408154496}
W20260709 13:08:24.272734 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.273263 29467 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:24.273345 29458 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.274394 29467 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.276052 29458 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.279964 28807 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.284137 29410 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:24.305918 29410 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.308373 29204 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.252425 29445 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:24.310703 22592 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:08:24.309782 22556 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.679s (SENT) [suppressed 14 similar messages]
I20260709 13:08:24.300683 28807 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.327586 21900 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:24.313468 29299 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:24.309767 29431 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.328673 29135 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.232851 29383 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.347842 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:24.315946 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 103117us
I20260709 13:08:24.313973 29445 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:24.324865 29454 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:24.359398 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.345775 29341 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:24.370107 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:24.372107 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.377172 22592 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:24.346170 29024 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.348445 29371 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.375985 21853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1169) took 1562 ms. Trace:
I20260709 13:08:24.381955 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1485) took 1475 ms. Trace:
I20260709 13:08:24.384459 22584 rpcz_store.cc:276] 0709 13:08:22.906286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.985551 (+ 79265us) service_pool.cc:224] Handling call
0709 13:08:23.013909 (+ 28358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.044923 (+ 31014us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:08:23.045669 (+   746us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.045876 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.045923 (+    47us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:23.045977 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.225733 (+1179756us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b18003be600. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e2522 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:24.380568 (+154835us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b18003be600. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6b02a6 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419
0709 13:08:24.381732 (+  1164us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b54007e0254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:24.381778 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.381902 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":2445,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5351,"spinlock_wait_cycles":91442176}
I20260709 13:08:24.334928 28807 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:24.387704 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 77945.4us
W20260709 13:08:24.391592 22556 leader_election.cc:336] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Timed out: RequestConsensusVote RPC to 127.20.66.129:38927 timed out after 1.786s (SENT)
I20260709 13:08:24.375528 29366 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.383152 21853 rpcz_store.cc:276] 0709 13:08:22.813808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.829887 (+ 16079us) service_pool.cc:224] Handling call
0709 13:08:22.852744 (+ 22857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.884167 (+ 31423us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:08:22.924566 (+ 40399us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.241919 (+1317353us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x7b18002eb9a0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f23a 00007f3b707d9a2a
0709 13:08:24.375489 (+133570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.375570 (+    81us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:24.375647 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.375744 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.375909 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18877,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":9114,"raft.queue_time_us":118,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"spinlock_wait_cycles":19006720,"thread_start_us":133216,"threads_started":1}
I20260709 13:08:24.381748 29383 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.352669 29431 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:24.415902 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 83145.6us
I20260709 13:08:24.418774 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.419154 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.408362 29364 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.419534 22591 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:24.390269 29024 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.387475 29371 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.419812 21928 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
W20260709 13:08:24.418589 21972 connection.cc:256] RPC call timeout handler was delayed by -1.6433062660000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 13:08:24.421763 22586 raft_consensus.cc:3060] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:08:24.422785 29374 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.996s	user 0.014s	sys 0.000s
I20260709 13:08:24.423280 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.452746 21934 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:24.423396 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.424710 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.454372 21927 raft_consensus.cc:3060] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:24.424533 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.426337 21932 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:08:24.426873 29240 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.432384 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:24.419826 22556 connection.cc:256] RPC call timeout handler was delayed by -1.0241456659999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
I20260709 13:08:24.435799 29366 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.411656 29410 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.437202 29204 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.437588 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.412472 29445 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.442356 29424 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:24.464283 21929 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: [15.13-17.14]
I20260709 13:08:24.481705 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:24.480492 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:24.483841 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.470082 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.488761 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.493686 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.494979 21934 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:24.468137 29130 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.497927 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.459656 29374 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:24.469627 28807 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.483852 29240 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:24.502578 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.488242 29452 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.468137 29411 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.505398 29411 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.503077 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.509821 21932 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:24.510169 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.499465 29445 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:24.512162 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.512730 21929 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:24.513422 29431 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.467414 29366 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.519600 29481 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.520175 29481 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.521163 29366 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.522241 29473 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:24.523018 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.522827 29473 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:24.521299 29400 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.628s	user 0.003s	sys 0.018s
I20260709 13:08:24.510533 29411 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:24.525410 29401 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 1.516s	user 0.002s	sys 0.015s
I20260709 13:08:24.525362 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3185) took 2238 ms. Trace:
I20260709 13:08:24.526096 22009 rpcz_store.cc:276] 0709 13:08:22.287105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.429844 (+142739us) service_pool.cc:224] Handling call
0709 13:08:22.430229 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.445176 (+ 14947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.445207 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.461294 (+ 16087us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.675904 (+214610us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:22.675959 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.676047 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.520726 (+1844679us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:24.525070 (+  4344us) raft_consensus.cc:1685] finished
0709 13:08:24.525097 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.525291 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12456,"raft.queue_time_us":12566,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":45593,"spinlock_wait_cycles":3913600,"thread_start_us":214096,"threads_started":1,"wal-append.queue_time_us":334996}
I20260709 13:08:24.528303 22587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1478) took 2665 ms. Trace:
I20260709 13:08:24.494532 29424 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.528777 22587 rpcz_store.cc:276] 0709 13:08:21.862636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.075793 (+213157us) service_pool.cc:224] Handling call
0709 13:08:22.076138 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.469187 (+393049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.469215 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.469720 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.742875 (+273155us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:08:22.892233 (+149358us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.901455 (+  9222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.901592 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.524807 (+1623215us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:24.527954 (+  3147us) raft_consensus.cc:1685] finished
0709 13:08:24.528062 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.528235 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19022,"prepare.queue_time_us":252813,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":66727,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":13697,"spinlock_wait_cycles":11529984,"thread_start_us":772945,"threads_started":3,"wal-append.queue_time_us":453422}
I20260709 13:08:24.536875 29371 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.537174 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.537778 29473 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.537894 21932 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:24.534722 21934 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:24.540251 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.502207 29130 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.544945 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.545785 21932 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:24.545533 21933 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
W20260709 13:08:24.547899 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86026.2us
I20260709 13:08:24.548418 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.513657 29452 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.549988 21932 raft_consensus.cc:1240] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:08:24.551198 29452 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.511108 29240 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.555964 29240 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.558537 21929 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:24.561414 21927 raft_consensus.cc:2468] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:24.538021 29424 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:24.566138 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 100461us
I20260709 13:08:24.569622 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.570649 22585 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 19.
I20260709 13:08:24.572364 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:24.573884 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.573959 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:24.578440 21932 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:08:24.580902 21972 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:24.568631 29344 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.578576 29431 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:24.584905 29431 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.585281 29431 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:24.585345 29363 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.589206 29447 raft_consensus.cc:2804] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:24.589829 29447 raft_consensus.cc:697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:24.590943 29130 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.590785 29447 consensus_queue.cc:237] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.583559 29238 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:08:24.593173 29049 consensus_queue.cc:1059] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:24.593705 29238 consensus_queue.cc:1243] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:08:24.594161 29049 raft_consensus.cc:3055] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:24.594408 29024 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.594551 29049 raft_consensus.cc:740] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:24.594954 29431 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.595613 29049 consensus_queue.cc:260] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.596680 29049 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:24.576423 29424 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:08:24.590986 29422 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.400s	user 0.004s	sys 0.012s
I20260709 13:08:24.610562 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:24.618885 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45073.8us
I20260709 13:08:24.578152 29410 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.028s
W20260709 13:08:24.620215 21972 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.620653 22556 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:24.605274 29431 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:24.605583 29363 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:24.624471 21972 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.603847 29424 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:24.623869 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.625205 29173 raft_consensus.cc:2804] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:24.631356 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.632841 22556 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.601773 29344 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.638197 29344 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.639091 21932 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:24.644639 29490 raft_consensus.cc:2804] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:24.645215 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.645682 29490 raft_consensus.cc:697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:24.646864 29490 consensus_queue.cc:237] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.631227 29173 raft_consensus.cc:697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 25 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:24.651140 29173 consensus_queue.cc:237] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.658432 29160 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:24.659075 29160 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:24.627440 29424 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:24.621889 29297 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:08:24.625388 29488 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:24.667112 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.671696 21935 raft_consensus.cc:1240] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:24.672740 21926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2751) took 2155 ms (client timeout 1765 ms). Trace:
W20260709 13:08:24.673197 21926 rpcz_store.cc:269] 0709 13:08:22.515971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.516127 (+   156us) service_pool.cc:224] Handling call
0709 13:08:24.671237 (+2155110us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540017eb54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:24.671369 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346220288}
I20260709 13:08:24.674638 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.677217 29298 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:24.679046 21926 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:24.679804 29433 raft_consensus.cc:3055] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:24.680197 29433 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:24.680289 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:24.690827 21972 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:24.692261 21972 leader_election.cc:336] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.692835 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.699404 22586 raft_consensus.cc:1275] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:24.699187 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.700253 21928 raft_consensus.cc:3060] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:24.685417 29297 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:08:24.702076 21972 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.698892 29344 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:08:24.703694 21972 leader_election.cc:336] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.704226 29344 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:24.705190 21972 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.706071 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.703266 29097 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.703182 29437 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:08:24.706868 29298 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.715251 29298 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.709760 29371 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.717348 29371 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.671119 29488 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.719890 29371 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.720573 29488 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.703158 29374 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.721455 29374 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.727154 29160 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.728384 21928 raft_consensus.cc:2468] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:24.736936 21931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3306) took 2482 ms. Trace:
I20260709 13:08:24.739841 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.711915 29097 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.744695 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.737433 21931 rpcz_store.cc:276] 0709 13:08:22.247010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.247113 (+   103us) service_pool.cc:224] Handling call
0709 13:08:22.247450 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.300483 (+ 53033us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:22.398031 (+ 97548us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.715044 (+317013us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.523976 (+1808932us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b1800062320. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:24.528368 (+  4392us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b1800062320. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b703774bc 00007f3b6d6c548b 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b707c7060 00007f3b707d9716 00007f3b707ea287
0709 13:08:24.528492 (+   124us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:24.529089 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:08:24.663890 (+134801us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:24.663964 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.664153 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.729551 (+ 65398us) raft_consensus.cc:1685] finished
0709 13:08:24.729572 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.729710 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28856,"prepare.queue_time_us":41632,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":444064,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":19168,"spinlock_wait_cycles":19851904,"thread_start_us":526862,"threads_started":3,"wal-append.queue_time_us":440991,"wal-append.run_cpu_time_us":4449,"wal-append.run_wall_time_us":1483814}
I20260709 13:08:24.743650 29341 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:24.746836 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.709352 29344 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.747818 21934 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:24.751832 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.745484 29240 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.755074 29160 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.756533 29097 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.757683 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.710952 29433 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.740106 29431 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:24.734481 29490 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:24.733472 29298 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.762373 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.772290 21931 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:24.771986 29490 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.773401 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.774075 21931 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:08:24.774447 29490 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.774938 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.776091 21931 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:24.776248 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.763069 29240 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.776886 21934 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:08:24.778620 29240 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.764504 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.779338 21929 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:24.780329 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.781054 29085 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.781575 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.782711 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.770279 29374 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.784512 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:24.789533 21929 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:24.786903 29017 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:24.791139 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.791769 29160 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.792636 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:24.792500 29160 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.793478 21929 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 25: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:24.794296 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.794406 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.794821 21929 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:24.795251 29160 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:24.795431 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.795571 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.796162 21933 raft_consensus.cc:2393] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 17: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:24.776932 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:24.797475 21931 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 25: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:24.797526 29416 raft_consensus.cc:3055] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:24.797994 29416 raft_consensus.cc:740] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:24.798741 21931 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:24.798837 29416 consensus_queue.cc:260] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.799954 29416 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:24.802037 29410 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:24.796787 21929 raft_consensus.cc:1240] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 20. Current term is 22. Ops: []
I20260709 13:08:24.784704 21857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3147) took 2324 ms. Trace:
I20260709 13:08:24.769416 29341 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:24.803546 21857 rpcz_store.cc:276] 0709 13:08:22.460379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.550641 (+ 90262us) service_pool.cc:224] Handling call
0709 13:08:22.550967 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.279236 (+728269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.784408 (+1505172us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b540028b754. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:24.784595 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673070976}
W20260709 13:08:24.804792 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.812760 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.812901 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:24.814626 22001 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:08:24.814653 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:24.816092 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.813882 22005 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:24.818382 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.818955 22005 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:08:24.802641 29410 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:24.830305 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.831753 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:24.832975 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.833389 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.833984 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:24.822464 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 70726.4us
I20260709 13:08:24.842727 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:24.831851 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:24.844401 22586 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:24.848855 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:24.850054 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 144879us
W20260709 13:08:24.850967 21972 leader_election.cc:336] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:24.853649 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.854857 21934 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:24.855110 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:24.897517 29496 log.cc:927] Time spent T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:08:24.897317 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.898607 22009 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:08:24.898635 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 56153.1us
I20260709 13:08:24.902221 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:24.903103 22591 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:08:24.854755 22005 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
W20260709 13:08:24.989347 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 87281.5us
I20260709 13:08:24.862353 21933 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:08:24.989053 22001 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 17.
I20260709 13:08:24.902575 29344 raft_consensus.cc:2764] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:08:24.993294 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:24.851238 28820 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:24.988966 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:24.990103 29085 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:24.995383 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.902365 28919 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:24.998510 22009 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:08:24.996153 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 91259.5us
W20260709 13:08:25.001068 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:24.991520 29498 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:25.006827 21972 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.000900 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:24.998497 29160 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.013898 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:25.000200 29489 log.cc:927] Time spent T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:08:25.002923 29495 log.cc:927] Time spent T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
W20260709 13:08:25.029446 21972 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:25.030232 21972 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.033417 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:25.037268 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:24.998142 28820 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.016983 29490 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:25.052124 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.053066 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.053673 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.031788 29333 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:25.039563 29431 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:25.049513 29389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.033s	user 0.020s	sys 0.026s
W20260709 13:08:25.069192 21972 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.027963 29424 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.017467 29498 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.031589 29160 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:25.090744 21972 leader_election.cc:336] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.098634 22001 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
W20260709 13:08:25.103569 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.117103 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:25.100456 29389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.141s	user 0.020s	sys 0.027s
W20260709 13:08:25.127619 21972 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.129546 29160 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.135569 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:25.136618 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.136792 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:25.137292 22591 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:25.138852 29498 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.138993 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3310) took 2374 ms. Trace:
I20260709 13:08:25.139333 21935 rpcz_store.cc:276] 0709 13:08:22.764411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.775024 (+ 10613us) service_pool.cc:224] Handling call
0709 13:08:24.671424 (+1896400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.673516 (+  2092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.138803 (+465287us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540017eb50. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:25.138938 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4313715072}
I20260709 13:08:25.140475 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.137352 21926 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:25.135764 29423 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:25.143839 29423 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:25.144733 29423 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.145748 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.146355 22001 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 17.
I20260709 13:08:25.146431 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.147996 22000 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 25. Current term is 26. Ops: []
I20260709 13:08:25.148804 29423 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:25.149411 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.149993 22005 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:25.151479 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.152323 22006 raft_consensus.cc:2393] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 25: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:08:25.155189 29405 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 1.998s	user 0.007s	sys 0.010s
I20260709 13:08:25.167523 29467 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.168475 29424 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:25.174592 28820 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:25.174994 29403 log.cc:927] Time spent T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.991s	user 0.000s	sys 0.007s
I20260709 13:08:25.194737 22004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41342 (request call id 3184) took 2947 ms. Trace:
I20260709 13:08:25.195217 22004 rpcz_store.cc:276] 0709 13:08:22.247475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.393477 (+146002us) service_pool.cc:224] Handling call
0709 13:08:22.393748 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.394069 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.394110 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.436174 (+ 42064us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.745569 (+309395us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:22.745630 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.745724 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:25.174441 (+2428717us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:25.194231 (+ 19790us) raft_consensus.cc:1685] finished
0709 13:08:25.194262 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:25.194638 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4807,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7306,"spinlock_wait_cycles":32873216,"thread_start_us":236931,"threads_started":1,"wal-append.queue_time_us":564833}
I20260709 13:08:25.060665 29490 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.186852 29481 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:25.190493 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.217792 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.179636 29410 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:25.197595 28820 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:25.199646 21927 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 17. Current term is 18. Ops: [15.13-17.14]
I20260709 13:08:25.202708 22008 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:25.203085 29240 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.203639 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.220865 22009 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:25.205391 21859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41266 (request call id 3150) took 2718 ms. Trace:
I20260709 13:08:25.221376 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:25.221735 21859 rpcz_store.cc:276] 0709 13:08:22.486940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.487304 (+   364us) service_pool.cc:224] Handling call
0709 13:08:22.487748 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.510449 (+ 22701us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:22.554661 (+ 44212us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.679242 (+124581us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.719701 (+ 40459us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.762849 (+ 43148us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.763223 (+   374us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:22.763274 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.763359 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.582011 (+1818652us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b1800436c60. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:25.154590 (+572579us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:25.205104 (+ 50514us) raft_consensus.cc:1685] finished
0709 13:08:25.205135 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:25.205323 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51177,"prepare.queue_time_us":52084,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":55435,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":42061,"spinlock_wait_cycles":70598784,"thread_start_us":97195,"threads_started":1,"wal-append.queue_time_us":482603}
I20260709 13:08:25.207581 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.211140 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.218654 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.225337 22006 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:25.226959 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.227631 22008 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:25.228778 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.227468 22591 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:25.232663 21927 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:25.234282 29490 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:25.236971 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 246630us
I20260709 13:08:25.249176 22586 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:25.220485 28820 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.233654 29097 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:25.278080 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:25.268761 22006 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:25.279225 22586 raft_consensus.cc:2368] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21.
I20260709 13:08:25.236184 29240 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.316140 22005 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:08:25.322614 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:25.323263 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 178723us
W20260709 13:08:25.332458 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:25.331067 29467 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:25.345031 21931 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:08:25.346644 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:25.347594 21972 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.352893 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:25.353660 21931 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:08:25.355206 21972 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.358026 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:25.352087 29416 consensus_meta.cc:287] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.548s	user 0.016s	sys 0.000s
I20260709 13:08:25.348095 29410 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:25.363366 22007 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:25.358111 28820 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.348414 29467 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:25.371526 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.371755 21899 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:25.377424 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:25.364812 29410 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.373085 28820 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.382283 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:25.380789 21972 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:25.394982 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.375809 29374 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:25.397789 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.386847 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:25.398739 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.392959 22005 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 24. Current term is 25. Ops: []
I20260709 13:08:25.403067 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.383159 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.404019 22008 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:25.403096 29240 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:25.406136 29446 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 1.674s	user 0.006s	sys 0.001s
I20260709 13:08:25.406834 22588 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:25.406029 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.409997 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.383096 29389 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:25.381670 29341 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:25.414876 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.415472 22586 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:25.415966 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.416777 22585 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:25.400508 29374 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.419135 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.408638 29466 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:25.428028 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.421851 21974 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:25.429098 22005 raft_consensus.cc:2393] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 19: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:25.435869 22007 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:25.412606 29389 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:25.411134 29359 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:08:25.415076 29341 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.440301 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:25.426033 29374 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:25.443214 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.425738 29466 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.410084 28820 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.478006 21930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33888 (request call id 2756) took 1055 ms. Trace:
I20260709 13:08:25.478298 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.483651 21930 rpcz_store.cc:276] 0709 13:08:24.422295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.423114 (+   819us) service_pool.cc:224] Handling call
0709 13:08:25.477767 (+1054653us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b5400229a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:25.477933 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2117821312}
I20260709 13:08:25.483978 22001 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:25.446353 21974 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:25.492311 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.503571 29467 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.510640 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:25.476126 29511 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:08:25.441443 29341 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:25.459758 29416 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:25.440485 29431 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:25.451383 29382 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:25.520565 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.527981 22001 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
W20260709 13:08:25.512666 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 66353.4us
W20260709 13:08:25.529053 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 125469us
I20260709 13:08:25.529467 21974 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:25.475790 29513 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:08:25.484706 29466 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:25.531426 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:25.516093 29499 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:25.519716 29416 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:25.536259 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.537093 22001 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 17: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:25.450790 29389 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.538699 29389 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:25.539387 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.541198 29418 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:08:25.541469 22589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1481) took 2636 ms. Trace:
I20260709 13:08:25.541880 22589 rpcz_store.cc:276] 0709 13:08:22.904913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.905056 (+   143us) service_pool.cc:224] Handling call
0709 13:08:22.940472 (+ 35416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:23.361751 (+421279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.541224 (+2179473us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54004cca54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:25.541415 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967088384}
I20260709 13:08:25.542322 21930 raft_consensus.cc:1275] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:25.533468 29284 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:25.543380 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:25.543442 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.544261 22005 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:25.520263 29305 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 13:08:25.548262 29054 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.550727 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.551996 29077 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:25.553256 29523 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:25.553790 29410 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.551968 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.535202 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:25.536444 21974 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:25.556958 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.557837 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.560230 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:25.536065 29416 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:25.561349 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.561041 21824 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:25.555653 29431 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:25.565372 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:25.564549 22001 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 25: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:25.569157 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.569994 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:25.567569 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.571772 29364 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:25.572311 29364 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:25.563841 29523 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:25.573225 29364 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.570860 22001 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
I20260709 13:08:25.559810 29410 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:25.581288 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:25.581496 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 72143.6us
I20260709 13:08:25.581947 22001 raft_consensus.cc:2468] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:25.580046 22557 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:25.561339 29305 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:25.582755 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.583643 22001 raft_consensus.cc:2393] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 19: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:25.584102 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:25.584465 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.586160 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:25.587796 22557 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:25.566586 29431 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.588461 29382 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:25.585809 29305 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:25.600344 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.587635 29490 raft_consensus.cc:2804] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:25.583132 29410 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:25.590368 29298 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:25.592654 29382 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.602816 29382 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:25.603264 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.608625 29410 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:25.588990 29411 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:25.613471 29382 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.614780 29359 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:08:25.623946 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.590992 29431 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:25.617539 29490 raft_consensus.cc:697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:25.626605 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:25.600607 29519 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:08:25.601040 29305 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:25.632160 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.654217 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:25.655930 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:25.657846 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:25.663826 22000 raft_consensus.cc:1275] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:08:25.673110 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 90953.7us
I20260709 13:08:25.674081 29374 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.676302 29374 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.676070 29521 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:25.676401 21825 leader_election.cc:400] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21.
I20260709 13:08:25.645773 29490 consensus_queue.cc:237] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:25.720685 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.629624 29298 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:25.770705 22005 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:25.670420 29464 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:25.755995 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.807619 22008 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:08:25.672762 29153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.940s	user 0.009s	sys 0.019s
W20260709 13:08:25.809722 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 407623us
I20260709 13:08:25.670612 29411 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:25.717715 29364 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.722325 28820 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:25.670871 29523 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:25.816077 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 110818us
I20260709 13:08:25.761986 29524 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:25.772202 29382 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:25.738013 29433 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:25.809971 29153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.077s	user 0.010s	sys 0.020s
I20260709 13:08:25.798455 21825 leader_election.cc:403] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 13:08:25.831053 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 448777us
I20260709 13:08:25.836807 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:25.763196 29525 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.802585 29298 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:25.838362 29298 raft_consensus.cc:3060] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:25.838532 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.835664 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.835390 29364 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.835840 29533 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:08:25.866245 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 190070us
I20260709 13:08:25.867180 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:25.867869 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:25.866582 22000 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:25.869552 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 211797us
W20260709 13:08:25.838307 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.751353 29360 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:08:25.815330 29411 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:25.838289 29525 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:25.886111 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 316154us
I20260709 13:08:25.872711 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:25.871675 21825 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:25.903162 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 97997us
I20260709 13:08:25.832254 29530 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.903945 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:25.904896 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:25.815109 29521 consensus_queue.cc:1243] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:08:25.869916 29464 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:25.901489 29054 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:25.881520 28807 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:25.914703 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:25.867558 29533 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:25.882141 29153 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:25.917237 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80522.7us
I20260709 13:08:25.905808 29411 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:25.904626 29360 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:25.910128 29530 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.913259 29521 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:25.913760 29375 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.930202 22001 raft_consensus.cc:1240] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:08:25.904688 29341 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.922598 29466 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.916152 29017 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.012390 22001 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:25.865401 29524 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.931438 29523 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.940685 29382 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:25.925618 29182 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:25.973551 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 71286.8us
I20260709 13:08:25.976797 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:26.022236 21900 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:25.991539 29374 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.978088 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:26.039064 21926 raft_consensus.cc:2368] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21.
I20260709 13:08:25.978749 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:25.980381 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:26.040335 21933 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:25.981129 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:25.982903 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:25.984897 29534 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:25.986279 29464 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:25.989289 29518 consensus_queue.cc:1059] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:25.914196 29054 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:25.998831 22000 raft_consensus.cc:2468] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:26.053319 22000 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 25. Current term is 26. Ops: []
W20260709 13:08:25.928710 29539 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:08:25.925992 29481 raft_consensus.cc:2764] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:08:25.930653 28820 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:25.921147 29533 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:25.971061 29521 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:25.913368 29416 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.018324 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.003912 29529 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:08:25.977201 29097 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:26.022045 21935 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
W20260709 13:08:26.003661 29527 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:08:26.041443 21932 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
W20260709 13:08:26.015766 29531 log.cc:927] Time spent T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:08:26.008813 29341 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.017141 29017 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:25.931273 29410 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.051294 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.093693 22001 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:26.054847 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.054930 22557 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:26.074476 22000 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:08:26.095589 22000 pending_rounds.cc:85] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 10
I20260709 13:08:26.095944 22000 pending_rounds.cc:107] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:08:26.019354 29535 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:26.005493 29305 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.046525 29382 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:26.003082 29375 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.011061 29466 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.068647 29054 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:26.096184 22000 raft_consensus.cc:2889] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:26.075105 28820 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:26.044648 29182 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.085724 29536 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:08:26.048527 29374 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.047683 29532 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:08:26.087188 29518 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:26.111716 21933 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:26.087074 29431 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.101159 29481 consensus_queue.cc:1048] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
W20260709 13:08:26.142470 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 237602us
I20260709 13:08:26.147409 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:26.127409 28820 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:26.085249 29360 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.139204 29341 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.098641 29535 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.163549 22009 raft_consensus.cc:1240] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 24. Ops: []
W20260709 13:08:26.164366 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 249725us
I20260709 13:08:26.141568 29525 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:26.165448 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 293870us
I20260709 13:08:26.165477 22556 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.175760 21825 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:26.176430 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:26.177125 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 153312us
I20260709 13:08:26.182090 21933 raft_consensus.cc:1275] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:26.192480 22001 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 18.
I20260709 13:08:26.192382 21823 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:26.201638 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:26.207118 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.207598 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.208029 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.208225 22591 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:26.177501 29360 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:08:26.257664 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.212826 29305 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:26.259658 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.213564 29017 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.184113 29466 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:26.214656 29410 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.217660 29182 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.276525 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.276633 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.277743 22003 raft_consensus.cc:2410] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:08:26.277838 22006 raft_consensus.cc:2393] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:26.282284 21972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.218890 29374 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.219393 29538 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:26.219774 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 3202) took 1395 ms. Trace:
I20260709 13:08:26.288411 22007 rpcz_store.cc:276] 0709 13:08:24.813929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.814068 (+   139us) service_pool.cc:224] Handling call
0709 13:08:26.209575 (+1395507us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400080a54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:26.209690 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215610240}
I20260709 13:08:26.220242 29521 consensus_queue.cc:1059] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:26.220600 29375 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.220721 29545 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:26.211098 29524 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.168040 29341 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.221657 29416 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.226445 29528 raft_consensus.cc:3055] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:26.237368 29359 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:26.238404 29535 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.301493 29545 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:26.238557 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:26.239418 21974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.309269 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.309829 22003 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
W20260709 13:08:26.242163 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.203960 29530 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.251362 29218 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.253479 29532 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:26.183913 29382 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.312441 29077 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.312530 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:26.313438 21930 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:26.313197 29077 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.164112 29543 raft_consensus.cc:2804] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:26.315454 29077 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.315694 29543 raft_consensus.cc:697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:26.316962 29543 consensus_queue.cc:237] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.318046 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.212924 29547 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:26.305848 29521 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:26.321419 29521 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:26.322317 29521 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.270921 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.289345 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.324633 22008 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:26.325263 29521 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:26.312562 29218 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.290769 21972 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:26.291590 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.329756 22593 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:26.293464 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.296675 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:26.332036 22589 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:26.273663 29466 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.269752 29525 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:26.274904 29389 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.307688 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.308025 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:26.339820 22584 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:08:26.340222 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 101767us
I20260709 13:08:26.308950 29545 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:26.345320 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.346339 22009 raft_consensus.cc:2410] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:08:26.292228 29538 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.294198 28807 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:26.334545 22591 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:26.336508 22004 raft_consensus.cc:2393] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:26.294924 29410 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.382515 29410 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:26.337577 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46911.4us
I20260709 13:08:26.387362 29305 consensus_queue.cc:1059] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:26.349483 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:26.307461 29528 raft_consensus.cc:740] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:26.388543 29410 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.388900 29528 consensus_queue.cc:260] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.384802 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:26.390934 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:26.347532 29534 raft_consensus.cc:2804] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:26.391471 29534 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:26.391918 21972 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.356143 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:26.355543 29359 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:26.356762 29153 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) with higher term. Message: Invalid argument: T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21.
I20260709 13:08:26.356192 29364 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.344878 29416 raft_consensus.cc:2804] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:26.333238 29532 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:26.358160 29218 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.358628 29411 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.359341 28820 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.362524 29533 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.363629 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:26.408449 21928 raft_consensus.cc:2376] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23. Re-sending same reply.
I20260709 13:08:26.374501 29547 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:26.377635 22009 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:26.378938 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.411360 22008 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:26.411909 22006 raft_consensus.cc:2376] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22. Re-sending same reply.
I20260709 13:08:26.342130 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.390405 29528 raft_consensus.cc:3060] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:26.392112 29534 raft_consensus.cc:3060] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:26.393923 29538 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:26.344722 29017 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:26.404767 29374 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:08:26.396304 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:26.405745 28919 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:26.402372 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 236960us
W20260709 13:08:26.404841 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 55476.1us
I20260709 13:08:26.344072 29525 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.416167 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:26.423292 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 233703us
W20260709 13:08:26.456564 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.458786 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:26.459354 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:26.460135 22584 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:26.394666 29097 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:26.431167 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 89124.4us
I20260709 13:08:26.292470 29298 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.404989 29532 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:26.404449 28807 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:26.405285 29410 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:08:26.487932 29410 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:08:26.489082 29411 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.489642 29467 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.490130 29467 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:26.447820 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51576.7us
I20260709 13:08:26.492136 29467 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:26.492261 21898 consensus_peers.cc:597] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:26.492882 21898 leader_election.cc:336] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:26.493340 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:26.337332 29466 raft_consensus.cc:3060] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:26.347150 29389 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.451354 21899 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:26.410301 22589 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:26.501451 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.502314 29533 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.502755 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.503897 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.504297 29431 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.451987 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.409361 29416 raft_consensus.cc:697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:26.506721 29416 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.465261 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.500476 29408 raft_consensus.cc:3055] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:08:26.509750 29408 raft_consensus.cc:740] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:26.500447 29389 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.510936 21900 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:26.518075 22589 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:26.469539 22585 raft_consensus.cc:1240] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 24. Current term is 25. Ops: []
I20260709 13:08:26.469791 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.471899 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.443468 29218 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.474295 21974 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:26.446000 29359 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:08:26.573329 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 62510.5us
W20260709 13:08:26.573578 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 121685us
W20260709 13:08:26.573804 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 99576.2us
I20260709 13:08:26.575371 29389 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.575819 29097 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:26.575888 29389 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.576596 29097 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:26.577432 29352 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:26.578092 29389 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:26.578141 29352 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:26.578821 29389 raft_consensus.cc:2764] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:08:26.481165 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:26.580310 29352 consensus_queue.cc:1048] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:08:26.582461 29218 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.584488 29218 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.456895 29017 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.474601 29049 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:26.452461 29374 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:26.457723 29433 consensus_meta.cc:287] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.004s
I20260709 13:08:26.486627 29550 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:08:26.486477 28807 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:26.500442 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.568571 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 96778.2us
I20260709 13:08:26.580744 29524 raft_consensus.cc:3060] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:26.581019 29411 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:26.570554 22585 raft_consensus.cc:1240] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 24. Current term is 25. Ops: []
I20260709 13:08:26.570727 22584 raft_consensus.cc:2468] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
W20260709 13:08:26.571406 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78548.6us
W20260709 13:08:26.572537 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 121240us
W20260709 13:08:26.621963 21898 leader_election.cc:336] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:26.482568 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:26.580809 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 111143us
I20260709 13:08:26.624723 22586 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 13:08:26.625905 22002 raft_consensus.cc:2410] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:08:26.589311 22589 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:26.474957 29523 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.404558 29551 consensus_queue.cc:1048] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
W20260709 13:08:26.594005 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.590209 29553 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:26.567741 29467 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.581039 29502 raft_consensus.cc:3055] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:26.429145 29426 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:08:26.622462 21898 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:26.631397 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.603044 29097 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:26.610455 29049 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:26.612468 29374 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:26.639104 21899 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:26.609090 29017 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:26.682984 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 55326.5us
W20260709 13:08:26.690565 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51566.4us
W20260709 13:08:26.692034 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 53370.6us
I20260709 13:08:26.691843 21899 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:26.580514 29466 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.700178 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78250.4us
I20260709 13:08:26.699103 21974 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:26.709092 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:26.709113 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:26.701354 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:26.718607 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:26.718803 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 354714us
I20260709 13:08:26.719841 21823 leader_election.cc:400] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21.
I20260709 13:08:26.719715 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.720849 21823 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:26.713588 21899 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:26.722216 21898 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:26.719674 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:26.722182 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:26.626114 29411 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:26.724826 22587 raft_consensus.cc:2393] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:26.730111 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.730973 22004 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:26.631400 29431 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.737123 22002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41342 (request call id 3203) took 1922 ms. Trace:
I20260709 13:08:26.750495 22002 rpcz_store.cc:276] 0709 13:08:24.814294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.814445 (+   151us) service_pool.cc:224] Handling call
0709 13:08:26.736859 (+1922414us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54001e0d54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:26.737053 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4162570112}
I20260709 13:08:26.642482 29382 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:26.697037 29049 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:26.689357 29502 raft_consensus.cc:740] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:26.717914 29372 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:26.706607 29240 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:26.717891 29298 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.726783 29518 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:08:26.791921 21757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 284) took 79972 ms (client timeout 15000 ms). Trace:
W20260709 13:08:26.727535 29054 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.598s	user 0.013s	sys 0.003s
I20260709 13:08:26.721168 28919 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:26.718212 29554 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:26.673103 29555 consensus_queue.cc:1059] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.107s
I20260709 13:08:26.727631 29411 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:26.673370 29467 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.738897 29341 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.827200 29466 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.827354 29359 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:26.829363 29359 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:26.830171 29359 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:26.830560 29359 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:26.831732 29359 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:26.751766 29375 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 13:08:26.798481 29077 raft_consensus.cc:2804] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:26.838841 29077 raft_consensus.cc:697] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:26.782485 29525 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.784855 29372 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:26.839819 29526 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:26.839967 29372 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:26.839746 29077 consensus_queue.cc:237] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.823663 28820 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.840845 29408 consensus_queue.cc:260] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.836691 29553 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:26.842336 29553 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.643636 29523 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.844596 29523 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.775332 29389 consensus_queue.cc:1059] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.070s
I20260709 13:08:26.845584 29389 raft_consensus.cc:3055] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:08:26.845911 29389 raft_consensus.cc:740] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:26.846721 29389 consensus_queue.cc:260] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.848011 29389 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:26.792512 29382 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.854430 29411 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.855970 29382 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.856719 29411 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.818874 29054 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:26.860689 29341 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.861436 29341 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.792826 21757 rpcz_store.cc:269] 0709 13:07:06.810401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.825360 (+ 14959us) service_pool.cc:224] Handling call
0709 13:08:26.778568 (+79953208us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:22.963563 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:23.037247 (+ 73684us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:23.037307 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:08:23.037322 (+    15us) tablet.cc:662] Decoding operations
0709 13:08:23.249147 (+211825us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:23.249240 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:23.249279 (+    39us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:08:23.250449 (+  1170us) tablet.cc:701] Row locks acquired
0709 13:08:23.250503 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:08:23.250687 (+   184us) write_op.cc:270] Start()
0709 13:08:23.250834 (+   147us) write_op.cc:276] Timestamp: P: 1783602503250620 usec, L: 0
0709 13:08:23.250855 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:08:23.251587 (+   732us) log.cc:844] Serialized 4868 byte log entry
0709 13:08:24.441208 (+1189621us) op_driver.cc:464] REPLICATION: finished
0709 13:08:24.527437 (+ 86229us) write_op.cc:301] APPLY: starting
0709 13:08:24.527540 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:24.556851 (+ 29311us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:24.556988 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:24.660327 (+103339us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:24.669304 (+  8977us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:08:24.675454 (+  6150us) write_op.cc:312] APPLY: finished
0709 13:08:24.676050 (+   596us) log.cc:844] Serialized 113 byte log entry
0709 13:08:24.677552 (+  1502us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:24.677756 (+   204us) write_op.cc:454] Released schema lock
0709 13:08:24.677860 (+   104us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:24.682694 (+  4834us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:24.682728 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":494078,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":85898,"apply.run_cpu_time_us":20842,"apply.run_wall_time_us":155217,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54359,"num_ops":12,"prepare.queue_time_us":462094,"prepare.run_cpu_time_us":20966,"prepare.run_wall_time_us":399695,"raft.queue_time_us":200230,"raft.run_cpu_time_us":8193,"raft.run_wall_time_us":54327,"replication_time_us":1190106,"thread_start_us":450481,"threads_started":4,"wal-append.queue_time_us":598307,"wal-append.run_cpu_time_us":32695,"wal-append.run_wall_time_us":1821605}]]}
I20260709 13:08:26.865166 29518 raft_consensus.cc:697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:26.866333 29518 consensus_queue.cc:237] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.871686 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:26.872979 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:26.826371 29017 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.826642 29467 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:26.781045 29431 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.725768 29305 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:08:26.880447 22008 raft_consensus.cc:2468] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 16.
W20260709 13:08:26.885794 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 299425us
I20260709 13:08:26.842300 29408 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:26.900480 29568 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.901311 29568 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.903875 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:26.904143 29568 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:26.904706 22004 raft_consensus.cc:1275] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:08:26.907593 21753 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 13, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 13 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.909890 21753 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 14, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 14 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:26.910286 29502 consensus_queue.cc:260] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.911521 29502 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:26.911697 21753 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 12, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 12 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:26.917655 21753 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 19, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 19 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:26.905639 29097 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:26.927101 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 468337us
W20260709 13:08:26.930143 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:26.921716 21753 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 22, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 22 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:26.941140 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:26.905480 29545 consensus_meta.cc:287] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.004s
I20260709 13:08:26.953246 21753 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 9 to 15, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 15 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:26.930045 29543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.008s	sys 0.007s
I20260709 13:08:26.816854 29538 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:26.968860 21753 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 20, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 20 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.989440 21972 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:26.966984 29543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.011s	sys 0.008s
I20260709 13:08:26.996932 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.008395 22003 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
I20260709 13:08:27.009994 21753 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:26.985131 29568 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:26.996342 29426 consensus_queue.cc:1048] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:08:27.043010 29372 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:27.052490 29521 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.727s	user 0.004s	sys 0.009s
I20260709 13:08:27.054889 22003 raft_consensus.cc:1275] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:27.058753 21753 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 20, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 20 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.006436 29563 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:26.966461 29545 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:27.072355 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 373304us
W20260709 13:08:27.075197 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 433066us
I20260709 13:08:27.095669 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:27.080394 29426 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:27.078263 21753 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 18, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), now has a pending config: VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130), VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 18 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.078043 29389 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:27.095532 29372 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.137195 21753 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 19, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 19 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:27.148000 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 428350us
W20260709 13:08:27.148949 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 426806us
I20260709 13:08:27.149873 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:27.151516 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:27.090785 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.157073 21753 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:27.107642 29563 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:27.160310 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:27.082834 29359 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.162850 21823 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:27.163501 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:27.161095 22587 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:27.145051 29532 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.000s
I20260709 13:08:27.126981 29568 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:27.136858 29240 raft_consensus.cc:697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:27.142091 29550 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:27.172026 21824 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:27.169042 29561 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:27.168303 29077 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.147464 29426 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:27.171605 21753 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 16, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 16 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.184653 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:27.187454 21899 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:27.136590 29389 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:27.197818 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 478556us
W20260709 13:08:27.188066 29572 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:08:27.213296 21898 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.180569 29360 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:27.218101 21753 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 20, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.224982 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:27.227522 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.228965 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.229758 21928 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
I20260709 13:08:27.229959 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.230777 21933 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:27.233667 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:27.235791 21898 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.241291 21753 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 17, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 17 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:27.196856 29562 log.cc:927] Time spent T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:08:27.230909 29581 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:27.258548 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.251828 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.260367 21930 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:27.192078 29582 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:27.253454 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:27.195351 29561 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.212635 28820 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:27.194300 29530 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:27.238585 21898 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:27.295262 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 56725.5us
W20260709 13:08:27.295993 21898 leader_election.cc:336] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.296404 21898 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.194898 29410 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:27.256685 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:27.198086 29545 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.188019 29467 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:27.261390 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:27.264447 22587 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
W20260709 13:08:27.306206 21898 leader_election.cc:336] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.306830 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:27.255129 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 165962us
I20260709 13:08:27.263983 21753 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 16, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:27.196801 29481 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.181512 29532 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.196473 29431 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:27.173485 29563 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.318774 29431 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:27.319347 29431 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.265041 29581 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.321313 29581 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:27.316931 29481 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.199040 29077 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.323544 29481 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.192646 29049 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:27.325506 29574 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:08:27.325907 22589 raft_consensus.cc:2393] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:27.256438 29538 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.198657 29568 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.305509 21930 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: [20.12-21.13]
I20260709 13:08:27.207405 29550 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.264701 29583 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:27.330839 22588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1630) took 1022 ms. Trace:
W20260709 13:08:27.332075 29534 consensus_meta.cc:287] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.005s
I20260709 13:08:27.273758 28820 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.333214 28820 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:27.274421 29561 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:27.317374 21898 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.331207 22588 rpcz_store.cc:276] 0709 13:08:26.308837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.309320 (+   483us) service_pool.cc:224] Handling call
0709 13:08:26.309634 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:26.309946 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:26.309974 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:26.310416 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.777132 (+466716us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:26.777173 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.778417 (+  1244us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:27.330645 (+552228us) raft_consensus.cc:1685] finished
0709 13:08:27.330670 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.330788 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15256,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":466282,"threads_started":1,"wal-append.queue_time_us":643160}
I20260709 13:08:27.314265 21753 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:27.340965 21898 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.340858 21753 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 17, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.341670 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.267964 29544 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 13:08:27.334697 22587 raft_consensus.cc:1240] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:27.293812 29582 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:27.199554 29240 consensus_queue.cc:237] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.342633 21753 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:27.346567 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:27.344820 21898 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:27.349792 21898 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.350411 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.317090 29467 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:27.352838 29582 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.353754 29467 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:27.316684 29524 consensus_meta.cc:287] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.004s
I20260709 13:08:27.315534 28919 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:27.362743 21898 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.362506 21753 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 11 to 13, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 13 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.316593 29375 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:27.325541 29532 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.298101 29530 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.328941 29550 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.332522 29534 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.369679 21898 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:27.377168 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 112662us
W20260709 13:08:27.381073 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:27.345955 29554 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:27.383126 21898 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.385205 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:27.367233 29576 log.cc:927] Time spent T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:08:27.385406 21753 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 19, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:27.364359 29551 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:27.365127 29524 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:27.397710 22005 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 24. Current term is 25. Ops: []
I20260709 13:08:27.397810 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.399756 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:27.406001 21898 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.405773 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.406009 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:27.387042 29555 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:27.412237 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.412485 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.413151 22003 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:27.413262 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.413528 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.413906 22000 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:27.414328 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.414661 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.415001 22000 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:27.415123 22003 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:27.380314 29563 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.416801 29550 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:27.417479 29550 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:27.417937 29550 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.420780 21933 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:08:27.405514 29375 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.422374 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.412261 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.425999 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:27.428207 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.434800 21899 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:27.442780 22006 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:27.406702 21898 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.448133 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.441728 21932 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:27.448345 22000 raft_consensus.cc:2410] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:08:27.448874 21899 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:27.448814 22001 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:27.453719 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:27.458472 22592 raft_consensus.cc:2393] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 25: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:27.458416 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:27.450212 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:27.463065 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.465576 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.466979 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.467783 22000 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 25.
W20260709 13:08:27.460212 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 187638us
I20260709 13:08:27.408160 29359 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.316843 29426 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.440927 29550 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.478308 21824 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:27.490046 22006 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
W20260709 13:08:27.479090 29528 consensus_meta.cc:287] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 1.050s	user 0.010s	sys 0.008s
I20260709 13:08:27.521636 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:27.491801 29589 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:08:27.479324 29551 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:27.477466 29153 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.525422 22593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1505) took 1057 ms. Trace:
I20260709 13:08:27.485172 28919 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:27.530133 22001 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:27.516911 29591 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.493152 29538 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:27.457312 29375 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.513765 29389 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:27.532600 22593 rpcz_store.cc:276] 0709 13:08:26.468526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.509784 (+ 41258us) service_pool.cc:224] Handling call
0709 13:08:26.510082 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:26.510284 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:26.510307 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:26.568547 (+ 58240us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.997882 (+429335us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:26.997939 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.998641 (+   702us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:27.525111 (+526470us) raft_consensus.cc:1685] finished
0709 13:08:27.525142 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.525341 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":362106,"threads_started":1,"wal-append.queue_time_us":621249}
I20260709 13:08:27.519891 29583 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:27.541718 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.526862 29359 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:27.469048 29544 consensus_queue.cc:1059] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.056s
W20260709 13:08:27.550552 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 144585us
W20260709 13:08:27.559772 21898 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:27.542759 29374 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.821s	user 0.012s	sys 0.000s
I20260709 13:08:27.546751 21824 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:27.537876 29364 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:27.545687 29538 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.562026 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.577672 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:27.553226 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 94951.2us
I20260709 13:08:27.576408 29525 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:27.587452 29341 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:27.591395 21898 leader_election.cc:336] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.585109 21935 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:27.591898 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.560519 29359 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.596239 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:27.602649 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.597211 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.597890 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:27.598227 21931 raft_consensus.cc:1240] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:27.607172 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.608078 21926 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:27.613082 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.614764 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:27.614938 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.546135 29591 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.587574 29525 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.621290 29525 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:27.625485 29595 log.cc:927] Time spent T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:08:27.594460 29240 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:27.626384 29240 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.625543 29591 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:27.628192 29577 log.cc:927] Time spent T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:08:27.584335 29054 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.630923 29525 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.632474 28807 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.633530 29534 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.633971 29525 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.636236 29581 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:27.642974 29502 consensus_meta.cc:287] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.002s
W20260709 13:08:27.644408 29588 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:08:27.646463 29375 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.602080 29024 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:27.653738 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.643254 29581 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.655815 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.571208 29544 consensus_queue.cc:1243] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:08:27.660928 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.661759 29544 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:08:27.636955 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 1db8966fba10424a8df85ad63defc925: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.662484 29544 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:27.566627 29374 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.664337 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.589027 29553 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.664965 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:27.665933 29364 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:27.615713 29481 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:27.668507 29525 consensus_queue.cc:1059] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:27.669130 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:27.669761 29528 raft_consensus.cc:3055] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:27.670135 29528 raft_consensus.cc:740] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:27.670903 29528 consensus_queue.cc:260] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.671190 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.671793 29528 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:27.671909 21934 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 19.
I20260709 13:08:27.675022 22593 raft_consensus.cc:1275] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:08:27.675519 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:27.676774 29408 consensus_meta.cc:287] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.000s
I20260709 13:08:27.677054 29565 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:27.562654 29585 consensus_queue.cc:1059] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.123s
I20260709 13:08:27.677860 21931 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:27.681385 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 8f2ed3e1870148589d1bba70ce75f160: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.628677 29240 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:27.681794 22556 leader_election.cc:336] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.684242 29549 raft_consensus.cc:3055] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:08:27.615638 29359 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:27.684105 22557 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:27.541153 29153 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.689621 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.690076 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.693758 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.591017 29538 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:27.672955 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:27.724942 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 112298us
I20260709 13:08:27.758337 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:27.759020 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.759088 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:27.759567 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.759986 21933 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:27.683382 29530 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:27.773644 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.774439 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.762027 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:27.755465 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 74466.1us
I20260709 13:08:27.778741 21935 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:27.771083 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:27.779422 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:27.767985 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:27.783285 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 93376.3us
I20260709 13:08:27.790167 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:27.796428 21929 raft_consensus.cc:2393] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 16: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:27.743707 29609 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.800446 29382 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:27.809512 29594 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:27.810163 29594 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.810565 29594 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:27.811102 29049 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:27.805274 29543 consensus_queue.cc:1048] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 13:08:27.787760 29433 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:27.813349 29433 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.813539 29543 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:27.813771 29433 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:27.743100 29549 raft_consensus.cc:740] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:27.757818 29305 raft_consensus.cc:3055] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:27.807446 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:27.821302 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 09efb42cfd6142bb9f816328042aea38: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.771090 29556 raft_consensus.cc:2804] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:08:27.833899 21898 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.834157 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:27.834802 21898 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:27.830375 21929 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 18: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:27.831569 29609 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:27.845209 29590 consensus_queue.cc:1059] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:08:27.840260 21933 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:27.849133 29153 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:27.849275 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48967us
I20260709 13:08:27.787303 29532 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:27.800689 29528 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:27.800926 29389 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:27.829207 29591 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:27.882495 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 80096.6us
I20260709 13:08:27.883401 29049 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:27.884310 29590 raft_consensus.cc:3055] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:27.885367 21824 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:27.888614 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.890188 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:27.805322 29557 consensus_queue.cc:1048] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.135s
I20260709 13:08:27.890555 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.860668 29568 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:27.890765 29077 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:27.940776 29561 consensus_meta.cc:287] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 13:08:27.829401 29305 raft_consensus.cc:740] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:27.945042 29305 consensus_queue.cc:260] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.946398 29305 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:27.892917 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.894213 22001 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:27.897936 29431 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:27.900466 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:27.903462 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:27.904194 29374 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:27.912286 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.957815 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.959344 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:27.961365 29582 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:27.961942 29153 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:27.918650 29556 raft_consensus.cc:697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:27.912714 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:27.967773 22591 raft_consensus.cc:3060] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:08:27.942183 29561 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.860313 29549 consensus_queue.cc:260] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.931964 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:27.989265 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 57447.1us
I20260709 13:08:27.938102 22584 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:27.906491 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:27.985733 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.998555 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:27.990757 22001 raft_consensus.cc:1240] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:27.934231 22000 raft_consensus.cc:2393] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 26: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:27.891891 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:28.002149 22007 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:27.938686 22004 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:27.934036 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 48775.3us
I20260709 13:08:28.006474 21899 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:27.925572 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.934268 29585 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:27.931033 29389 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:27.994132 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:28.011413 29609 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:27.931209 29591 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.012458 29591 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:27.984666 29580 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.993541 29431 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:28.013181 29553 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:28.013270 29580 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.013701 29431 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:27.921793 29606 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:08:28.016587 29431 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:27.933684 29590 raft_consensus.cc:740] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:28.017261 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.017805 29580 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.009577 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.018023 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.018229 29568 raft_consensus.cc:2764] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:08:28.017820 29590 consensus_queue.cc:260] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.017833 21930 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:28.019276 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.034325 21933 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:27.964098 29556 consensus_queue.cc:237] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.023569 21926 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:28.023546 21931 raft_consensus.cc:2368] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23.
I20260709 13:08:28.047487 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:27.952746 29364 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:28.048369 21931 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:28.056532 21933 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:27.969969 29554 consensus_meta.cc:287] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.006s
I20260709 13:08:28.058030 22001 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:28.169802 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:28.170544 22001 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:28.062798 22591 raft_consensus.cc:2468] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:28.066851 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:28.067482 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.070269 29561 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.071741 29591 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.073391 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.080261 29481 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.035173 29528 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.085387 29609 consensus_queue.cc:1059] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:28.086469 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:28.088212 28820 consensus_meta.cc:287] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.000s
I20260709 13:08:28.128825 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.134231 22000 raft_consensus.cc:1240] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:28.033398 29585 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:28.197670 29585 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.169802 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.199492 21933 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:28.088984 29389 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.043958 29563 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:28.144949 21930 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:28.201155 29563 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.201665 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.202251 21930 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:28.203199 22000 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:28.062410 29077 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:28.149322 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 313635us
W20260709 13:08:28.206463 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 432828us
I20260709 13:08:28.177173 29554 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:28.149893 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 346225us
I20260709 13:08:28.215070 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.215732 21934 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:28.129053 29466 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:28.157327 21926 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25.
I20260709 13:08:28.085274 29580 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:08:28.145788 29594 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:28.162420 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 96939.8us
I20260709 13:08:28.157065 29534 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.156629 29382 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:28.173650 21929 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:28.174227 21900 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:28.243954 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:28.174466 21935 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:28.144804 29538 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.185703 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.264696 21931 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:28.189520 29341 raft_consensus.cc:3055] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:28.191250 29591 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.195164 29597 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:28.200507 22001 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:28.053508 29592 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:28.205511 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.283093 21933 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:28.206941 21898 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:28.285030 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 78120.8us
I20260709 13:08:28.091972 29590 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:28.207275 22556 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.287544 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 80288.1us
I20260709 13:08:28.288177 22003 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:28.219697 21930 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:28.290632 21930 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:28.226503 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:28.230453 22588 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:28.222618 21974 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:28.243978 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:28.185568 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 99213us
I20260709 13:08:28.246114 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:28.252826 29594 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.223480 29555 raft_consensus.cc:2764] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
W20260709 13:08:28.254870 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:28.257618 21927 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:28.259805 29614 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:28.261338 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:28.261648 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:28.258904 29563 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.269373 21931 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
W20260709 13:08:28.270120 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 96014.4us
I20260709 13:08:28.270695 22592 raft_consensus.cc:1240] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 24. Current term is 25. Ops: [24.17-24.17]
W20260709 13:08:28.288496 22556 leader_election.cc:336] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.186957 29528 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.292814 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:28.187126 29481 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.357929 22001 raft_consensus.cc:1275] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:28.205406 28820 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.300173 21933 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:28.360219 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.360698 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.360885 21931 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:28.361145 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:28.362238 21931 raft_consensus.cc:1240] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 21. Current term is 22. Ops: [20.12-21.13]
I20260709 13:08:28.272950 29305 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 13:08:28.301347 21898 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.217504 29077 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:28.366077 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 64775.8us
I20260709 13:08:28.238591 29466 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.268157 29341 raft_consensus.cc:740] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:28.300628 29518 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:28.347097 29382 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.263614 29583 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:28.354362 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 65907.4us
I20260709 13:08:28.366114 29532 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.355363 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:28.325819 21930 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: []
W20260709 13:08:28.366500 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:28.302671 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 80295.2us
I20260709 13:08:28.273938 29597 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.266706 29534 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.327250 29431 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:28.352203 29544 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.282122 29592 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:28.287716 29550 raft_consensus.cc:2804] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:28.386186 22008 raft_consensus.cc:2468] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
W20260709 13:08:28.354609 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99776.2us
I20260709 13:08:28.349517 29565 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
W20260709 13:08:28.387483 22556 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.473564 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 86102.6us
W20260709 13:08:28.474524 22556 leader_election.cc:336] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.475514 22556 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.475941 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:28.390648 21927 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:28.281808 29545 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 13:08:28.371062 29582 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:28.451828 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49115.8us
I20260709 13:08:28.370826 29521 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.454587 29597 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:28.451749 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49688.6us
W20260709 13:08:28.468624 21972 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.506404 21972 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.507186 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.400677 29530 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:28.454200 21933 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:28.488487 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:28.506949 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:28.387001 29554 consensus_queue.cc:1048] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:08:28.521276 29580 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:28.518705 21972 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.473510 29592 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:28.456120 29532 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:28.565748 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 272996us
W20260709 13:08:28.559406 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.502523 29550 raft_consensus.cc:697] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:28.537638 29359 consensus_meta.cc:287] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.001s
W20260709 13:08:28.572817 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.574846 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.575268 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:28.538851 29431 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:28.474246 29544 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.578805 21825 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:28.596196 21926 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:28.509539 29521 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.505187 29582 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.620733 21926 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:28.621531 29582 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.621961 29521 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.457779 29341 consensus_queue.cc:260] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.579579 29359 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.575712 29594 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.561497 29049 consensus_queue.cc:1048] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:08:28.572095 29389 consensus_queue.cc:1059] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.043s
W20260709 13:08:28.613510 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:28.608464 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94832.9us
I20260709 13:08:28.621980 21926 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:08:28.654183 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 298920us
W20260709 13:08:28.655313 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.656522 21823 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:28.628225 29433 consensus_meta.cc:287] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.814s	user 0.013s	sys 0.000s
I20260709 13:08:28.610726 29528 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.670812 29389 raft_consensus.cc:3055] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:28.671490 29389 raft_consensus.cc:740] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:28.613803 29538 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.676210 29544 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.678656 29549 raft_consensus.cc:3060] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:28.678931 29532 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:28.661562 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.615268 29431 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:28.655100 22556 consensus_peers.cc:597] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:28.699633 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.702219 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:28.650887 29582 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:28.709218 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 52806.1us
I20260709 13:08:28.665215 29554 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.713909 21823 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:28.731449 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 156580us
W20260709 13:08:28.744127 21972 leader_election.cc:336] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.744554 21972 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:28.745509 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.703742 29626 log.cc:927] Time spent T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:08:28.703730 29629 log.cc:927] Time spent T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:08:28.703378 29599 log.cc:927] Time spent T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:08:28.749487 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:28.700040 28820 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.659900 29594 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.696641 29433 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.754108 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:28.672616 29389 consensus_queue.cc:260] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.730823 29466 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.730247 29411 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:28.717151 29528 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:28.732398 29077 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.744127 29563 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:28.759691 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:28.702456 29431 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.750068 29591 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:28.749176 29305 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:28.703624 29592 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.752275 29481 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:28.771268 22006 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:28.703603 29640 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.134s
I20260709 13:08:28.781935 29594 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.760581 29534 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.738739 29554 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.765411 29556 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:28.792270 21823 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:28.798657 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:28.793330 29640 raft_consensus.cc:3055] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:28.793841 29305 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:28.796094 29631 log.cc:927] Time spent T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:08:28.743129 29550 consensus_queue.cc:237] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.797461 29466 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:28.777130 29049 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:28.804049 29640 raft_consensus.cc:740] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:28.824584 29640 consensus_queue.cc:260] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.806958 29431 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.815090 29049 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.827960 29049 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:28.806669 29305 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.808933 21928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1348) took 1073 ms. Trace:
I20260709 13:08:28.829262 21928 rpcz_store.cc:276] 0709 13:08:27.735785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.283217 (+547432us) service_pool.cc:224] Handling call
0709 13:08:28.283678 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:28.284121 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.284158 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:28.284742 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:08:28.611062 (+326320us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:28.611132 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.611222 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:28.808722 (+197500us) raft_consensus.cc:1685] finished
0709 13:08:28.808743 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.808875 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14248,"raft.queue_time_us":3130,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":46335,"thread_start_us":254773,"threads_started":1,"wal-append.queue_time_us":391542}
I20260709 13:08:28.810674 29592 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.784085 29077 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:28.831147 29077 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:28.831599 29077 raft_consensus.cc:3060] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:28.786276 29528 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:28.808811 29433 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:28.816141 29632 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:08:28.835711 29597 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:28.816442 29630 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:08:28.836030 29528 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.837133 21929 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:28.843590 29359 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.820447 29518 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:28.811803 29521 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:08:28.800604 29582 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:28.822515 21935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37776 (request call id 1347) took 1087 ms. Trace:
I20260709 13:08:28.845422 21935 rpcz_store.cc:276] 0709 13:08:27.735464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.268141 (+532677us) service_pool.cc:224] Handling call
0709 13:08:28.273467 (+  5326us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:08:28.344550 (+ 71083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.344584 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:08:28.345326 (+   742us) log.cc:844] Serialized 78 byte log entry
0709 13:08:28.587119 (+241793us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:28.587179 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.587282 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:28.822264 (+234982us) raft_consensus.cc:1685] finished
0709 13:08:28.822285 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.822446 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43122,"raft.queue_time_us":27164,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":27879,"thread_start_us":241294,"threads_started":1,"wal-append.queue_time_us":385720}
W20260709 13:08:28.851253 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:28.851933 22556 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.852396 22556 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:28.853195 29532 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:28.853569 22556 leader_election.cc:336] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.854568 22556 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:28.855070 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:28.855600 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.856133 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:28.856496 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:28.847975 29466 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:28.859766 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:28.860705 22556 leader_election.cc:336] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:28.861570 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:28.845088 29521 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:28.864439 22556 consensus_peers.cc:597] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:28.862330 29532 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:28.866891 29554 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.870555 21927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33902 (request call id 3344) took 1273 ms. Trace:
I20260709 13:08:28.870782 21932 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:28.871045 21927 rpcz_store.cc:276] 0709 13:08:27.596761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.597657 (+   896us) service_pool.cc:224] Handling call
0709 13:08:28.870246 (+1272589us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b540042a4d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:28.870468 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502202240}
I20260709 13:08:28.849890 29582 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:28.888190 29582 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:28.866189 29521 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:08:28.896463 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:28.896889 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:28.858127 29466 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:28.900133 22006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41328 (request call id 3440) took 1007 ms. Trace:
I20260709 13:08:28.897675 22592 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 25.
W20260709 13:08:28.909411 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:28.905357 22006 rpcz_store.cc:276] 0709 13:08:27.892525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.892661 (+   136us) service_pool.cc:224] Handling call
0709 13:08:28.899900 (+1007239us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b540038aed4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:28.900066 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888450432}
I20260709 13:08:28.915216 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.916054 21927 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:08:28.923135 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.923835 21932 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:28.929214 21927 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:28.928844 21900 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:28.889657 29049 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:28.924140 29590 consensus_meta.cc:287] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.002s
I20260709 13:08:28.890105 29077 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:28.943581 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 182471us
I20260709 13:08:28.947129 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.947047 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:28.949288 22007 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:08:28.949633 22007 pending_rounds.cc:85] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 12
I20260709 13:08:28.949877 22007 pending_rounds.cc:107] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:08:28.950169 22007 raft_consensus.cc:2889] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:28.951226 29644 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:08:28.952726 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.953655 29049 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.944013 29528 consensus_queue.cc:1059] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:08:28.949790 29585 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:08:28.955132 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.951885 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.956090 21932 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:28.956704 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.957283 21933 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:28.958509 29077 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:28.961138 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.967267 21933 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:28.941884 29554 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:28.945768 29597 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:28.979254 29590 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:28.972041 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:28.981840 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:28.984087 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:28.998689 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:28.999347 21933 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:28.999953 21927 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:08:29.001648 21823 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:29.014030 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.014739 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.015202 21933 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 20: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:29.025692 21928 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:29.025978 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:28.970715 29554 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.025331 29528 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:29.062279 29528 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:29.062510 29530 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.063226 29530 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.050107 29592 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.064405 29592 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.065186 29518 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:29.034292 29077 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:08:29.065765 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.065711 29518 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.066512 29077 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:29.066890 29077 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:28.998831 29359 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:29.063280 29528 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.067823 29518 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.068042 29359 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.068446 29359 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:29.067720 29077 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.069860 21929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33902 (request call id 3356) took 1423 ms. Trace:
I20260709 13:08:29.070240 21929 rpcz_store.cc:276] 0709 13:08:27.646626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.257090 (+610464us) service_pool.cc:224] Handling call
0709 13:08:28.257399 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.837522 (+580123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.069648 (+232126us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54003066d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:29.069805 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332781824}
I20260709 13:08:29.071569 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.072253 21929 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:29.049059 29568 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.067616 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.083163 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.006500 29590 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:29.005942 29644 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:29.024407 29585 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:29.086799 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.067514 29466 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.091069 21927 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:29.082324 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:29.097878 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.099596 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.100876 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.101478 22000 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:08:29.042384 29554 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:29.102473 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.112794 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.115252 21927 raft_consensus.cc:1240] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:29.084002 29049 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:29.093909 29585 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:29.123314 29580 consensus_meta.cc:287] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.000s
I20260709 13:08:29.094931 29341 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:29.037215 29582 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:08:29.122823 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:29.145459 22585 raft_consensus.cc:2393] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 23: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:29.097718 29077 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:29.101608 29644 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:29.147429 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.093475 29364 consensus_queue.cc:1059] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:08:29.149439 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.093665 29568 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.145366 29585 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:29.170907 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.141439 29550 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:29.101161 29543 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:08:29.174913 29364 raft_consensus.cc:3055] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:29.142445 29466 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.146970 29580 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:29.163985 29592 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:29.183182 21972 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:29.185148 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:29.186869 21972 leader_election.cc:336] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:29.188290 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.188788 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:29.192237 29659 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:08:29.193995 21972 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.194258 22009 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25.
I20260709 13:08:29.145069 29049 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.194551 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:29.200155 29153 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:29.200640 29153 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:29.203495 21972 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.181483 29364 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:29.205895 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:29.218012 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:29.209954 29530 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.213192 29568 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:29.219126 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:29.224965 21972 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:29.226804 21972 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.227326 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:29.196586 29592 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:29.202205 29594 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:29.229064 29663 log.cc:927] Time spent T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:08:29.193269 29466 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.230837 29651 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:29.228390 29382 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:29.183358 29550 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.194276 29590 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.256421 29642 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:29.297695 29521 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.300308 29049 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.256582 29382 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.311326 29642 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.255311 29364 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.305239 29521 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 27 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.403590 29538 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.442399 29538 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.444991 29538 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.406142 29642 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.413414 29364 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:29.418066 29466 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.420029 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:29.422070 21825 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:29.423385 22007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3448) took 1368 ms. Trace:
I20260709 13:08:29.459262 22007 rpcz_store.cc:276] 0709 13:08:28.055102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.056025 (+   923us) service_pool.cc:224] Handling call
0709 13:08:28.065829 (+  9804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.950481 (+884652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.423068 (+472587us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54001e19d4. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:29.423307 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031107840}
I20260709 13:08:29.427220 29585 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.431560 29554 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.439785 22009 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:08:29.420756 29550 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.251331 29644 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:29.483624 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61619.4us
I20260709 13:08:29.481531 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:29.486631 29627 log.cc:927] Time spent T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.949s	user 0.005s	sys 0.000s
I20260709 13:08:29.481293 29563 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:29.492113 29521 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.486210 29466 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:29.500336 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.503651 22585 raft_consensus.cc:2410] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:29.499790 22009 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:29.501178 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.501358 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.502205 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.502277 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.506420 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.506695 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:29.508939 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.481070 29556 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:08:29.516100 29561 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:29.519259 29676 consensus_queue.cc:1059] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:29.519737 29544 consensus_queue.cc:1059] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:29.521138 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.522316 29554 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.503930 29543 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:29.524240 29585 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.524322 22590 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:29.526515 29543 raft_consensus.cc:493] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.527055 29676 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:29.527834 29676 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.515611 29382 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.529874 29676 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:29.496052 29528 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:29.530501 29528 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.531002 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:29.531920 29528 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.507939 29669 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:08:29.534257 29543 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:29.498831 29521 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.537194 29521 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.548260 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.548888 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.549585 21932 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
W20260709 13:08:29.555661 21900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.557178 21934 raft_consensus.cc:2468] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:29.543260 29678 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:08:29.575626 21824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.20.66.131:36521 timed out after 1.517s (SENT) [suppressed 149 similar messages]
I20260709 13:08:29.561919 29609 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.577868 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.578723 22591 raft_consensus.cc:1275] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:08:29.584893 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:29.581028 29565 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.597762 22009 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:29.587540 29305 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.590818 29565 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.590798 29581 consensus_queue.cc:1048] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
W20260709 13:08:29.617431 21973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.637166 29557 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:29.637814 29557 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.638839 29565 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.609431 29543 raft_consensus.cc:515] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.641624 29543 leader_election.cc:290] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:29.606165 29359 consensus_meta.cc:287] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.004s
I20260709 13:08:29.642519 29359 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.581558 29678 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.644654 29359 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.646061 29678 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.587459 29609 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:29.661424 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.649760 29389 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:29.648025 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 386250us
I20260709 13:08:29.592203 29153 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.664204 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:29.664403 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.667204 22009 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 24: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
I20260709 13:08:29.640705 29557 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.673110 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.685249 21934 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 26: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:29.687466 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.688498 22002 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:08:29.688422 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:29.689682 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.693809 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.695135 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.695147 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:29.699337 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.700573 22006 raft_consensus.cc:2410] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:08:29.700637 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.701608 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:29.702103 22007 raft_consensus.cc:3060] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:29.702304 22591 raft_consensus.cc:2410] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:08:29.669365 29525 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:29.704623 29525 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.707073 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:29.709041 21927 raft_consensus.cc:1240] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:08:29.707726 22009 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:29.710944 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.711722 21931 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:29.700542 29426 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:29.720780 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.724584 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:29.727478 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.735409 22591 raft_consensus.cc:1240] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:29.737030 22003 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 21: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:29.739923 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.684806 29591 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:29.740761 21934 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:08:29.708244 29592 consensus_queue.cc:1059] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:08:29.745492 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.751359 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:29.751441 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.745949 29426 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.747820 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:29.748508 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.753122 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:29.753582 21927 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:29.753998 22001 raft_consensus.cc:2393] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 20: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:29.753274 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.700682 29646 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:29.700337 29563 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:29.709894 29651 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:08:29.760932 22006 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:29.763999 21934 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 27.
W20260709 13:08:29.749500 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 173888us
I20260709 13:08:29.705631 29557 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:29.754733 22002 raft_consensus.cc:2468] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 18.
I20260709 13:08:29.755106 22007 raft_consensus.cc:2468] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 26.
I20260709 13:08:29.767213 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.766505 21934 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:29.661823 29305 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.768162 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.769696 22557 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:29.770886 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:29.772879 22007 raft_consensus.cc:2393] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:29.741243 29597 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:29.753036 29049 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:29.754794 29153 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:29.775127 22006 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 27.
I20260709 13:08:29.778870 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.780511 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:29.780618 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 73668us
I20260709 13:08:29.781848 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.789423 22003 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 26: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:29.777190 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:29.789757 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.789840 21972 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:29.791144 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.792145 22007 raft_consensus.cc:2410] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:29.783784 21927 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:29.787279 21931 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:29.775633 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:29.776594 29681 log.cc:927] Time spent T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:08:29.767593 29557 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:29.802717 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:29.788020 29651 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.774410 29592 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:29.758253 29591 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.780622 29521 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.817054 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:29.822444 22002 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:29.824118 22003 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:29.783804 29382 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:29.768785 29669 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:08:29.766384 29646 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.767359 29563 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.857182 29563 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:29.857846 29563 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.858299 29563 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.860185 29563 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.819981 29557 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:29.801667 29642 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.810099 29676 raft_consensus.cc:2804] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:08:29.850033 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47427.6us
I20260709 13:08:29.853065 29525 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.784410 29556 consensus_queue.cc:1059] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 13:08:29.849509 29561 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.855198 29646 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.784865 29049 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:29.828056 29544 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 13:08:29.868341 21823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.851018 29382 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.843634 29521 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.870985 29426 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.816869 29651 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:08:29.871758 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 102179us
I20260709 13:08:29.833617 29592 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.825802 29649 raft_consensus.cc:2804] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:29.877254 21824 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:29.900781 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:29.860805 29466 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:29.785708 29597 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.873277 29551 raft_consensus.cc:493] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:29.895918 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 94614.5us
I20260709 13:08:29.887698 29049 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:29.867664 29678 consensus_queue.cc:1059] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:08:29.923522 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:29.923914 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.924355 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.931083 21927 raft_consensus.cc:2393] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:29.930431 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 152422us
I20260709 13:08:29.926141 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.929030 29582 consensus_queue.cc:1059] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:08:29.925396 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:29.930760 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:29.938993 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.878185 29642 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.939483 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:29.900066 29676 raft_consensus.cc:697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 26 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:29.911348 29359 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:29.912776 29609 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:29.944772 21972 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.919214 28820 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:29.946604 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:29.916869 29649 raft_consensus.cc:697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:29.948882 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:29.949904 21972 leader_election.cc:336] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.950914 21972 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:29.886250 29561 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.967113 29561 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.955072 29687 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:29.969988 29549 raft_consensus.cc:493] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.970541 29549 raft_consensus.cc:515] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.975157 29431 raft_consensus.cc:3055] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:29.975796 29431 raft_consensus.cc:740] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:29.944378 29563 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:29.962399 29609 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.932972 29551 raft_consensus.cc:515] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.964138 29642 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.937696 29077 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.942927 29646 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:29.973503 29640 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:29.972721 29549 leader_election.cc:290] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:29.996513 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 46654.7us
W20260709 13:08:30.033504 21972 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:30.034627 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.035111 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.008746 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.020856 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:30.020876 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 120142us
W20260709 13:08:30.041075 21972 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.041942 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:30.042977 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 83856.1us
I20260709 13:08:30.043843 22007 raft_consensus.cc:1240] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:30.045164 21898 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:30.047482 21898 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.047993 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:30.030347 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 69887.7us
I20260709 13:08:30.048659 22585 raft_consensus.cc:3055] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:30.049218 22585 raft_consensus.cc:740] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
W20260709 13:08:30.049525 21898 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.050088 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:30.052506 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.048498 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.064127 22007 raft_consensus.cc:2393] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 24: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:30.065680 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:29.977936 29411 raft_consensus.cc:3055] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:29.981215 29597 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.981267 29649 consensus_queue.cc:237] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.005362 29305 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.989446 29521 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.992522 29551 leader_election.cc:290] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.073649 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:30.067875 21898 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.990746 29592 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:29.984438 29591 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:30.002023 29382 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:29.996169 29563 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:29.996690 29687 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:30.073530 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.051903 29426 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.085615 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:30.052636 29553 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:30.086341 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.077793 29692 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.093531 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.115973 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:30.079847 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.997298 29077 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:29.977594 29676 consensus_queue.cc:237] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.105980 22591 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:30.075809 29389 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:30.075706 29691 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.108001 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.074424 29411 raft_consensus.cc:740] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:30.075857 29568 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:30.074594 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 158745us
I20260709 13:08:30.133540 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.133710 29557 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.135601 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:30.137238 29153 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:30.138892 29153 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:30.140116 22589 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
W20260709 13:08:30.141304 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:29.981649 29609 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.141763 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.142655 29411 consensus_queue.cc:260] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.119855 29692 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.144245 29411 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:30.110113 21898 leader_election.cc:336] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.098459 29651 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.105181 29563 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.145260 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.145771 29692 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:30.111706 29426 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:29.976972 29431 consensus_queue.cc:260] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:30.146901 21898 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.147586 21898 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.074115 29581 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:30.123644 29597 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:30.111758 29694 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:30.131233 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.147753 29431 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:30.130026 29691 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.150362 29077 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:30.151494 29426 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.152092 29691 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:30.151508 29702 log.cc:927] Time spent T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:08:30.154042 29609 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.154652 29609 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:30.149430 29553 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:30.155292 21898 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.146899 29563 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.136039 29557 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:30.149375 29597 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.160501 29597 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:30.161877 29382 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:30.157239 21824 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:30.166863 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:30.167723 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:30.121557 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.168974 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:30.169757 29530 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:30.170428 29530 raft_consensus.cc:2804] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:30.170766 29530 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.171115 29530 raft_consensus.cc:3060] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:30.172528 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.173254 22007 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:30.173728 29525 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:30.174413 29525 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:30.176766 29679 raft_consensus.cc:3055] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:30.177314 29679 raft_consensus.cc:740] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:30.178251 29679 consensus_queue.cc:260] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.183166 29530 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.184147 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:30.185542 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 144517us
W20260709 13:08:30.186269 21972 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.186681 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.190343 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:30.193846 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42af043129e482498af1e2c3882a042" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.197469 22000 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:30.199935 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.201664 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.203408 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.204270 22006 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:30.205403 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.206161 22006 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:30.207185 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.207870 22006 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 20.
I20260709 13:08:30.208977 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.158598 21898 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.212245 22008 raft_consensus.cc:1275] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:08:30.239282 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 53041.1us
I20260709 13:08:30.243121 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.243773 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ef7ed88974a427b9cab09af73283182" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.246245 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.247102 22007 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:30.248533 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:30.250240 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.251968 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.253937 22008 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:30.254300 21823 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:30.250751 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:30.250520 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 90923.3us
I20260709 13:08:30.259089 21823 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:30.179738 29679 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:30.266405 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.266920 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.270721 22004 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:30.237462 29359 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.238056 29525 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:30.274494 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.242141 29426 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:30.270529 21926 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:30.277029 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.240516 29561 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:08:30.241729 29651 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 28 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:30.289850 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.296578 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.297228 22590 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:30.297497 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:30.297228 22588 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:30.310923 29481 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:30.211385 29549 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:08:30.311961 29644 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.313318 29153 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:30.314224 29565 raft_consensus.cc:3055] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:30.314667 29565 raft_consensus.cc:740] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:30.315523 21972 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.316419 29153 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.287679 29359 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.324654 29359 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.291149 29578 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:30.291922 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.271703 29024 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:30.276279 29696 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:30.343794 21930 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:30.286427 21974 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.293601 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.295207 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:30.345752 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59386.9us
I20260709 13:08:30.346371 21928 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:08:30.326102 29530 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:30.272703 29077 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:30.256343 29544 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:30.290838 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.331143 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:30.336269 22004 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:30.339815 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.346653 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.345598 21932 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:30.372445 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:30.250372 29049 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.287297 29597 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:30.372391 22557 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:30.386817 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:30.394280 21928 raft_consensus.cc:2468] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 28.
W20260709 13:08:30.436889 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 97141.7us
I20260709 13:08:30.312661 29644 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.341292 29561 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 27 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:30.340132 29691 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:30.453395 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:30.380116 29481 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:30.403548 29703 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:30.398564 29597 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.380533 29431 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:30.379388 29696 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:30.366493 29426 raft_consensus.cc:2764] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:08:30.395102 29411 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:30.451071 29649 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:30.475562 29049 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.476133 22003 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:30.477630 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 205917us
I20260709 13:08:30.479625 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:30.499325 21972 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:30.500743 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:30.403193 29563 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:30.345660 29153 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.379935 29578 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:30.483100 22585 consensus_queue.cc:260] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:30.505461 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 350173us
I20260709 13:08:30.527943 22585 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:30.528023 21898 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.375034 29024 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.481689 29481 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.481943 29703 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:30.529454 21898 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.525069 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:30.506626 29568 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:08:30.530819 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 271800us
W20260709 13:08:30.520305 29686 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.776s	user 0.002s	sys 0.006s
I20260709 13:08:30.494561 29077 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:30.538337 21972 leader_election.cc:336] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.539660 21929 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:30.530184 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.546084 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.551313 21926 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 26: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:30.526667 29563 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.526626 29431 raft_consensus.cc:2804] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:08:30.521708 29411 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:30.526511 28820 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:30.557116 21899 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:30.564796 21929 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
W20260709 13:08:30.556808 21972 leader_election.cc:336] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.563041 21899 leader_election.cc:304] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:30.572119 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.573110 21926 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:30.574270 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.575328 21935 raft_consensus.cc:2410] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 13:08:30.566390 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.577749 22556 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.571602 21972 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.540678 29481 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:30.608875 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.597868 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.548954 29568 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:30.541989 29703 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:30.615839 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 59079.1us
I20260709 13:08:30.617905 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:30.618804 21933 raft_consensus.cc:2376] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23. Re-sending same reply.
W20260709 13:08:30.595788 29590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.024s	sys 0.066s
W20260709 13:08:30.623555 29704 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:08:30.624451 29708 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:08:30.625217 29578 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.629439 29710 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:30.630630 29649 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:30.629845 29411 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.567397 29563 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:30.633258 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.569702 28820 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:30.638185 29590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.024s	sys 0.067s
I20260709 13:08:30.584877 29585 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.646133 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:30.639581 21972 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:30.647212 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:30.616510 29692 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:30.615322 29431 raft_consensus.cc:697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:30.700253 21926 raft_consensus.cc:2368] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19.
I20260709 13:08:30.659675 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.637544 29568 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:30.632122 29676 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:30.690359 29557 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:30.697763 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:30.706604 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:30.708647 22584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57084 (request call id 1467) took 1616 ms. Trace:
I20260709 13:08:30.699563 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:30.712327 22009 raft_consensus.cc:1240] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:30.712750 22584 rpcz_store.cc:276] 0709 13:08:29.092109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.092295 (+   186us) service_pool.cc:224] Handling call
0709 13:08:29.092855 (+   560us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:29.114044 (+ 21189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:29.114090 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:29.123833 (+  9743us) log.cc:844] Serialized 52 byte log entry
0709 13:08:29.617366 (+493533us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b18003c2380. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6c4e07 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b7036920b 00007f3b7036ed23 00007f3b7036f0d1 00007f3b7078aa0f
0709 13:08:29.692367 (+ 75001us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:29.692429 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.692527 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:30.287794 (+595267us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:30.708358 (+420564us) raft_consensus.cc:1685] finished
0709 13:08:30.708388 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:30.708576 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9093,"raft.queue_time_us":24341,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9393,"spinlock_wait_cycles":60464256,"thread_start_us":73604,"threads_started":1,"wal-append.queue_time_us":620152}
I20260709 13:08:30.698288 28820 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:30.585301 29561 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.709566 29609 raft_consensus.cc:2749] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:08:30.748234 29431 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:30.750777 29382 consensus_meta.cc:287] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 13:08:30.751220 29382 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:30.756664 29597 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.713201 29692 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.763485 29692 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:30.765239 29153 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:30.767678 22586 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:30.719672 29649 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:30.712543 29585 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.771366 29585 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:30.723783 29557 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.774784 29557 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:30.649240 29696 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:30.801167 29667 log.cc:927] Time spent T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 1.505s	user 0.007s	sys 0.000s
I20260709 13:08:30.780201 29703 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.789331 29644 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.826689 29703 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:30.861622 21824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.001s	sys 0.023s
W20260709 13:08:30.862597 21824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.001s	sys 0.023s
W20260709 13:08:30.864494 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 301358us
W20260709 13:08:30.870677 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.881425 21823 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:30.881767 21928 raft_consensus.cc:1240] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:30.882184 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.884245 21935 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 26: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:30.890651 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.893867 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:30.900223 29024 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:30.946314 29651 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.946489 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:30.947697 29651 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:30.957124 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.961323 29727 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.962689 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:30.962560 29727 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:30.963938 21932 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
W20260709 13:08:30.966580 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 326830us
I20260709 13:08:30.958568 29525 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.955874 21930 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
I20260709 13:08:30.975560 22001 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:30.990761 22586 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:30.992004 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:30.999130 29651 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.002914 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.003911 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.005920 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.007400 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.009403 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.010185 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.010907 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.011448 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:31.013299 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 132523us
I20260709 13:08:31.014154 29712 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.038755 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.039646 22000 raft_consensus.cc:2468] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:31.039287 29712 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.015340 29549 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:31.015795 22005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1509) took 2794 ms. Trace:
I20260709 13:08:31.017956 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:31.019431 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44903.9us
I20260709 13:08:31.022889 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.023869 29553 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:31.026585 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:31.027343 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 58418.2us
I20260709 13:08:31.020632 22003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32904 (request call id 1527) took 1121 ms. Trace:
I20260709 13:08:31.037992 29717 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.038285 22584 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:31.046747 29549 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.047832 29712 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:31.048177 21898 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.049261 21928 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:31.054013 21935 raft_consensus.cc:2468] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:31.038643 29545 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.062165 29545 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.050052 22003 rpcz_store.cc:276] 0709 13:08:29.899093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.900039 (+   946us) service_pool.cc:224] Handling call
0709 13:08:29.920828 (+ 20789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:30.504078 (+583250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.019871 (+515793us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b540040c254. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:31.020367 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232697600}
I20260709 13:08:31.066236 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.066474 21898 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.066736 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.067770 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.058995 29549 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:31.069888 29565 consensus_queue.cc:260] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.071502 29565 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:31.071889 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:31.055039 22009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41328 (request call id 3470) took 1309 ms. Trace:
I20260709 13:08:31.056268 22592 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:31.073340 22009 rpcz_store.cc:276] 0709 13:08:29.745897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.746718 (+   821us) service_pool.cc:224] Handling call
0709 13:08:29.747019 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:30.713909 (+966890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.035414 (+321505us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b5400165b54. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cfe01 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:31.054838 (+ 19424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219541760}
I20260709 13:08:31.074802 22004 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:08:31.057256 29717 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.048908 28820 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.081607 21972 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:31.082455 29717 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.083513 28820 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.070580 21928 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:31.083817 29153 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:31.084605 29727 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.085520 29534 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:31.085731 21928 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:31.086097 29534 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.086525 21930 raft_consensus.cc:2376] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25. Re-sending same reply.
W20260709 13:08:31.087221 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:31.087595 21928 raft_consensus.cc:1275] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:08:31.088037 21898 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.088577 21898 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.089413 29669 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:08:31.090423 21898 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.091038 21898 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.092347 29669 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:31.086621 29534 raft_consensus.cc:3060] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:31.051493 29553 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.093509 29688 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:08:31.093596 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 43913.5us
I20260709 13:08:31.083313 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.095994 22008 raft_consensus.cc:1240] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:31.096272 29341 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:31.044860 22005 rpcz_store.cc:276] 0709 13:08:28.220699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.250336 (+ 29637us) service_pool.cc:224] Handling call
0709 13:08:28.682275 (+431939us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:28.682636 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.682663 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:29.112930 (+430267us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b2c000337c8. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6f0c8a11 00007f3b6f0d8372 00007f3b7080aa01 00007f3b707cf43a 00007f3b707ce83e 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2
0709 13:08:29.122618 (+  9688us) log.cc:844] Serialized 52 byte log entry
0709 13:08:29.228480 (+105862us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:29.228519 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.228593 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:30.800143 (+1571550us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b7036a86e 00007f3b70369bb6 00007f3b703697d8 00007f3b70373221 00007f3b703731d9 00007f3b70373169
0709 13:08:30.885508 (+ 85365us) raft_consensus.cc:1685] finished
0709 13:08:30.885553 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.015317 (+129764us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540042bdd0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707ce8ad 00007f3b75793419 00007f3b6ed0abd4 00007f3b6ed0ab62 00007f3b6ed0aa91 00007f3b6ed0aa0c 00007f3b6ed09cc2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:31.015548 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14922,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":3366,"spinlock_wait_cycles":1008682368,"thread_start_us":105499,"threads_started":1,"wal-append.queue_time_us":169052}
I20260709 13:08:31.100492 21899 leader_election.cc:304] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:31.137352 29545 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.101519 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.138188 29545 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:31.138553 21826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:31.102188 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.139120 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.140583 21933 raft_consensus.cc:1240] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:31.101866 29721 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:31.127724 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:31.141851 22589 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:31.142299 29721 raft_consensus.cc:2749] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.097672 29550 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:31.088060 21930 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:31.143936 21927 raft_consensus.cc:1240] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:31.111303 29153 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:31.083256 29551 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:31.112708 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.113758 22588 raft_consensus.cc:2410] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:31.086158 21935 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:31.086633 29703 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:31.127858 29525 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.128899 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:31.129230 21898 leader_election.cc:336] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.131524 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:31.134438 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 51349us
I20260709 13:08:31.137928 21899 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:31.153731 21930 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:31.155797 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.163635 29717 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:31.164224 29717 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.164695 29717 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:31.175823 29717 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.177467 29729 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:31.177980 29729 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.178297 29729 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:31.179700 29717 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.183077 22589 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:31.187354 29703 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:31.190323 21898 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.191613 29550 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:31.192106 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62987.9us
I20260709 13:08:31.189117 29703 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:31.205071 29563 consensus_meta.cc:287] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.007s
I20260709 13:08:31.205812 29563 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.207458 29553 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.125747 29688 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:31.210196 29688 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.211632 29688 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:31.213809 21753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 195) took 28554 ms (client timeout 15000 ms). Trace:
I20260709 13:08:31.222738 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:31.223389 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.236408 29341 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.241046 29341 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.244429 29341 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.237519 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.244848 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.246146 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.246863 21927 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 19.
I20260709 13:08:31.247692 29717 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:31.249042 29549 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.224844 29525 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.252943 21825 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.252450 29721 consensus_queue.cc:1048] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:31.257211 21974 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:31.258605 29590 raft_consensus.cc:2804] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:31.259294 29590 raft_consensus.cc:697] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:31.262646 22591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:31.263384 22591 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:08:31.263630 21898 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:31.265870 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:31.266556 29568 consensus_meta.cc:287] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.011s
I20260709 13:08:31.268445 21931 raft_consensus.cc:1275] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:31.239642 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d337058bd1f4df1bd938c43960820be" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.275108 21928 raft_consensus.cc:1240] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:08:31.232537 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:31.269649 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:31.270730 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:31.285974 22556 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.262689 29649 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:31.260825 29590 consensus_queue.cc:237] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.287892 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.224635 29563 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:31.291554 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59726.7us
I20260709 13:08:31.252390 29549 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.304399 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.307080 29725 raft_consensus.cc:2804] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:31.307853 29725 raft_consensus.cc:697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:31.309084 29717 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:31.267143 29568 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.312208 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c6a16f96334a6a8bd95d99ea2ffc7e" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.317925 21931 raft_consensus.cc:3060] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:31.314306 29721 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.280776 29729 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:31.214457 21753 rpcz_store.cc:269] 0709 13:08:02.665186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.734763 (+ 69577us) service_pool.cc:224] Handling call
0709 13:08:31.213426 (+28478663us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:28.251830 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:28.252589 (+   759us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:28.252636 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:08:28.252663 (+    27us) tablet.cc:662] Decoding operations
0709 13:08:28.766057 (+513394us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:28.774667 (+  8610us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:28.780089 (+  5422us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:08:28.822844 (+ 42755us) tablet.cc:701] Row locks acquired
0709 13:08:28.822875 (+    31us) write_op.cc:260] PREPARE: finished
0709 13:08:28.823095 (+   220us) write_op.cc:270] Start()
0709 13:08:28.823287 (+   192us) write_op.cc:276] Timestamp: P: 1783602508823026 usec, L: 0
0709 13:08:28.823325 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:08:28.824386 (+  1061us) log.cc:844] Serialized 8570 byte log entry
0709 13:08:29.102018 (+277632us) op_driver.cc:464] REPLICATION: finished
0709 13:08:29.529316 (+427298us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b6d6c50ac 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6e1c02 00007f3b74205ef0
0709 13:08:29.614220 (+ 84904us) write_op.cc:301] APPLY: starting
0709 13:08:29.614307 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:30.139427 (+525120us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:30.139461 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:30.502257 (+362796us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:30.637032 (+134775us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:08:30.649581 (+ 12549us) write_op.cc:312] APPLY: finished
0709 13:08:30.697863 (+ 48282us) log.cc:844] Serialized 188 byte log entry
0709 13:08:30.992792 (+294929us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:30.993540 (+   748us) write_op.cc:454] Released schema lock
0709 13:08:30.993709 (+   169us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:30.994621 (+   912us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:30.994650 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":498354,"apply.run_cpu_time_us":75839,"apply.run_wall_time_us":1380193,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92033,"num_ops":21,"prepare.queue_time_us":434604,"prepare.run_cpu_time_us":12519,"prepare.run_wall_time_us":601892,"raft.queue_time_us":190074,"raft.run_cpu_time_us":24335,"raft.run_wall_time_us":557791,"replication_time_us":224266,"spinlock_wait_cycles":24970624,"thread_start_us":1232599,"threads_started":5,"wal-append.queue_time_us":259542,"wal-append.run_cpu_time_us":100027,"wal-append.run_wall_time_us":1196818}]]}
W20260709 13:08:31.358511 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 59855.6us
I20260709 13:08:31.348681 29341 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:31.363314 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 63686.2us
W20260709 13:08:31.361135 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 91573.4us
I20260709 13:08:31.367442 22556 leader_election.cc:400] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 18. Current term is 19.
I20260709 13:08:31.311395 29725 consensus_queue.cc:237] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.374284 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:31.371429 21972 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.376739 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:31.383621 21934 consensus_meta.cc:287] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.599s	user 0.014s	sys 0.001s
I20260709 13:08:31.384739 29550 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:31.394928 22585 consensus_meta.cc:287] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.829s	user 0.020s	sys 0.001s
I20260709 13:08:31.395094 29550 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.395982 22585 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:31.396886 22585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57098 (request call id 1545) took 1348 ms. Trace:
I20260709 13:08:31.397303 22585 rpcz_store.cc:276] 0709 13:08:30.048148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.048255 (+   107us) service_pool.cc:224] Handling call
0709 13:08:30.048542 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:30.450232 (+401690us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b4c0039f800. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6ea9ba20 00007f3b6eaaca29 00007f3b6ea72644 00007f3b6ea99f2d 00007f3b6ea99b93 00007f3b707ca18b 00007f3b707cacb5 00007f3b707c5f30 00007f3b707c77de 00007f3b707d1b49 00007f3b707d2416
0709 13:08:31.396479 (+946247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.396804 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56383872}
W20260709 13:08:31.398880 29532 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:08:31.401935 22586 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:31.405526 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 241296us
I20260709 13:08:31.369859 28820 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:08:31.409801 22557 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.377688 29556 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:31.411505 28820 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:31.412492 29676 consensus_meta.cc:287] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.007s
I20260709 13:08:31.413151 29676 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:31.414976 29568 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:31.417958 21934 raft_consensus.cc:2468] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:31.375133 29525 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:31.418993 29738 log.cc:927] Time spent T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.224s	user 0.000s	sys 0.006s
W20260709 13:08:31.427783 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 45503.8us
I20260709 13:08:31.432875 22591 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 26.
I20260709 13:08:31.354017 29721 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.439627 22584 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:31.412286 28820 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.443074 21931 raft_consensus.cc:2468] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 26.
I20260709 13:08:31.367949 29534 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:31.449391 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.455370 29591 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:31.456082 21754 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:31.456403 29591 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.457737 21754 catalog_manager.cc:5697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 8 to 15, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 15 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.379626 29341 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.459800 29591 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:31.460475 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 94769.9us
W20260709 13:08:31.461265 21972 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:31.461656 29341 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:31.461923 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.462368 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.462736 29649 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:31.463845 29649 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:31.451917 29532 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.464759 29550 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.465126 29532 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.467516 29532 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:31.474965 29721 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.475648 21754 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 9 to 21, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 21 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.476862 21974 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:31.480820 29565 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:08:31.484669 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.489264 22586 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:31.498381 21899 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:31.519778 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.518101 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:31.523648 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.498587 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:31.533496 22591 raft_consensus.cc:3055] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:31.534085 22591 raft_consensus.cc:740] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:31.547516 22584 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:31.549590 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:31.544237 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:31.553607 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.551074 22006 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:31.530691 29550 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:31.553653 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:31.563795 22002 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:31.558633 22008 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:31.498219 29565 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 13:08:31.491678 29549 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
W20260709 13:08:31.567736 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 243821us
I20260709 13:08:31.571480 22009 raft_consensus.cc:2410] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 13:08:31.573673 22001 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:31.578896 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 115253us
I20260709 13:08:31.582103 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.563563 29550 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
W20260709 13:08:31.575511 29481 consensus_meta.cc:287] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: Time spent flushing consensus metadata: real 0.805s	user 0.011s	sys 0.007s
I20260709 13:08:31.582546 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:31.593511 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69063.3us
I20260709 13:08:31.597400 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.599412 21931 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:31.601547 22007 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:31.603080 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.596808 21754 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 18 to 21, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 21 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:31.600708 21972 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.561765 29717 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:08:31.604058 21928 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:31.608613 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.587980 29550 raft_consensus.cc:3060] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:31.579730 29561 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.621992 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.634025 22591 consensus_queue.cc:260] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.639514 22591 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:31.640161 29743 log.cc:927] Time spent T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:08:31.641939 29739 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:08:31.611135 29729 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.644558 29725 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:31.644938 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 59091.5us
I20260709 13:08:31.646011 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.646931 29725 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:31.608949 29481 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.633520 21754 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 19 to 22, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 22 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.654407 21754 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 16 to 19, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.655997 21929 raft_consensus.cc:2368] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21.
I20260709 13:08:31.660022 28820 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.660375 21933 raft_consensus.cc:1275] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:08:31.650763 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.663955 29591 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:31.671876 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 92151.5us
I20260709 13:08:31.612601 29077 raft_consensus.cc:2804] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:08:31.682271 29561 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.689819 29481 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:31.692472 21898 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.692976 21898 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:31.693465 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 61629.3us
W20260709 13:08:31.696483 21898 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.696817 21824 leader_election.cc:304] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:31.697389 29534 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:31.697886 21898 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.698114 29534 raft_consensus.cc:2749] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:31.698369 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:31.627393 29153 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:08:31.698786 29696 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:31.699489 29554 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:31.700336 29554 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:31.707335 22002 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
W20260709 13:08:31.707604 21972 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.656812 21754 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:31.712402 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.710218 21824 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.715191 22591 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:31.717414 29550 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.711588 29077 raft_consensus.cc:697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:31.737797 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.737613 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.740342 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:31.744395 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.734047 29525 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:31.748896 22009 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
W20260709 13:08:31.748261 29688 consensus_meta.cc:287] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: Time spent flushing consensus metadata: real 0.536s	user 0.013s	sys 0.003s
I20260709 13:08:31.743465 29561 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:31.756211 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 46101.3us
I20260709 13:08:31.761089 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:31.742974 29534 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:08:31.762487 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 87465.3us
I20260709 13:08:31.763959 29024 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:31.764720 29024 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:31.753650 29532 raft_consensus.cc:2764] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
I20260709 13:08:31.767246 29024 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:31.758559 29721 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.767884 29721 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.759364 29525 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.776086 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.777773 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.791079 22586 raft_consensus.cc:3055] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:31.791478 22586 raft_consensus.cc:740] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:31.792273 22586 consensus_queue.cc:260] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.792501 29725 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:31.793303 22586 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:31.793507 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:31.794694 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:31.797379 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 64353.9us
I20260709 13:08:31.799510 29532 consensus_queue.cc:1048] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:31.799944 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.800813 29077 consensus_queue.cc:237] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.802811 22586 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:08:31.811436 22586 pending_rounds.cc:85] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 13
I20260709 13:08:31.813836 21754 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:31.814385 22586 pending_rounds.cc:107] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:08:31.815562 22586 raft_consensus.cc:2889] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:31.815343 21754 catalog_manager.cc:5697] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 9 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.818068 29669 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:31.819422 22001 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:31.820547 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:31.821374 29688 raft_consensus.cc:2764] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 13:08:31.822017 29688 consensus_queue.cc:1048] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:31.824554 29525 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:31.825814 21929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37776 (request call id 1364) took 1236 ms (client timeout 1502 ms). Trace:
W20260709 13:08:31.826239 21929 rpcz_store.cc:269] 0709 13:08:30.589577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.597574 (+  7997us) service_pool.cc:224] Handling call
0709 13:08:31.825593 (+1228019us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400206554. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707d32fa 00007f3b75794b17 00007f3b6ed0f6d4 00007f3b6ed0f662 00007f3b6ed0f591 00007f3b6ed0f50c 00007f3b6ed0e7c2 00007f3b6eb0a48c 00007f3b6eb098c6 00007f3b6eb0926f
0709 13:08:31.825736 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2403270528}
I20260709 13:08:31.805459 29550 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.807009 21933 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:08:31.808115 21825 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:31.831087 22008 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:31.837527 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cf1fd218a894ed39170b8db4ecc6ee8" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.846127 21932 raft_consensus.cc:2393] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 26: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:31.847630 21929 raft_consensus.cc:1240] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Rejecting Update request from peer 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:31.851974 21754 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 4 to 19, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:31.852007 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78076.2us
I20260709 13:08:31.854565 21824 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:31.858917 29721 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:31.858767 29565 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:31.838838 29525 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:31.860483 29525 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:31.861672 29525 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.863441 29651 consensus_queue.cc:1048] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:31.863449 21754 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:31.865723 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:31.847319 29549 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:08:31.883159 29153 raft_consensus.cc:2804] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:31.883800 29153 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.884255 29153 raft_consensus.cc:3060] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:31.885200 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 49091.9us
I20260709 13:08:31.891948 21929 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:08:31.894542 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:31.891187 29549 consensus_queue.cc:1243] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 34f3b83da84448bf8aac62b37a1486eb log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:08:31.907696 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.910055 21934 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:31.880497 29077 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:08:31.903144 29545 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:31.919751 22590 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 13:08:31.921794 21824 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:31.941465 29760 log.cc:927] Time spent T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:08:31.925204 29754 log.cc:927] Time spent T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.188s	user 0.002s	sys 0.002s
I20260709 13:08:31.945549 29642 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:31.907696 29669 consensus_queue.cc:1059] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
W20260709 13:08:31.955117 29758 log.cc:927] Time spent T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:08:31.961937 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:31.963945 29565 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:31.964437 29565 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.964807 29565 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:31.965373 29669 raft_consensus.cc:3055] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:31.965761 29669 raft_consensus.cc:740] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:31.966684 29669 consensus_queue.cc:260] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:31.967916 29669 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:31.915450 29591 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:08:31.969439 29591 consensus_queue.cc:1243] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer 821b3831b8a348c0826bd3a20a857f00 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:08:31.963553 21824 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:31.982494 21898 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:31.997512 21899 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:31.980667 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:31.990144 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:32.003226 21926 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 25: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:31.999399 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:31.998577 21824 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:32.006940 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:31.980223 29759 log.cc:927] Time spent T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:08:32.007761 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:32.002012 21929 pending_rounds.cc:85] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 12
I20260709 13:08:32.010877 21929 pending_rounds.cc:107] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:32.011150 21929 raft_consensus.cc:2889] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:32.015578 29642 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.016196 29642 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.013609 29551 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:32.017935 29551 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.018285 29551 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:32.018333 29642 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.021353 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:32.020958 21754 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 12 to 18, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 18 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:32.032809 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.972975 29565 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.037551 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.038264 21754 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 12 to 17, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 17 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:32.039278 29565 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.039615 22591 pending_rounds.cc:85] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Aborting all ops after (but not including) 15
I20260709 13:08:32.040037 22591 pending_rounds.cc:107] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:08:32.040340 22591 raft_consensus.cc:2889] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:32.041312 22001 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 LEADER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 27. Current term is 28. Ops: []
W20260709 13:08:32.025635 21898 leader_election.cc:336] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.042552 21898 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.043545 22006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:31.999837 29549 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:32.003727 29550 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:32.040148 21754 catalog_manager.cc:5697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 7 to 17, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:32.045843 29545 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:08:32.066679 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:32.078336 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.063416 29717 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.084372 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:32.085608 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:32.086615 22585 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 25: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:32.076448 29766 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 13:08:32.060325 29077 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 13:08:32.077625 29553 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:32.176885 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 163020us
W20260709 13:08:32.181941 21898 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.184350 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.145285 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:32.146442 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:32.147579 22585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:32.190773 29773 log.cc:927] Time spent T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:08:32.191260 29153 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.192245 29565 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.198199 22556 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.203205 29551 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:32.210036 29757 log.cc:927] Time spent T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.346s	user 0.002s	sys 0.000s
I20260709 13:08:32.212756 22588 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:08:32.140220 21934 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:32.219843 22007 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:32.224668 22001 raft_consensus.cc:1240] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 22. Current term is 23. Ops: []
W20260709 13:08:32.240520 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 53034.2us
I20260709 13:08:32.240690 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:32.241333 22009 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:32.158325 29717 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.220117 29591 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:32.247907 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 66009.6us
W20260709 13:08:32.249146 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 102772us
I20260709 13:08:32.226998 29651 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.226919 29077 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.258224 21935 raft_consensus.cc:1240] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 13:08:32.260959 21898 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.249112 29563 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 13:08:32.262915 29551 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.263437 29565 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:32.250625 29550 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:32.265599 29565 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:32.265661 29077 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.267892 29077 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:32.265621 29591 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:32.270627 29591 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:32.271521 21926 raft_consensus.cc:3060] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:32.271731 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.278198 22588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:32.290831 29651 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.294395 29153 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.294449 29481 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:32.295428 29481 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.298097 29481 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.298803 29550 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:32.299705 29550 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:32.304685 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 78664.2us
I20260709 13:08:32.305670 21824 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:32.313326 21930 raft_consensus.cc:1275] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:08:32.317850 22009 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:32.318076 21926 raft_consensus.cc:1275] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 13:08:32.329820 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.333843 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:32.334182 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.334306 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:32.339689 29550 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.341466 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.257584 29554 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:08:32.346469 21928 raft_consensus.cc:2368] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21.
I20260709 13:08:32.336231 29642 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:32.374550 22587 raft_consensus.cc:2410] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 13:08:32.368657 29525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.012s
I20260709 13:08:32.361778 29545 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
W20260709 13:08:32.408530 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45838.1us
I20260709 13:08:32.381721 22556 leader_election.cc:400] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Vote denied by peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191) with higher term. Message: Invalid argument: T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb for earlier term 20. Current term is 21.
I20260709 13:08:32.345978 29688 consensus_queue.cc:1048] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:08:32.348289 29549 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:32.409706 21824 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:32.367695 29778 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.379971 29525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.008s	sys 0.012s
I20260709 13:08:32.363834 29568 consensus_queue.cc:1048] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.016s
W20260709 13:08:32.434209 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 61835.3us
I20260709 13:08:32.431519 22585 raft_consensus.cc:2468] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:32.441865 21900 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:32.454914 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.457391 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.456336 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:32.414016 29696 consensus_queue.cc:1059] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:08:32.429301 29778 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:32.462394 29696 raft_consensus.cc:3055] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:32.463001 29696 raft_consensus.cc:740] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:32.463713 29778 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:32.464372 29696 consensus_queue.cc:260] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.466928 29793 raft_consensus.cc:2804] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:32.458081 21928 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:32.467514 29793 raft_consensus.cc:697] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:32.468493 29696 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:32.469291 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:32.470408 20746 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 21885)
I20260709 13:08:32.443557 29651 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:32.453824 29786 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:32.458240 29563 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:32.456292 28820 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:32.488077 29563 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:32.494668 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:32.502287 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 125661us
I20260709 13:08:32.485034 29786 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:32.506889 29786 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.494765 29780 consensus_queue.cc:1059] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.143s
I20260709 13:08:32.510445 29780 consensus_queue.cc:1243] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer 7d07aa0dcdd1484ab87925c9a4604b81 log is divergent from this leader: its last log entry 28.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:08:32.501251 29563 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:32.510972 29561 raft_consensus.cc:3055] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:08:32.497363 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:32.511739 29561 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:32.511960 29545 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:32.512817 29561 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.513931 29561 raft_consensus.cc:3060] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:08:32.517650 29481 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:32.518710 22559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.533869 29717 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.534974 29549 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.537134 22593 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:08:32.531013 28820 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:32.540668 29795 log.cc:927] Time spent T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.081s	user 0.002s	sys 0.001s
I20260709 13:08:32.541901 28820 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:32.542304 21898 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:32.542733 29797 log.cc:927] Time spent T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.082s	user 0.003s	sys 0.000s
I20260709 13:08:32.542810 21898 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.544173 29721 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:32.551317 29791 log.cc:927] Time spent T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:08:32.552493 29649 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:08:32.468554 29793 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:32.574210 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.575953 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:32.580263 21898 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.580859 21898 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.585029 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:32.585949 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:32.586997 21898 leader_election.cc:336] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.587431 21898 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:32.595348 21898 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.588153 29554 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:32.604815 21898 leader_election.cc:336] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.610057 21898 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:32.611275 21898 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:32.616096 21898 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.599912 29550 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:32.627683 21898 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.611502 29721 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:32.614531 29717 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.632279 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:32.633073 21929 raft_consensus.cc:1275] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:08:32.652858 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:32.655961 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 49234.1us
W20260709 13:08:32.659978 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:32.660303 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.651757 29551 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:32.669471 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.651762 29717 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.674345 29717 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.677507 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:32.679858 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.681175 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.682394 21930 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:32.682313 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10076817fc0f46b4943de726ac0b508c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.688776 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:32.689517 21931 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:32.714619 21927 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:32.693938 21930 raft_consensus.cc:1240] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Rejecting Update request from peer 8f3b35a35257416db3eaaf364cf6b52b for earlier term 27. Current term is 28. Ops: []
I20260709 13:08:32.695231 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.722254 22005 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:32.713781 29717 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:32.693445 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:32.723358 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:32.713697 29787 log.cc:927] Time spent T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.249s	user 0.000s	sys 0.003s
I20260709 13:08:32.728101 29717 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.730304 29792 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:32.730822 29792 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.731179 29792 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:32.726317 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.721704 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:32.734961 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.756476 21931 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:32.761262 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.757507 29782 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:32.757256 29077 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:32.762004 29780 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:32.772692 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 112397us
I20260709 13:08:32.765184 29551 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:32.776494 21972 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.776824 21972 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:32.778326 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:32.779142 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:32.774672 29782 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.781519 29049 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:32.780586 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.782595 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:32.783771 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.784173 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:32.784904 29802 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 13:08:32.786592 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:32.786931 29563 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:32.779186 29551 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.790831 29568 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:32.795452 29551 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.781599 29782 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:32.793133 29049 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:32.798135 29049 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.798502 29049 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:32.802037 29717 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.803356 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.804121 22005 raft_consensus.cc:2393] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:08:32.804752 29805 log.cc:927] Time spent T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:08:32.777137 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 91295us
I20260709 13:08:32.807341 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:32.814157 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.815956 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.817206 29077 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:32.817483 29563 consensus_queue.cc:1048] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:08:32.824710 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.825498 29780 consensus_queue.cc:1059] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:32.826570 29696 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:32.826758 29780 raft_consensus.cc:3055] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:32.827246 29780 raft_consensus.cc:740] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:32.828117 29780 consensus_queue.cc:260] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.831727 29153 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:32.833588 21823 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.834566 29780 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:32.841676 29717 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:32.842929 29813 log.cc:927] Time spent T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:08:32.846411 29534 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:32.848240 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.848754 21898 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:32.850262 21898 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.850706 21898 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.850853 21823 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:32.851758 21898 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.852187 21898 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.852263 29561 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:32.853310 29551 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:32.853497 29717 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:32.854084 29551 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:32.854135 29717 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.854956 29551 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:32.835100 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:32.856819 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:32.856943 29153 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:32.857862 29153 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:32.860251 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:32.861315 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 50725.4us
I20260709 13:08:32.857362 29717 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.879914 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.845979 29792 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.893386 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:32.896718 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:32.904559 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 52752us
I20260709 13:08:32.908834 29786 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:08:32.915195 28820 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:32.896884 29534 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:32.933154 29565 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:32.950431 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:32.951164 22005 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:32.942554 29792 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.957901 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.958246 21934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.958622 21926 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:32.960356 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:32.960652 21930 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:08:32.963143 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 40a1bb4717e14331a67545a343ce1d87: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.944944 29782 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:32.972898 29782 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:32.932521 29049 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:32.991696 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 50967.9us
W20260709 13:08:32.992980 21898 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:32.995217 21898 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:32.999503 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:33.000490 22008 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f3b35a35257416db3eaaf364cf6b52b in current term 25: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
W20260709 13:08:33.001191 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 182941us
I20260709 13:08:32.996230 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:32.975293 29049 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:33.006690 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:33.007297 21928 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:32.996397 29818 log.cc:927] Time spent T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:33.033435 21928 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 25.
I20260709 13:08:33.041579 21934 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:08:33.004686 29551 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:33.045878 22000 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:33.036545 22005 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:33.052908 21824 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
W20260709 13:08:33.075856 22558 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.052928 21823 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:33.144563 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:33.145659 21928 raft_consensus.cc:2393] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 25: Already voted for candidate 8f3b35a35257416db3eaaf364cf6b52b in this term.
W20260709 13:08:33.149583 21823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 96707.9us
I20260709 13:08:33.150851 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:33.122008 29545 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:08:33.167402 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 166341us
I20260709 13:08:33.162884 29778 raft_consensus.cc:493] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.148027 29642 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:33.189178 29642 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:33.194269 29642 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:33.195212 29642 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:33.189342 29778 raft_consensus.cc:515] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:33.180395 29153 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:33.179454 29545 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:33.212498 29305 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:33.247951 29551 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:33.249898 21932 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:33.256174 29545 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:33.259207 21932 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:33.269809 22005 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:08:33.280247 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.291186 22005 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:08:33.300122 29839 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.300946 29839 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:33.301277 29545 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:33.280247 29778 leader_election.cc:290] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:33.311848 29591 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:33.312705 29591 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:33.313417 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 188331us
W20260709 13:08:33.314314 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:33.314796 21972 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.315128 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.316381 29649 raft_consensus.cc:2749] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:33.316746 22593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:33.317534 22593 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:33.316339 29591 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:33.319212 29049 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:33.319319 29153 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:33.319761 29049 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:33.322980 29481 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:33.323642 29481 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:33.330204 29839 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:33.331338 29831 log.cc:927] Time spent T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:08:33.331764 22592 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:33.333273 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.361779 21929 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:33.365568 29550 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:33.366382 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:33.372700 29077 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:33.373275 29077 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:33.373824 29077 raft_consensus.cc:3060] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:33.375267 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 309ab90abff84c36a30bd5a2a0363d2c: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.380956 21823 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:33.382360 29153 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:33.382880 29153 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:33.383672 22002 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:08:33.383966 29153 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:33.380988 29696 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:08:33.389135 29696 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:33.397209 29553 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:08:33.397945 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 0103ac417f394657ac6cbe827df3b844: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.403628 22002 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:33.407531 22005 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:33.410326 29561 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:33.417928 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:33.421113 22584 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:08:33.423077 22587 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:08:33.441534 29049 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:33.445516 29077 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:33.460448 21972 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.461149 21972 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.462222 29077 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:33.462489 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 294306us
I20260709 13:08:33.463938 29049 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:33.471112 29696 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:08:33.471755 20746 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 21885)
I20260709 13:08:33.479310 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:33.487299 29481 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:33.493341 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:33.514277 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 93428.6us
W20260709 13:08:33.514425 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 50191.2us
I20260709 13:08:33.515798 29843 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.516429 29843 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:33.518800 29843 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:33.520511 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.521067 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.521829 29843 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:33.523635 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.524806 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.525959 29843 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:33.527554 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:33.531376 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91310e68464d64aa8324092182518f" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:33.462633 21972 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.535502 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:33.536204 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 73593.1us
W20260709 13:08:33.536854 21972 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.551298 29696 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:33.557032 29696 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:33.610312 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.620801 21823 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:33.622048 29153 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:33.634109 21975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:33.615952 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:33.638334 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 100575us
I20260709 13:08:33.646901 21823 leader_election.cc:304] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:33.635345 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62558.9us
I20260709 13:08:33.655040 22590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:33.634949 29717 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:33.659754 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 266642us
I20260709 13:08:33.659904 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.670408 21931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:33.671725 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:33.674950 21931 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:33.657421 29696 raft_consensus.cc:2749] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:33.682317 29305 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.685652 29305 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:33.687974 29305 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:33.690382 21898 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.690527 21823 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:33.691102 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:33.692941 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.695096 21931 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:33.709244 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:33.715474 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:33.722087 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:33.740743 21930 raft_consensus.cc:2393] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 21: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:33.706837 29839 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:33.670894 29717 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:33.743129 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 60179.1us
W20260709 13:08:33.743556 21974 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc33f94111684d68a49eb643f8be2203. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:33.740695 29153 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.768633 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.775791 22590 raft_consensus.cc:2468] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:33.782558 21824 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:33.773489 29153 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:33.788166 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:33.795142 21972 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:33.795571 22007 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:08:33.795342 21900 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.795913 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:33.787364 29839 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:33.804626 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 51478.2us
I20260709 13:08:33.810324 21974 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:33.815969 29649 raft_consensus.cc:2804] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:33.818975 29696 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:33.807628 29793 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:33.853468 29859 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:33.854496 29717 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:33.855304 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.808542 29860 raft_consensus.cc:2749] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:08:33.860854 22008 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 13:08:33.861260 22008 pending_rounds.cc:85] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting all ops after (but not including) 13
I20260709 13:08:33.861506 22008 pending_rounds.cc:107] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 13:08:33.861748 22008 raft_consensus.cc:2889] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:33.871557 29792 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:33.874358 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:33.880621 29153 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:33.883488 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 69611.1us
I20260709 13:08:33.889178 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:33.895658 21825 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:33.895292 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 144641us
W20260709 13:08:33.908939 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:33.923473 21898 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:33.967851 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48690us
I20260709 13:08:33.969471 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:33.972889 21825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 54922.1us
I20260709 13:08:33.893586 29649 raft_consensus.cc:697] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:33.974402 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:33.984222 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 168008us
I20260709 13:08:33.984638 21935 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:08:33.969875 29717 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:34.022668 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 188131us
I20260709 13:08:33.989205 29789 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.046098 29783 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:34.064843 29860 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:34.068766 29860 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.071238 29860 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:34.012236 29717 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:34.043291 29789 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.126665 29649 consensus_queue.cc:237] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:34.189561 29870 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:08:34.196147 29717 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:34.197778 29789 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.206050 29789 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:34.223865 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 253034us
I20260709 13:08:34.221596 29878 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:34.231156 29878 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.210227 29789 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.233428 29878 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.233510 29789 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.244798 21935 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:08:34.254061 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.254921 22008 raft_consensus.cc:2393] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 23: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:34.293776 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.294766 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.297217 21823 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:34.295644 21930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.308753 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.311074 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:34.314267 29783 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:08:34.315982 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.316591 22008 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:34.318926 21757 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 23, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 23 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.320892 21757 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 21, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.327382 21757 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 17, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.336161 21824 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:34.305989 21930 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:34.351523 21757 catalog_manager.cc:5697] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.294651 29545 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:34.357941 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:34.364302 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:34.353993 29561 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:34.363863 21757 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 14 to 19, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 19 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.367624 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:34.365808 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 341155us
W20260709 13:08:34.374881 21754 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 310) took 26025 ms (client timeout 15000 ms). Trace:
W20260709 13:08:34.371487 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 386179us
I20260709 13:08:34.381981 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.382788 21926 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
W20260709 13:08:34.375000 21898 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:34.385766 21757 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.352713 29783 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:08:34.390901 21898 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:34.364032 29545 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.371448 29561 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:34.365899 29844 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:08:34.385151 29846 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:34.381364 29549 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:34.388564 29842 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:34.375409 21754 rpcz_store.cc:269] 0709 13:08:08.349188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.864705 (+2515517us) service_pool.cc:224] Handling call
0709 13:08:34.374814 (+23510109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:32.634815 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:32.770956 (+136141us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:32.771029 (+    73us) write_op.cc:435] Acquired schema lock
0709 13:08:32.771053 (+    24us) tablet.cc:662] Decoding operations
0709 13:08:32.774014 (+  2961us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:32.774113 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:32.774137 (+    24us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:08:32.802033 (+ 27896us) tablet.cc:701] Row locks acquired
0709 13:08:32.802088 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:08:32.802286 (+   198us) write_op.cc:270] Start()
0709 13:08:32.802432 (+   146us) write_op.cc:276] Timestamp: P: 1783602512802213 usec, L: 0
0709 13:08:32.802447 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:08:32.803202 (+   755us) log.cc:844] Serialized 5571 byte log entry
0709 13:08:33.049151 (+245949us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b1800044620. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765b75b1 00007f3b6d6e23f3 00007f3b6d6e47c1 00007f3b6d6e4779 00007f3b6d6e4709 00007f3b6d6e46d1 00007f3b6d6e39cd 000000000038f354 000000000038f199 00007f3b6d6c583a
0709 13:08:33.064688 (+ 15537us) op_driver.cc:464] REPLICATION: finished
0709 13:08:33.247386 (+182698us) write_op.cc:301] APPLY: starting
0709 13:08:33.247494 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:33.337641 (+ 90147us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:33.337684 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:33.347132 (+  9448us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:33.347449 (+   317us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:08:33.347496 (+    47us) write_op.cc:312] APPLY: finished
0709 13:08:33.348226 (+   730us) log.cc:844] Serialized 129 byte log entry
0709 13:08:33.349257 (+  1031us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:33.349400 (+   143us) write_op.cc:454] Released schema lock
0709 13:08:33.349450 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:33.349848 (+   398us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:33.349874 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":416934,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":182429,"apply.run_cpu_time_us":21090,"apply.run_wall_time_us":102449,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62731,"num_ops":14,"prepare.queue_time_us":176190,"prepare.run_cpu_time_us":16821,"prepare.run_wall_time_us":315244,"raft.queue_time_us":8413,"raft.run_cpu_time_us":36011,"raft.run_wall_time_us":153393,"replication_time_us":262068,"spinlock_wait_cycles":31934208,"thread_start_us":339776,"threads_started":4,"wal-append.queue_time_us":96934,"wal-append.run_cpu_time_us":7677,"wal-append.run_wall_time_us":1301927}]]}
I20260709 13:08:34.413966 21757 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 20, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 20 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.420514 21757 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 23 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.406328 29024 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:34.406675 29561 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:34.429534 21757 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 20, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:34.437764 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:34.449972 21757 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 20, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 20 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:34.454873 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:34.460144 21972 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:34.460906 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:34.462342 21972 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.433952 29829 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:34.436321 29842 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:34.472203 20746 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 21885)
I20260709 13:08:34.478631 21757 catalog_manager.cc:5697] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 22 to 25, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 25 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.485728 29842 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:34.486326 21757 catalog_manager.cc:5697] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.487962 21757 catalog_manager.cc:5697] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 20 to 23, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 23 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.464421 29591 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:34.524528 21757 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:34.537519 29887 log.cc:927] Time spent T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 13:08:34.541067 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:34.555003 21757 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 18 to 22, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.481725 29829 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:34.586189 21757 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 21, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 21 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:34.586973 21898 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 149 similar messages]
I20260709 13:08:34.597116 21757 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 2 to 12, term changed from 18 to 23, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129), VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130), VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 23 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.608014 21757 catalog_manager.cc:5697] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 10 to 20, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 20 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.619976 21757 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 17, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.626366 29024 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:34.631999 29842 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:34.583033 29481 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:34.633131 29481 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.636204 29481 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.638379 29545 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.635222 21757 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.659282 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.598053 29872 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:34.677788 29481 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:34.678422 29481 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:34.682876 21898 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:34.683674 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:34.687119 21757 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 18 to 19, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:34.682351 29896 ts_tablet_manager.cc:933] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.698364 29829 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.668118 29872 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:34.704172 21757 catalog_manager.cc:5697] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 18, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:34.703121 21898 leader_election.cc:336] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:34.712423 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.713481 21929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.714077 21929 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 22.
I20260709 13:08:34.685458 29481 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.714927 21823 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:34.716022 21927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:34.718405 21757 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.722890 21927 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:34.719146 29778 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:34.729817 29778 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:34.753273 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:34.765249 29024 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.765877 29024 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:34.770375 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 184667us
I20260709 13:08:34.774077 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.785917 22584 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:34.792948 29872 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:34.791224 29782 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:34.797575 29782 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:34.799742 29782 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:34.795308 29897 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:34.806972 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.809406 29561 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.750216 29778 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:34.811559 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.813760 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:34.820227 29901 log.cc:927] Time spent T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:08:34.824044 21899 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:34.825448 29554 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:08:34.827410 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:34.829064 21899 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:34.831257 22556 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:34.833309 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.833946 22002 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 21.
W20260709 13:08:34.835688 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 372495us
I20260709 13:08:34.845070 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:34.850476 22002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.861303 21933 raft_consensus.cc:1275] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:34.862666 21900 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:34.853112 29878 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:34.841501 29828 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:34.865255 29828 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:34.868620 29828 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:34.863988 29902 log.cc:927] Time spent T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:08:34.874953 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:34.890949 29565 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:34.872815 29852 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:34.894783 21824 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:34.895027 29778 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:34.896195 29836 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:34.896610 29561 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.897280 29778 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.877805 29896 tablet_copy_client.cc:323] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:08:34.898692 29024 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:34.899922 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet bc33f94111684d68a49eb643f8be2203 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:08:34.900591 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet bc33f94111684d68a49eb643f8be2203 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-bc33f94111684d68a49eb643f8be2203
I20260709 13:08:34.910513 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:34.911360 21928 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:34.916260 29828 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:34.930275 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:34.930918 22008 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:08:34.930728 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51625us
I20260709 13:08:34.915416 29873 raft_consensus.cc:2955] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:34.918190 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:34.936223 21935 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:34.920068 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 62804.6us
I20260709 13:08:34.922776 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:34.926924 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f466d510c1b46d481ab83686b150d61" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:34.933807 21928 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:34.934777 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:34.946166 22004 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:34.946794 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:34.947290 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:34.950497 29828 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:34.959626 22004 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 24.
I20260709 13:08:34.961396 21900 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:34.963510 21823 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:34.963980 22008 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:34.974917 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:34.973603 29884 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:34.980845 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:34.969923 29553 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:34.980481 29903 log.cc:927] Time spent T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 13:08:34.974071 29778 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:34.997789 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:34.993952 21926 raft_consensus.cc:2955] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:35.004012 29846 raft_consensus.cc:2749] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:35.000273 29717 raft_consensus.cc:493] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.006273 29717 raft_consensus.cc:515] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:35.000206 29789 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.010725 29789 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:35.015529 21824 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b; no voters: 
I20260709 13:08:34.989982 29859 consensus_queue.cc:1048] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:08:35.016485 29778 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:35.020896 29778 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:35.021981 29844 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.022507 29844 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:35.024621 29844 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:35.034929 21935 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f3b35a35257416db3eaaf364cf6b52b in term 23.
I20260709 13:08:35.038846 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:35.034969 29153 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.040683 29153 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:35.043285 29153 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:35.045989 29789 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:35.047416 29717 leader_election.cc:290] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:35.053216 29782 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:35.053920 29782 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Becoming Leader. State: Replica: 8f3b35a35257416db3eaaf364cf6b52b, State: Running, Role: LEADER
I20260709 13:08:35.054855 29782 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:35.058954 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:35.069110 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:35.074693 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:35.079965 29873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51498 (request call id 1) took 1678 ms. Trace:
W20260709 13:08:35.084676 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.088537 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a76564a0ee84b74a3af01c79300d4b7" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:35.087138 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:35.106556 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "634269b557d04cbe87cfab961aaf3f91" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:35.123620 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:35.135637 21824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 47819.4us
I20260709 13:08:35.136869 21824 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
W20260709 13:08:35.145745 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.140628 29782 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:35.164749 21823 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:35.080780 29873 rpcz_store.cc:276] 0709 13:08:33.401376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.401526 (+   150us) service_pool.cc:224] Handling call
0709 13:08:33.407247 (+  5721us) log.cc:844] Serialized 506 byte log entry
0709 13:08:35.079817 (+1672570us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":741,"raft.run_cpu_time_us":1977,"raft.run_wall_time_us":2524,"spinlock_wait_cycles":72832,"thread_start_us":41655,"threads_started":1,"wal-append.queue_time_us":37559,"wal-append.run_cpu_time_us":1843,"wal-append.run_wall_time_us":1010806}
I20260709 13:08:35.166311 21823 leader_election.cc:304] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:35.167351 29844 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:35.170104 29786 raft_consensus.cc:2749] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:35.171325 22019 tablet_copy_source_session.cc:215] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:35.174635 21823 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:35.176049 29844 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:35.180130 29896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc33f94111684d68a49eb643f8be2203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:35.220897 22584 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Rejecting Update request from peer 7d07aa0dcdd1484ab87925c9a4604b81 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:35.226886 29873 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
W20260709 13:08:35.237669 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:35.239214 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.238372 29859 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:35.245710 29859 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:35.246595 29859 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:35.248075 29859 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:35.268127 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.273890 29153 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.274529 29153 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:35.280735 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:35.283259 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.284570 29153 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:35.286298 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecc321cb1e14e99a3838b646f82e71a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:35.300154 21825 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:35.303390 29153 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:35.300309 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:35.312825 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.357342 29896 tablet_copy_client.cc:806] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:35.358054 29896 tablet_copy_client.cc:670] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:35.373118 29896 tablet_copy_client.cc:538] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:35.438465 29153 raft_consensus.cc:493] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34f3b83da84448bf8aac62b37a1486eb)
I20260709 13:08:35.439152 29153 raft_consensus.cc:515] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:35.443326 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:35.447995 29153 leader_election.cc:290] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:35.449437 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cf25498fcd4e4594d809b889d236d3" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:35.460711 21935 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:08:35.462430 21825 leader_election.cc:304] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:35.462864 29696 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:35.463693 29844 raft_consensus.cc:2749] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:35.474116 21757 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 196) took 7815 ms. Trace:
W20260709 13:08:35.479219 20746 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 21885)
I20260709 13:08:35.474557 21757 rpcz_store.cc:276] 0709 13:08:27.658466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.658629 (+   163us) service_pool.cc:224] Handling call
0709 13:08:35.474007 (+7815378us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:35.026830 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:35.027411 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:35.027445 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:08:35.027466 (+    21us) tablet.cc:662] Decoding operations
0709 13:08:35.029239 (+  1773us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:35.029299 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:35.029331 (+    32us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:08:35.031044 (+  1713us) tablet.cc:701] Row locks acquired
0709 13:08:35.031065 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:08:35.031238 (+   173us) write_op.cc:270] Start()
0709 13:08:35.061356 (+ 30118us) write_op.cc:276] Timestamp: P: 1783602515031183 usec, L: 0
0709 13:08:35.061384 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:08:35.098772 (+ 37388us) log.cc:844] Serialized 9211 byte log entry
0709 13:08:35.260513 (+161741us) op_driver.cc:464] REPLICATION: finished
0709 13:08:35.349587 (+ 89074us) write_op.cc:301] APPLY: starting
0709 13:08:35.352880 (+  3293us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:35.404888 (+ 52008us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:35.421140 (+ 16252us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:35.449246 (+ 28106us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:35.449808 (+   562us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:08:35.449853 (+    45us) write_op.cc:312] APPLY: finished
0709 13:08:35.450265 (+   412us) log.cc:844] Serialized 204 byte log entry
0709 13:08:35.451034 (+   769us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.451225 (+   191us) write_op.cc:454] Released schema lock
0709 13:08:35.451287 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:35.464043 (+ 12756us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.464074 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":591950,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":88789,"apply.run_cpu_time_us":29204,"apply.run_wall_time_us":114445,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100405,"num_ops":23,"prepare.queue_time_us":118942,"prepare.run_cpu_time_us":28386,"prepare.run_wall_time_us":112598,"raft.queue_time_us":97826,"raft.run_cpu_time_us":70477,"raft.run_wall_time_us":78072,"replication_time_us":198980,"thread_start_us":259951,"threads_started":4,"wal-append.queue_time_us":42319}]]}
I20260709 13:08:35.507138 29896 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:08:35.509135 22001 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:35.534574 29153 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:35.513473 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:35.521996 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.573964 21756 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 9 to 17, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 17 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.588833 21756 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 12 to 22, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 22 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.577764 21935 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:35.592319 29565 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:35.576763 22001 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 8f3b35a35257416db3eaaf364cf6b52b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:08:35.597944 29565 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:35.605938 21756 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 10 to 15, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 15 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.614697 21756 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 11 to 16, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 16 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:35.624058 21756 catalog_manager.cc:5697] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 1 to 18. New cstate: current_term: 18 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:35.627991 21756 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 12 to 21, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.744191 29789 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:35.744793 29789 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:35.755712 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:35.756004 29789 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:35.756161 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:35.758117 21824 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:35.759549 29789 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:35.782599 21749 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 16 to 17, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.784590 21749 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 21 to 22, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.787381 21749 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:35.795645 21749 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 17 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:35.797510 21749 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 17 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 18 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:35.790679 21756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 291) took 31556 ms (client timeout 15000 ms). Trace:
W20260709 13:08:35.804673 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:35.800369 21756 rpcz_store.cc:269] 0709 13:08:04.234480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.234630 (+   150us) service_pool.cc:224] Handling call
0709 13:08:05.318895 (+1084265us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:35.790569 (+30471674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:35.638585 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:35.639074 (+   489us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:35.639117 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:08:35.639139 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:35.639843 (+   704us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:35.639898 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:35.639925 (+    27us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:08:35.644425 (+  4500us) tablet.cc:701] Row locks acquired
0709 13:08:35.644445 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:08:35.644587 (+   142us) write_op.cc:270] Start()
0709 13:08:35.644695 (+   108us) write_op.cc:276] Timestamp: P: 1783602515644541 usec, L: 0
0709 13:08:35.644718 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:35.645381 (+   663us) log.cc:844] Serialized 2482 byte log entry
0709 13:08:35.647863 (+  2482us) op_driver.cc:464] REPLICATION: finished
0709 13:08:35.648595 (+   732us) write_op.cc:301] APPLY: starting
0709 13:08:35.648678 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:35.652368 (+  3690us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:35.652462 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:35.692946 (+ 40484us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:35.693386 (+   440us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:08:35.693438 (+    52us) write_op.cc:312] APPLY: finished
0709 13:08:35.693711 (+   273us) log.cc:844] Serialized 65 byte log entry
0709 13:08:35.694480 (+   769us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.694599 (+   119us) write_op.cc:454] Released schema lock
0709 13:08:35.694650 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:35.695230 (+   580us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.695258 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":49,"spinlock_wait_cycles":1056820992,"child_traces":[["op",{"apply.queue_time_us":494,"apply.run_cpu_time_us":9725,"apply.run_wall_time_us":46585,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25116,"num_ops":6,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":3321,"prepare.run_wall_time_us":7122,"replication_time_us":2958}]]}
I20260709 13:08:35.811225 21749 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 17 to 18, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 18 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.817406 21749 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 20 to 22, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.846717 21749 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 19 to 20, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 20 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.856920 21749 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 16 to 17, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 17 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:35.885191 29896 log.cc:927] Time spent T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Append to log took a long time: real 0.057s	user 0.004s	sys 0.000s
I20260709 13:08:35.890291 21749 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 5 to 19. New cstate: current_term: 19 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.947285 29896 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:35.948915 29896 tablet_bootstrap.cc:492] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:08:35.949795 29896 ts_tablet_manager.cc:1403] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.443s	user 0.045s	sys 0.012s
I20260709 13:08:35.967993 29896 raft_consensus.cc:359] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:35.978644 29896 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
W20260709 13:08:35.985553 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:35.979703 29896 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:35.989542 29896 ts_tablet_manager.cc:1434] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 13:08:35.996054 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-bc33f94111684d68a49eb643f8be2203 received from {username='slave'} at 127.0.0.1:32904
I20260709 13:08:35.996601 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-bc33f94111684d68a49eb643f8be2203 on tablet bc33f94111684d68a49eb643f8be2203 with peer 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:36.032719 21749 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 286) took 20946 ms (client timeout 15000 ms). Trace:
I20260709 13:08:36.033587 21758 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 9 to 13, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 13 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:36.035617 21758 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
W20260709 13:08:36.045079 21749 rpcz_store.cc:269] 0709 13:08:15.086749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.761281 (+6674532us) service_pool.cc:224] Handling call
0709 13:08:36.032621 (+14271340us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:35.935826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:35.936315 (+   489us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:35.936351 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:08:35.936368 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:35.937295 (+   927us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:35.937350 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:35.937378 (+    28us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:08:35.938101 (+   723us) tablet.cc:701] Row locks acquired
0709 13:08:35.938119 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:08:35.938261 (+   142us) write_op.cc:270] Start()
0709 13:08:35.938363 (+   102us) write_op.cc:276] Timestamp: P: 1783602515938217 usec, L: 0
0709 13:08:35.938378 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:08:35.938967 (+   589us) log.cc:844] Serialized 4019 byte log entry
0709 13:08:35.941234 (+  2267us) op_driver.cc:464] REPLICATION: finished
0709 13:08:35.941671 (+   437us) write_op.cc:301] APPLY: starting
0709 13:08:35.941744 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:35.946462 (+  4718us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:35.946561 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:35.958344 (+ 11783us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:35.958492 (+   148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:08:35.959834 (+  1342us) write_op.cc:312] APPLY: finished
0709 13:08:35.961617 (+  1783us) log.cc:844] Serialized 97 byte log entry
0709 13:08:35.968901 (+  7284us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.975970 (+  7069us) write_op.cc:454] Released schema lock
0709 13:08:35.976031 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:35.977469 (+  1438us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.978228 (+   759us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5785,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":241,"apply.run_cpu_time_us":11839,"apply.run_wall_time_us":34689,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41860,"num_ops":10,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":3446,"prepare.run_wall_time_us":13326,"replication_time_us":2755}]]}
I20260709 13:08:36.044902 21758 catalog_manager.cc:5697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 7 to 12, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 12 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.059144 29153 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.059752 29153 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.061691 29153 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.064671 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:36.066088 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b8c4da9f5144e2b8007c8ffc563a584" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:36.072404 21825 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:36.074132 29789 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:36.136796 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): ChangeConfig:REMOVE_PEER RPC failed for tablet 2167f46d5e1049268b9ce66dbf46f781: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:36.169901 21744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4458afcc40954b359ea857cddbfdd681 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 34f3b83da84448bf8aac62b37a1486eb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.160230 21755 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 7 to 10, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 10 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:36.178283 21758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 290) took 61371 ms (client timeout 15000 ms). Trace:
W20260709 13:08:36.180136 21758 rpcz_store.cc:269] 0709 13:07:34.806911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.807334 (+   423us) service_pool.cc:224] Handling call
0709 13:07:35.597302 (+789968us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b5400011210. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6ab0b2 00007f3b76624a89 00007f3b765bcda6 00007f3b7658e088 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc
0709 13:08:36.178182 (+60580880us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:36.061283 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:36.061842 (+   559us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:36.061876 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:08:36.061890 (+    14us) tablet.cc:662] Decoding operations
0709 13:08:36.062428 (+   538us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:36.062488 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:36.062517 (+    29us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:08:36.062870 (+   353us) tablet.cc:701] Row locks acquired
0709 13:08:36.062890 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:08:36.063042 (+   152us) write_op.cc:270] Start()
0709 13:08:36.063158 (+   116us) write_op.cc:276] Timestamp: P: 1783602516062990 usec, L: 0
0709 13:08:36.063181 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:36.066273 (+  3092us) log.cc:844] Serialized 1317 byte log entry
0709 13:08:36.073843 (+  7570us) op_driver.cc:464] REPLICATION: finished
0709 13:08:36.074359 (+   516us) write_op.cc:301] APPLY: starting
0709 13:08:36.074438 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:36.076118 (+  1680us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:36.076200 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:36.077939 (+  1739us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:36.078098 (+   159us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:08:36.078141 (+    43us) write_op.cc:312] APPLY: finished
0709 13:08:36.078331 (+   190us) log.cc:844] Serialized 41 byte log entry
0709 13:08:36.078483 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.078587 (+   104us) write_op.cc:454] Released schema lock
0709 13:08:36.078634 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:36.078975 (+   341us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.079008 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12307,"reported_tablets":49,"spinlock_wait_cycles":49928064,"child_traces":[["op",{"apply.queue_time_us":269,"apply.run_cpu_time_us":4571,"apply.run_wall_time_us":4571,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12558,"num_ops":3,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":2949,"prepare.run_wall_time_us":5280,"replication_time_us":10522}]]}
I20260709 13:08:36.203508 29789 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.205371 29789 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:36.209676 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:36.213605 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8594e4ce37964bf09c89143bed59f40e" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:36.215500 21824 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:36.216372 29153 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:36.222535 21756 catalog_manager.cc:5697] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 16 to 23. New cstate: current_term: 23 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.207877 29789 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:36.228691 21756 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 17, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 17 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.230412 21756 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.235960 29789 raft_consensus.cc:493] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.236529 29789 raft_consensus.cc:515] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.238663 29789 leader_election.cc:290] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.239558 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:36.240057 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9ae799c8f043d7b4fc384aa1edf5ce" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:36.243521 21825 leader_election.cc:304] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:36.244261 29789 raft_consensus.cc:2749] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:36.236889 21756 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 20, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.263293 21756 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 20 to 21, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 21 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.272855 21756 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 23 to 25, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 25 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.274704 21756 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 20 to 21, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 21 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.286235 29153 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.276953 21756 catalog_manager.cc:5697] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 22 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 23 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:36.286795 29153 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:36.290721 29153 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:36.292904 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:36.294332 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3a64e6b0242fa9fda4766cc68016b" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:36.299427 21824 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:36.307267 29153 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:08:36.326280 21756 catalog_manager.cc:5697] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 21 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 23 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.327951 21756 catalog_manager.cc:5697] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 22 to 25, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 25 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.330046 21756 catalog_manager.cc:5697] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.331679 21756 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 21 to 24, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 24 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.333014 21756 catalog_manager.cc:5697] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 24, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 24 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:36.334865 21756 catalog_manager.cc:5697] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 19, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 19 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:36.336169 21755 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 308) took 89201 ms (client timeout 15000 ms). Trace:
I20260709 13:08:36.336404 21756 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 22 to 23, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 23 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.337843 21756 catalog_manager.cc:5697] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 18, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:36.336690 21755 rpcz_store.cc:269] 0709 13:07:07.135120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.143087 (+  7967us) service_pool.cc:224] Handling call
0709 13:08:36.336090 (+89193003us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:36.175582 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:36.176149 (+   567us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:36.176189 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:08:36.176207 (+    18us) tablet.cc:662] Decoding operations
0709 13:08:36.176592 (+   385us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:36.176650 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:36.176676 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:36.176843 (+   167us) tablet.cc:701] Row locks acquired
0709 13:08:36.176883 (+    40us) write_op.cc:260] PREPARE: finished
0709 13:08:36.177034 (+   151us) write_op.cc:270] Start()
0709 13:08:36.177165 (+   131us) write_op.cc:276] Timestamp: P: 1783602516176976 usec, L: 0
0709 13:08:36.177188 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:36.177847 (+   659us) log.cc:844] Serialized 541 byte log entry
0709 13:08:36.179850 (+  2003us) op_driver.cc:464] REPLICATION: finished
0709 13:08:36.180634 (+   784us) write_op.cc:301] APPLY: starting
0709 13:08:36.180709 (+    75us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:36.181488 (+   779us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:36.181580 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:36.182216 (+   636us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:36.182359 (+   143us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:08:36.182402 (+    43us) write_op.cc:312] APPLY: finished
0709 13:08:36.182577 (+   175us) log.cc:844] Serialized 25 byte log entry
0709 13:08:36.182744 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.182813 (+    69us) write_op.cc:454] Released schema lock
0709 13:08:36.182849 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:36.183208 (+   359us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.183237 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":84344,"reported_tablets":49,"spinlock_wait_cycles":21888,"child_traces":[["op",{"apply.queue_time_us":565,"apply.run_cpu_time_us":2526,"apply.run_wall_time_us":2535,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":2555,"prepare.run_wall_time_us":2556,"replication_time_us":2530}]]}
I20260709 13:08:36.342504 21756 catalog_manager.cc:5697] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 20 to 26, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 26 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:36.354962 21756 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 22, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.361306 21756 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.362691 21756 catalog_manager.cc:5697] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.365445 21756 catalog_manager.cc:5697] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 20 to 21, leader changed from 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.366873 21756 catalog_manager.cc:5697] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 13 to 16, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 16 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:36.368430 21756 catalog_manager.cc:5697] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 22, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 22 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:36.444135 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:36.446338 21898 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:36.467348 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:36.479673 20746 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 21885)
I20260709 13:08:36.489396 29872 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:36.490087 29872 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.493460 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:36.494261 22592 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:36.494875 29872 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.497418 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:36.498172 22001 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:36.501647 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:36.502768 29872 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:36.503163 29872 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:36.503527 29872 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:36.507603 22592 raft_consensus.cc:1217] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEARNER]: Deduplicated request from leader. Original: 19.12->[19.13-19.13]   Dedup: 19.13->[]
W20260709 13:08:36.516607 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:36.534221 21928 consensus_queue.cc:237] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.546183 22592 raft_consensus.cc:1275] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:36.547890 29922 consensus_queue.cc:1048] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:36.549317 29872 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:36.553668 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:36.553817 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:36.568859 22004 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:36.570257 21899 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:36.571828 29872 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.572484 29872 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:36.572959 29872 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:36.574031 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:36.573916 29872 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.575040 22589 raft_consensus.cc:2393] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 20: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:36.592105 21756 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53918 (request call id 197) took 1051 ms. Trace:
I20260709 13:08:36.605371 29922 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 6 to 14, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.607267 22592 raft_consensus.cc:2955] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 6 to 14, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:36.599481 21756 rpcz_store.cc:276] 0709 13:08:35.540772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.812572 (+271800us) service_pool.cc:224] Handling call
0709 13:08:36.592007 (+779435us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:36.404617 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:36.405247 (+   630us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:36.405285 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:08:36.405312 (+    27us) tablet.cc:662] Decoding operations
0709 13:08:36.407075 (+  1763us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:36.407145 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:36.407176 (+    31us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:08:36.409200 (+  2024us) tablet.cc:701] Row locks acquired
0709 13:08:36.409220 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:08:36.409355 (+   135us) write_op.cc:270] Start()
0709 13:08:36.409470 (+   115us) write_op.cc:276] Timestamp: P: 1783602516409311 usec, L: 0
0709 13:08:36.409492 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:08:36.410232 (+   740us) log.cc:844] Serialized 9081 byte log entry
0709 13:08:36.413799 (+  3567us) op_driver.cc:464] REPLICATION: finished
0709 13:08:36.415651 (+  1852us) write_op.cc:301] APPLY: starting
0709 13:08:36.415729 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:36.450760 (+ 35031us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:36.450863 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:36.472363 (+ 21500us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:36.472818 (+   455us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:08:36.472869 (+    51us) write_op.cc:312] APPLY: finished
0709 13:08:36.473318 (+   449us) log.cc:844] Serialized 204 byte log entry
0709 13:08:36.473491 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.473701 (+   210us) write_op.cc:454] Released schema lock
0709 13:08:36.473770 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:36.474077 (+   307us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.474104 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17372,"reported_tablets":35,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":1640,"apply.run_cpu_time_us":26168,"apply.run_wall_time_us":58404,"cfile_cache_hit":92,"cfile_cache_hit_bytes":96278,"num_ops":23,"prepare.queue_time_us":10193,"prepare.run_cpu_time_us":5722,"prepare.run_wall_time_us":8681,"replication_time_us":4158}]]}
W20260709 13:08:36.625810 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:36.638365 29846 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.639257 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4458afcc40954b359ea857cddbfdd681 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 5)
I20260709 13:08:36.639199 29846 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.641508 29846 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:36.643178 29565 raft_consensus.cc:493] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.643283 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:36.643805 29565 raft_consensus.cc:515] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.644794 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:36.644093 22001 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:36.646479 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:36.647984 29864 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:36.648418 29864 raft_consensus.cc:493] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.648777 29864 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:36.651108 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:36.653446 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5d11192c5d4d72b0bb9f3ebe9b715a" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:36.657673 21824 leader_election.cc:304] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:36.648792 29565 leader_election.cc:290] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.664424 29696 raft_consensus.cc:2749] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:36.690353 21755 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index -1 to 13, term changed from 18 to 19, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131), NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:36.696058 29864 raft_consensus.cc:515] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.712555 22001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49448
I20260709 13:08:36.713369 22001 raft_consensus.cc:606] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Received request to transfer leadership
I20260709 13:08:36.723244 29864 leader_election.cc:290] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:36.724373 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:36.726025 29696 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.726552 29696 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.729363 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96543776baa4cdda2e6862202de4e52" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:36.729909 22008 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:36.726826 22004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49448
I20260709 13:08:36.735193 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:36.735642 22004 raft_consensus.cc:606] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Received request to transfer leadership
I20260709 13:08:36.739252 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "430edce57310489986e4f739148461a6" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:36.728816 29696 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.740669 21823 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:36.741822 29696 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:36.753736 29789 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.754287 29789 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.756489 29789 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:36.742205 22008 raft_consensus.cc:2468] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
W20260709 13:08:36.746652 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:36.757457 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:36.750208 22004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203"
dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49448
I20260709 13:08:36.759723 22004 raft_consensus.cc:606] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Received request to transfer leadership
I20260709 13:08:36.757119 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c70dbf50c904384a2ab23b1d7a9dbfd" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:36.765156 21823 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:36.765938 29696 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:36.766213 22557 leader_election.cc:304] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:36.760248 21751 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 10 to 14, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 14 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
W20260709 13:08:36.761307 21972 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:36.767916 29846 raft_consensus.cc:2804] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:36.768437 29846 raft_consensus.cc:697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:36.769270 29846 consensus_queue.cc:237] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.770496 21751 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 13 to 17, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 17 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:36.774020 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:36.781229 21806 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:08:36.781813 21806 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for bc33f94111684d68a49eb643f8be2203 already in progress
W20260709 13:08:36.782151 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for bc33f94111684d68a49eb643f8be2203 already in progress
I20260709 13:08:36.810376 21755 catalog_manager.cc:5697] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 6 to 14, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:36.825727 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:36.837992 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:36.863976 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 4458afcc40954b359ea857cddbfdd681 with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:49448
I20260709 13:08:36.869887 21755 catalog_manager.cc:5697] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 21, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 21 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:36.893889 29934 tablet_replica.cc:333] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:36.913427 29934 raft_consensus.cc:2243] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:36.914640 29934 raft_consensus.cc:2272] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:36.918295 29934 ts_tablet_manager.cc:1916] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:08:36.935802 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:36.948722 29077 raft_consensus.cc:993] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: : Instructing follower 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:08:36.949193 29077 raft_consensus.cc:1081] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Signalling peer 821b3831b8a348c0826bd3a20a857f00 to start an election
I20260709 13:08:36.951751 21932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203"
dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
 from {username='slave'} at 127.0.0.1:33888
I20260709 13:08:36.952371 21932 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:36.952718 21932 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:36.968220 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:36.968802 29934 ts_tablet_manager.cc:1929] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.13
I20260709 13:08:36.974292 29934 log.cc:1199] T 4458afcc40954b359ea857cddbfdd681 P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/4458afcc40954b359ea857cddbfdd681
I20260709 13:08:36.976468 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 4458afcc40954b359ea857cddbfdd681 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:08:36.991640 29789 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.992859 29789 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:36.992597 21932 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:36.996622 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:36.997342 21932 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:37.000262 21898 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:37.005573 22004 raft_consensus.cc:3055] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:37.005972 22004 raft_consensus.cc:740] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:37.006798 22004 consensus_queue.cc:260] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:37.014005 22004 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:37.026923 22004 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:37.027890 21899 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:37.028111 29789 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:37.029239 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:37.029430 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:37.030054 29932 raft_consensus.cc:2804] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:37.031565 29932 raft_consensus.cc:697] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:37.032037 21823 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:37.032388 29932 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:37.036942 29789 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:37.053452 21752 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 289) took 89741 ms (client timeout 15000 ms). Trace:
W20260709 13:08:37.053992 21752 rpcz_store.cc:269] 0709 13:07:07.312566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.313023 (+   457us) service_pool.cc:224] Handling call
0709 13:07:09.009434 (+1696411us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b5400011210. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6ab0b2 00007f3b76624a89 00007f3b765bcda6 00007f3b7658e088 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc
0709 13:07:10.214347 (+1204913us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70739090 00007f3b707d7f3a 00007f3b7658e1cc 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c 00007f3b6eb098c6
0709 13:08:37.053346 (+86838999us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":450579,"reported_tablets":49,"spinlock_wait_cycles":123719168}
I20260709 13:08:37.064113 22004 raft_consensus.cc:1240] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Rejecting Update request from peer 821b3831b8a348c0826bd3a20a857f00 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:37.066887 29932 consensus_queue.cc:1059] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:37.067782 29932 raft_consensus.cc:3055] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:37.068138 29932 raft_consensus.cc:740] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:37.068928 29932 consensus_queue.cc:260] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:37.070231 29932 raft_consensus.cc:3060] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:37.085496 29696 raft_consensus.cc:493] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.086189 29696 raft_consensus.cc:515] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:37.089061 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:37.092345 22592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41480ad8e164cf2aa88df180060345c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:37.094094 21825 leader_election.cc:304] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:37.094964 29789 raft_consensus.cc:2749] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:37.102977 29696 leader_election.cc:290] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:37.116465 21751 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 309) took 60855 ms (client timeout 15000 ms). Trace:
W20260709 13:08:37.117892 21751 rpcz_store.cc:269] 0709 13:07:36.260594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.261200 (+   606us) service_pool.cc:224] Handling call
0709 13:07:37.592155 (+1330955us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b5400011210. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d6aab71 00007f3b6d6ab0b2 00007f3b76624a89 00007f3b765bcda6 00007f3b7658e088 00007f3b7668c37e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc
0709 13:08:37.116370 (+59524215us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:36.775933 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:36.776468 (+   535us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:36.776504 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:08:36.776522 (+    18us) tablet.cc:662] Decoding operations
0709 13:08:36.776942 (+   420us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:36.777012 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:36.777039 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:08:36.777357 (+   318us) tablet.cc:701] Row locks acquired
0709 13:08:36.777376 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:36.777516 (+   140us) write_op.cc:270] Start()
0709 13:08:36.777762 (+   246us) write_op.cc:276] Timestamp: P: 1783602516777471 usec, L: 0
0709 13:08:36.777789 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:08:36.778405 (+   616us) log.cc:844] Serialized 929 byte log entry
0709 13:08:36.780356 (+  1951us) op_driver.cc:464] REPLICATION: finished
0709 13:08:36.780756 (+   400us) write_op.cc:301] APPLY: starting
0709 13:08:36.780823 (+    67us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:36.782475 (+  1652us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:36.782560 (+    85us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:36.785683 (+  3123us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:36.785841 (+   158us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:08:36.785885 (+    44us) write_op.cc:312] APPLY: finished
0709 13:08:36.786091 (+   206us) log.cc:844] Serialized 33 byte log entry
0709 13:08:36.786257 (+   166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.786337 (+    80us) write_op.cc:454] Released schema lock
0709 13:08:36.786380 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:36.786686 (+   306us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:36.786713 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":126919,"reported_tablets":49,"spinlock_wait_cycles":189158656,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":4018,"apply.run_wall_time_us":5904,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8372,"num_ops":2,"prepare.queue_time_us":138,"prepare.run_cpu_time_us":2765,"prepare.run_wall_time_us":2765,"replication_time_us":2467}]]}
I20260709 13:08:37.123116 21753 catalog_manager.cc:5697] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 18 to 21, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 21 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.124586 21753 catalog_manager.cc:5697] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 13 to 19, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 19 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.126379 21753 catalog_manager.cc:5697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 18 to 21, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 21 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.127846 21753 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.153968 21753 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.155421 21753 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.157370 21753 catalog_manager.cc:5697] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 21 to 24, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 24 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:37.170135 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.177791 21753 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 17 to 25. New cstate: current_term: 25 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.179788 21753 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.181399 21753 catalog_manager.cc:5697] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 17 to 18, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:37.176853 21750 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 285) took 59210 ms (client timeout 15000 ms). Trace:
W20260709 13:08:37.193568 21750 rpcz_store.cc:269] 0709 13:07:37.966827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.968435 (+  1608us) service_pool.cc:224] Handling call
0709 13:08:37.171597 (+59203162us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b6800136160. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b765aade1 00007f3b765880ca 00007f3b76586808 00007f3b76584e40 00007f3b7668cc3e 00007f3b74b4d2a4 00007f3b74b4d232 00007f3b74b4d161 00007f3b74b4d0dc 00007f3b74b4c392 00007f3b6eb0a48c
0709 13:08:37.176769 (+  5172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62514,"reported_tablets":48,"spinlock_wait_cycles":3491200}
W20260709 13:08:37.262640 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.270905 22004 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:37.272572 29846 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:37.312990 29696 raft_consensus.cc:493] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:37.313650 29696 raft_consensus.cc:515] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:37.315944 29696 leader_election.cc:290] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:37.318065 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:37.319468 21757 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.322047 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3ae8d7d5514b8eb2514ea857b270c1" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:37.335044 21824 leader_election.cc:304] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:37.337616 29565 raft_consensus.cc:2749] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:37.347872 21757 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.349750 21757 catalog_manager.cc:5697] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 21 to 22, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.350945 21935 raft_consensus.cc:1275] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 13:08:37.359930 21935 pending_rounds.cc:85] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Aborting all ops after (but not including) 16
I20260709 13:08:37.360198 21935 pending_rounds.cc:107] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 13:08:37.360440 21935 raft_consensus.cc:2889] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:37.361800 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet 4458afcc40954b359ea857cddbfdd681 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:49448
I20260709 13:08:37.363323 29846 consensus_queue.cc:1048] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:37.365427 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet 4458afcc40954b359ea857cddbfdd681 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:08:37.369959 21757 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.375094 21753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51626 (request call id 311) took 9305 ms. Trace:
I20260709 13:08:37.372480 21757 catalog_manager.cc:5697] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 12 to 20, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 20 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.375551 21753 rpcz_store.cc:276] 0709 13:08:28.069553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.415447 (+3345894us) service_pool.cc:224] Handling call
0709 13:08:37.375029 (+5959582us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:37.201471 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:37.202027 (+   556us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:37.202092 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:08:37.202114 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:37.203154 (+  1040us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:37.203216 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:37.203248 (+    32us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:08:37.204089 (+   841us) tablet.cc:701] Row locks acquired
0709 13:08:37.204109 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:08:37.204266 (+   157us) write_op.cc:270] Start()
0709 13:08:37.204383 (+   117us) write_op.cc:276] Timestamp: P: 1783602517204219 usec, L: 0
0709 13:08:37.204406 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:37.205718 (+  1312us) log.cc:844] Serialized 4034 byte log entry
0709 13:08:37.208211 (+  2493us) op_driver.cc:464] REPLICATION: finished
0709 13:08:37.208681 (+   470us) write_op.cc:301] APPLY: starting
0709 13:08:37.208765 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:37.214588 (+  5823us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:37.214708 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:37.222865 (+  8157us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:37.223084 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:08:37.223142 (+    58us) write_op.cc:312] APPLY: finished
0709 13:08:37.223436 (+   294us) log.cc:844] Serialized 97 byte log entry
0709 13:08:37.229948 (+  6512us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:37.230072 (+   124us) write_op.cc:454] Released schema lock
0709 13:08:37.230122 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:37.260843 (+ 30721us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:37.260878 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":87889,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":269,"apply.run_cpu_time_us":12855,"apply.run_wall_time_us":52112,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41860,"num_ops":10,"prepare.queue_time_us":2827,"prepare.run_cpu_time_us":3971,"prepare.run_wall_time_us":11694,"replication_time_us":3670}]]}
I20260709 13:08:37.400882 21757 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.403568 21757 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:37.480391 20746 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 21885)
I20260709 13:08:37.482863 29789 raft_consensus.cc:493] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.483440 29789 raft_consensus.cc:515] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:37.485411 29789 leader_election.cc:290] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:37.491730 29565 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.495595 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:37.492283 29565 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:37.501981 29565 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:37.497807 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e33356352ee4f06be9c7f962467110c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:37.503656 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:37.510953 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:37.519168 21823 leader_election.cc:304] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:37.533574 21823 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:37.534976 29789 raft_consensus.cc:2749] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:37.535655 29789 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:37.621322 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4458afcc40954b359ea857cddbfdd681" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:37.637784 29789 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.638295 29789 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:37.652326 21742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 287718us
I20260709 13:08:37.658640 29789 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:37.659111 22586 raft_consensus.cc:3060] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 19 LEARNER]: Advancing to term 20
W20260709 13:08:37.663542 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:37.664433 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.668903 21754 catalog_manager.cc:5697] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 15 to 18, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 18 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.678886 22004 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:37.680562 29932 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:08:37.683277 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): ChangeConfig:REMOVE_PEER RPC failed for tablet 2167f46d5e1049268b9ce66dbf46f781: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:37.683399 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:37.699576 21757 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51604 (request call id 287) took 4301 ms. Trace:
I20260709 13:08:37.703609 21754 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 22 to 25, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 25 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.708714 21754 catalog_manager.cc:5697] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 18 to 21, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.710784 21754 catalog_manager.cc:5697] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 18 to 22, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 22 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.712783 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cd5c4a1eef4004b3f1a9728e83a459" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:37.718225 22586 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:37.700011 21757 rpcz_store.cc:276] 0709 13:08:33.397921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.499125 (+2101204us) service_pool.cc:224] Handling call
0709 13:08:37.699485 (+2200360us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:37.454528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:37.455089 (+   561us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:37.455195 (+   106us) write_op.cc:435] Acquired schema lock
0709 13:08:37.455216 (+    21us) tablet.cc:662] Decoding operations
0709 13:08:37.456111 (+   895us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:37.456176 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:37.456205 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:08:37.456867 (+   662us) tablet.cc:701] Row locks acquired
0709 13:08:37.456886 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:37.457075 (+   189us) write_op.cc:270] Start()
0709 13:08:37.457204 (+   129us) write_op.cc:276] Timestamp: P: 1783602517457030 usec, L: 0
0709 13:08:37.457225 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:08:37.458358 (+  1133us) log.cc:844] Serialized 2870 byte log entry
0709 13:08:37.461058 (+  2700us) op_driver.cc:464] REPLICATION: finished
0709 13:08:37.461518 (+   460us) write_op.cc:301] APPLY: starting
0709 13:08:37.461601 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:37.466437 (+  4836us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:37.466755 (+   318us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:37.483778 (+ 17023us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:37.484062 (+   284us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:08:37.484132 (+    70us) write_op.cc:312] APPLY: finished
0709 13:08:37.484532 (+   400us) log.cc:844] Serialized 73 byte log entry
0709 13:08:37.484776 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:37.484943 (+   167us) write_op.cc:454] Released schema lock
0709 13:08:37.486775 (+  1832us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:37.490476 (+  3701us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:37.490506 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16519,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":271,"apply.run_cpu_time_us":12659,"apply.run_wall_time_us":28800,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29302,"num_ops":7,"prepare.queue_time_us":3385,"prepare.run_cpu_time_us":3777,"prepare.run_wall_time_us":14646,"replication_time_us":3688}]]}
W20260709 13:08:37.737571 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 79377.8us
W20260709 13:08:37.739106 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.747519 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:37.750025 20746 tablet_replica.cc:333] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.749874 29956 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:37.740054 29948 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.751924 29948 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:37.751252 20746 raft_consensus.cc:2243] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.755512 29948 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:37.765189 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:37.778275 21823 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:37.797596 29565 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:37.799569 21753 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 25 to 28, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 28 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.786247 20746 raft_consensus.cc:2272] T 873e001f77374da4a034a00eaaea672f P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.814733 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e8056fad5248118e00e80f8a81538c" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:37.821842 20746 tablet_replica.cc:333] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
W20260709 13:08:37.821650 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.823344 20746 raft_consensus.cc:2243] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:37.829242 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:37.829906 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.830837 20746 raft_consensus.cc:2272] T 430edce57310489986e4f739148461a6 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.832556 29951 raft_consensus.cc:1064] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:08:37.832948 21823 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:37.834374 29953 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:37.835879 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.836653 20746 tablet_replica.cc:333] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.839401 29951 consensus_queue.cc:237] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:37.847865 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:37.851207 21753 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.852733 21753 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.854211 21753 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 16 to 25. New cstate: current_term: 25 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.854812 22589 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:37.855615 21753 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.856855 29957 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:37.858685 20746 raft_consensus.cc:2243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.859433 20746 pending_rounds.cc:70] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:37.859514 22008 raft_consensus.cc:1275] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:37.859889 21753 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 18 to 23, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129). New cstate: current_term: 23 leader_uuid: "8f3b35a35257416db3eaaf364cf6b52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:37.862150 29957 consensus_queue.cc:1048] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:37.863605 21754 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 292) took 8275 ms. Trace:
I20260709 13:08:37.862182 21753 catalog_manager.cc:5697] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 17 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.864192 21754 rpcz_store.cc:276] 0709 13:08:29.588101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.463700 (+4875599us) service_pool.cc:224] Handling call
0709 13:08:37.863543 (+3399843us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:37.725387 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:37.725931 (+   544us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:37.725966 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:08:37.725987 (+    21us) tablet.cc:662] Decoding operations
0709 13:08:37.726568 (+   581us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:37.726623 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:37.726712 (+    89us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:08:37.727375 (+   663us) tablet.cc:701] Row locks acquired
0709 13:08:37.727394 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:37.727534 (+   140us) write_op.cc:270] Start()
0709 13:08:37.727648 (+   114us) write_op.cc:276] Timestamp: P: 1783602517727489 usec, L: 0
0709 13:08:37.727669 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:08:37.728449 (+   780us) log.cc:844] Serialized 1706 byte log entry
0709 13:08:37.730937 (+  2488us) op_driver.cc:464] REPLICATION: finished
0709 13:08:37.731289 (+   352us) write_op.cc:301] APPLY: starting
0709 13:08:37.731368 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:37.733768 (+  2400us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:37.733862 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:37.735908 (+  2046us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:37.736055 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:08:37.736092 (+    37us) write_op.cc:312] APPLY: finished
0709 13:08:37.736289 (+   197us) log.cc:844] Serialized 49 byte log entry
0709 13:08:37.736441 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:37.736544 (+   103us) write_op.cc:454] Released schema lock
0709 13:08:37.736580 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:37.736886 (+   306us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:37.736914 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":44272,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":156,"apply.run_cpu_time_us":5572,"apply.run_wall_time_us":5574,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16744,"num_ops":4,"prepare.queue_time_us":176,"prepare.run_cpu_time_us":3338,"prepare.run_wall_time_us":6771,"replication_time_us":3158}]]}
I20260709 13:08:37.859704 20746 pending_rounds.cc:77] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 17 index: 14 } timestamp: 7305635752061575168 op_type: NO_OP noop_request { }
I20260709 13:08:37.886579 20746 raft_consensus.cc:2889] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.887238 20746 raft_consensus.cc:2272] T 0b8c4da9f5144e2b8007c8ffc563a584 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.883876 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.890694 20746 tablet_replica.cc:333] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.889958 29951 raft_consensus.cc:2955] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 13 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:37.896147 20746 raft_consensus.cc:2243] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.903851 20746 pending_rounds.cc:70] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 3 pending ops.
I20260709 13:08:37.904958 20746 pending_rounds.cc:77] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 24 index: 12 } timestamp: 7305635831652880384 op_type: NO_OP noop_request { }
I20260709 13:08:37.905426 20746 raft_consensus.cc:2889] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.905810 20746 pending_rounds.cc:77] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305635750420795392 op_type: NO_OP noop_request { }
I20260709 13:08:37.906176 20746 raft_consensus.cc:2889] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.906522 20746 pending_rounds.cc:77] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 19 index: 10 } timestamp: 7305635681102909440 op_type: NO_OP noop_request { }
I20260709 13:08:37.906879 20746 raft_consensus.cc:2889] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.907562 20746 raft_consensus.cc:2272] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.913244 20746 tablet_replica.cc:333] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.914237 20746 raft_consensus.cc:2243] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.915359 20746 raft_consensus.cc:2272] T 1d830a9119594381aa253bf173c99566 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.918119 20746 tablet_replica.cc:333] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.919085 20746 raft_consensus.cc:2243] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.920166 20746 raft_consensus.cc:2272] T 5f588420255042dda396ba6419abf6bc P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.923154 20746 tablet_replica.cc:333] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.924182 20746 raft_consensus.cc:2243] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.924832 20746 pending_rounds.cc:70] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:37.925172 20746 pending_rounds.cc:77] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 21 index: 15 } timestamp: 7305635845846228992 op_type: NO_OP noop_request { }
I20260709 13:08:37.925617 20746 raft_consensus.cc:2889] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.926002 20746 raft_consensus.cc:2272] T be9ae799c8f043d7b4fc384aa1edf5ce P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.928853 20746 tablet_replica.cc:333] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.929950 20746 raft_consensus.cc:2243] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.931566 20746 pending_rounds.cc:70] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:37.932032 20746 pending_rounds.cc:77] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 27 index: 16 } timestamp: 7305635885143003136 op_type: NO_OP noop_request { }
I20260709 13:08:37.931466 21750 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 20, leader changed from 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:37.932461 20746 raft_consensus.cc:2889] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.933120 20746 raft_consensus.cc:2272] T 6ed3a64e6b0242fa9fda4766cc68016b P 8f3b35a35257416db3eaaf364cf6b52b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.894696 22589 raft_consensus.cc:2955] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 13 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:37.935981 20746 tablet_replica.cc:333] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.936972 20746 raft_consensus.cc:2243] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.937937 20746 raft_consensus.cc:2272] T a226c66b13a04eb389353b8e2deb175c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.941150 20746 tablet_replica.cc:333] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.942126 20746 raft_consensus.cc:2243] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.943337 20746 raft_consensus.cc:2272] T 40a1bb4717e14331a67545a343ce1d87 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.947080 20746 tablet_replica.cc:333] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.948046 20746 raft_consensus.cc:2243] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.948907 20746 pending_rounds.cc:70] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:37.949309 20746 pending_rounds.cc:77] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 23 index: 12 } timestamp: 7305635797942837248 op_type: NO_OP noop_request { }
I20260709 13:08:37.949718 20746 raft_consensus.cc:2889] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.950162 20746 raft_consensus.cc:2272] T 8594e4ce37964bf09c89143bed59f40e P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.953631 20746 tablet_replica.cc:333] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.954739 20746 raft_consensus.cc:2243] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.955612 20746 pending_rounds.cc:70] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:37.955936 20746 pending_rounds.cc:77] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305635847130411008 op_type: NO_OP noop_request { }
I20260709 13:08:37.956342 20746 raft_consensus.cc:2889] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.956745 20746 raft_consensus.cc:2272] T 676bc9491c5a4083bdd0b754d904c1d6 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.959682 20746 tablet_replica.cc:333] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.960820 20746 raft_consensus.cc:2243] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.963189 20746 raft_consensus.cc:2272] T 94e8056fad5248118e00e80f8a81538c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.968631 20746 tablet_replica.cc:333] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.969650 20746 raft_consensus.cc:2243] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.970247 20746 pending_rounds.cc:70] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:37.970536 20746 pending_rounds.cc:77] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305635781794672640 op_type: NO_OP noop_request { }
I20260709 13:08:37.970949 20746 raft_consensus.cc:2889] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:37.971338 20746 raft_consensus.cc:2272] T 10076817fc0f46b4943de726ac0b508c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.973654 22008 raft_consensus.cc:2955] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 13 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:37.975059 20746 tablet_replica.cc:333] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.976002 20746 raft_consensus.cc:2243] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.981230 20746 raft_consensus.cc:2272] T 739311613bd748d68267b6f4478680c7 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.986241 20746 tablet_replica.cc:333] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:37.993587 20746 raft_consensus.cc:2243] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.994400 20746 raft_consensus.cc:2272] T 5ef7ed88974a427b9cab09af73283182 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.997241 20746 tablet_replica.cc:333] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.010483 20746 raft_consensus.cc:2243] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.011454 20746 pending_rounds.cc:70] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:38.011786 20746 pending_rounds.cc:77] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305635839961948160 op_type: NO_OP noop_request { }
I20260709 13:08:38.012193 20746 raft_consensus.cc:2889] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:38.012624 20746 raft_consensus.cc:2272] T d41480ad8e164cf2aa88df180060345c P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.012413 21750 catalog_manager.cc:5697] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 13 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:38.015535 20746 tablet_replica.cc:333] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.016435 20746 raft_consensus.cc:2243] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.017028 20746 raft_consensus.cc:2272] T 634269b557d04cbe87cfab961aaf3f91 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.019810 20746 tablet_replica.cc:333] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.029755 20746 raft_consensus.cc:2243] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.030690 20746 raft_consensus.cc:2272] T 0ecc321cb1e14e99a3838b646f82e71a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.033893 20746 tablet_replica.cc:333] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.034889 20746 raft_consensus.cc:2243] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.036474 20746 raft_consensus.cc:2272] T 2167f46d5e1049268b9ce66dbf46f781 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.047619 20746 tablet_replica.cc:333] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.059036 20746 raft_consensus.cc:2243] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.060144 20746 raft_consensus.cc:2272] T 4458afcc40954b359ea857cddbfdd681 P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.063166 20746 tablet_replica.cc:333] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.064136 20746 raft_consensus.cc:2243] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.064760 20746 raft_consensus.cc:2272] T 81cd5c4a1eef4004b3f1a9728e83a459 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.068362 20746 tablet_replica.cc:333] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.069347 20746 raft_consensus.cc:2243] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.070453 20746 raft_consensus.cc:2272] T 09efb42cfd6142bb9f816328042aea38 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.073187 20746 tablet_replica.cc:333] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.073966 20746 raft_consensus.cc:2243] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.076308 20746 raft_consensus.cc:2272] T 90e34f5775b8400e8e9bb1a13ff27479 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.112264 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:38.131850 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:38.337214 29696 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.337916 29696 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:38.340615 29696 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:38.342446 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:38.345717 22009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:38.347226 21824 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:38.348503 29696 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:38.377453 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 1d830a9119594381aa253bf173c99566: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:38.386096 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 7e16208fe6d441998baf1cb8b8546d10: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:38.388922 20746 tablet_replica.cc:333] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.389874 20746 raft_consensus.cc:2243] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.390486 20746 raft_consensus.cc:2272] T 8a76564a0ee84b74a3af01c79300d4b7 P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.395251 21806 auto_rebalancer.cc:629] Failed to schedule move for tablet 09efb42cfd6142bb9f816328042aea38: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:38.397964 20746 tablet_replica.cc:333] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.399066 20746 raft_consensus.cc:2243] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.399861 29696 raft_consensus.cc:493] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.400820 29696 raft_consensus.cc:515] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:38.401952 20746 raft_consensus.cc:2272] T 9fc728770e5c4aa1ac36a662cebb8598 P 8f3b35a35257416db3eaaf364cf6b52b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.404503 29696 leader_election.cc:290] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:38.404721 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:38.404959 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee2529af5954101bde1189a694cafa5" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:38.406349 21824 leader_election.cc:304] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:38.407636 29696 raft_consensus.cc:2749] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:38.409108 21928 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:38.415730 20746 tablet_replica.cc:333] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.416592 20746 raft_consensus.cc:2243] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:38.542352 20746 raft_consensus.cc:2272] T 2ee2529af5954101bde1189a694cafa5 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.551324 29789 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:38.551851 29789 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:38.626093 29789 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:38.638710 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 222931us
I20260709 13:08:38.822785 29077 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:38.823559 29077 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:38.825453 21742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99261.1us
I20260709 13:08:38.835153 29077 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:38.884980 20746 tablet_replica.cc:333] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.886224 20746 raft_consensus.cc:2243] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:38.887132 20746 raft_consensus.cc:2272] T 71a9145a148c4edebd5d8df1c9956298 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:38.907146 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:38.917919 20746 tablet_replica.cc:333] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:38.918903 20746 raft_consensus.cc:2243] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Raft consensus shutting down.
W20260709 13:08:38.925800 21898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.006290 20746 raft_consensus.cc:2272] T 635368d86daf4188a9dcb9cc2f31ce10 P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:38.997098 21898 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.034315 20746 tablet_replica.cc:333] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.037015 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4021839a030d44ab886833307930ad33" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:39.013621 21824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.054373 20746 raft_consensus.cc:2243] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.071410 22000 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:39.074785 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:39.094803 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): ChangeConfig:REMOVE_PEER RPC failed for tablet a5e5e01f84ee40afb9b688bf0b446c6c: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.103175 20746 raft_consensus.cc:2272] T f7024293bd494d23afe75d0322f5445b P 8f3b35a35257416db3eaaf364cf6b52b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.122619 21825 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:39.123800 29950 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:39.124807 20746 tablet_replica.cc:333] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.125742 20746 raft_consensus.cc:2243] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.126341 20746 raft_consensus.cc:2272] T 97cf25498fcd4e4594d809b889d236d3 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.129047 29937 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.129863 29937 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.133277 29937 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:39.137553 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.138934 29952 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:39.141147 29937 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.141875 29937 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.150796 29937 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:39.157805 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.158432 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:39.159946 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:39.161521 29932 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.162240 29932 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.167693 20746 tablet_replica.cc:333] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
W20260709 13:08:39.168438 21900 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 676bc9491c5a4083bdd0b754d904c1d6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.180261 29963 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.180833 29963 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.182935 29963 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.168648 20746 raft_consensus.cc:2243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:39.169149 21898 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.169818 29937 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:39.187845 29932 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:39.188040 29952 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:39.170996 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:39.174613 21881 debug-util.cc:398] Leaking SignalData structure 0x7b0800351820 after lost signal to thread 20749
I20260709 13:08:39.195591 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:39.192099 21898 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.196421 29937 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:39.188680 29952 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:39.199199 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8b40 after lost signal to thread 21792
W20260709 13:08:39.199990 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08003513c0 after lost signal to thread 21884
W20260709 13:08:39.200335 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:39.200794 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 92863.8us
W20260709 13:08:39.201808 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08000db7a0 after lost signal to thread 21959
I20260709 13:08:39.202699 29937 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:39.203531 21898 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.203249 29937 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.208539 29937 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:39.204906 29952 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.208765 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:39.213153 21972 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.213901 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.214174 29967 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.214731 29967 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.215914 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:39.217466 29967 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:39.202124 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08000db760 after lost signal to thread 21960
I20260709 13:08:39.218194 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:39.219763 21881 debug-util.cc:398] Leaking SignalData structure 0x7b08001929a0 after lost signal to thread 22033
I20260709 13:08:39.220515 21900 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:39.220990 21881 debug-util.cc:398] Leaking SignalData structure 0x7b080033ca60 after lost signal to thread 22617
I20260709 13:08:39.223218 29952 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.223773 29952 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.225850 29952 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.221936 21972 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.227084 29937 raft_consensus.cc:2749] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:39.227201 29872 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:39.229478 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.232554 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.234012 29937 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:39.234915 29937 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.235385 29937 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.239166 29937 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:39.242338 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:39.244791 20746 raft_consensus.cc:2272] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.247713 20746 tablet_replica.cc:333] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.248592 20746 raft_consensus.cc:2243] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.251045 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.251773 22001 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:39.253043 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.254518 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.255110 22001 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:39.256305 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.256883 22001 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:39.258555 29967 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.259181 29967 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.264394 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.265067 21932 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
W20260709 13:08:39.267025 21972 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.267503 21972 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.268797 29983 raft_consensus.cc:2749] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:39.272439 29967 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:39.272464 29947 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.273402 29947 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.275766 29947 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.272651 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:39.276841 29872 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:39.278824 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.279618 29932 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:39.279973 29932 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.280333 29932 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:39.285539 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.286254 22000 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 19.
I20260709 13:08:39.287406 22557 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:39.288218 29947 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:39.288586 29947 raft_consensus.cc:493] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.288980 29947 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:39.295073 21754 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 293) took 1261 ms. Trace:
I20260709 13:08:39.295572 21754 rpcz_store.cc:276] 0709 13:08:38.033217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.033419 (+   202us) service_pool.cc:224] Handling call
0709 13:08:39.294966 (+1261547us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":271,"reported_tablets":40}
I20260709 13:08:39.300951 29932 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.307457 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.308313 29872 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:39.308709 29872 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.309679 29872 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:39.309790 21825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.313712 21754 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 13, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:39.315304 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45100.6us
I20260709 13:08:39.318902 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.319986 21932 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:39.322294 20746 raft_consensus.cc:2272] T 309ab90abff84c36a30bd5a2a0363d2c P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.322762 29952 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.308674 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.326992 29952 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.328084 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:39.329115 29952 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:39.323464 22556 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.325403 29789 raft_consensus.cc:493] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:39.330200 29789 raft_consensus.cc:515] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.333401 29789 leader_election.cc:290] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:39.334148 21898 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.334583 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.337291 29963 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:39.337728 29963 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.338089 29963 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:39.338163 21898 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.317975 29872 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.341763 29952 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.342685 29952 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:39.346357 21898 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.346781 29952 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:39.354301 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 45689us
I20260709 13:08:39.324024 29932 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:39.357424 21898 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.358434 29872 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.329313 29947 raft_consensus.cc:515] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.370179 29967 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:39.370728 29967 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.371131 29967 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:39.372048 29947 leader_election.cc:290] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.373868 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f588420255042dda396ba6419abf6bc" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:39.374539 22001 raft_consensus.cc:3060] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:39.375728 22556 leader_election.cc:336] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:39.370278 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.377677 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0103ac417f394657ac6cbe827df3b844" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.380242 21823 leader_election.cc:304] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:39.381289 29789 raft_consensus.cc:2749] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:39.382798 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90e34f5775b8400e8e9bb1a13ff27479" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:39.383452 22003 raft_consensus.cc:3060] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:39.384928 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.382961 22000 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:08:39.390174 21972 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.390698 21972 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.391764 29983 raft_consensus.cc:2749] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:39.393117 22003 raft_consensus.cc:2468] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
I20260709 13:08:39.395212 20746 tablet_replica.cc:333] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.396029 20746 raft_consensus.cc:2243] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.395951 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:39.397111 20746 raft_consensus.cc:2272] T 1db8966fba10424a8df85ad63defc925 P 8f3b35a35257416db3eaaf364cf6b52b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.398152 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.398772 22008 raft_consensus.cc:3060] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:39.399278 22003 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:39.402285 20746 tablet_replica.cc:333] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.403232 20746 raft_consensus.cc:2243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.403898 20746 raft_consensus.cc:2272] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.406840 20746 tablet_replica.cc:333] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.407718 20746 raft_consensus.cc:2243] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.408921 20746 pending_rounds.cc:70] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:39.409252 20746 pending_rounds.cc:77] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 18 index: 14 } timestamp: 7305635835069763584 op_type: NO_OP noop_request { }
I20260709 13:08:39.409618 20746 raft_consensus.cc:2889] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:39.409969 20746 raft_consensus.cc:2272] T 4021839a030d44ab886833307930ad33 P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.412777 20746 tablet_replica.cc:333] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.413643 20746 raft_consensus.cc:2243] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.414678 20746 raft_consensus.cc:2272] T 951fdd0c64f542cbab511fb32cfa91cb P 8f3b35a35257416db3eaaf364cf6b52b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.417520 20746 tablet_replica.cc:333] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.419471 20746 raft_consensus.cc:2243] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.420068 20746 pending_rounds.cc:70] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Trying to abort 1 pending ops.
I20260709 13:08:39.420310 20746 pending_rounds.cc:77] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305635830106419200 op_type: NO_OP noop_request { }
I20260709 13:08:39.420725 20746 raft_consensus.cc:2889] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:39.421167 20746 raft_consensus.cc:2272] T 0103ac417f394657ac6cbe827df3b844 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.422654 29963 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.425484 20746 tablet_replica.cc:333] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.429140 20746 raft_consensus.cc:2243] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.429953 20746 raft_consensus.cc:2272] T 1e33356352ee4f06be9c7f962467110c P 8f3b35a35257416db3eaaf364cf6b52b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.431396 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.437402 22001 raft_consensus.cc:2468] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:39.438879 22557 leader_election.cc:304] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.441350 21899 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.443300 21899 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.444710 22008 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:39.446025 29077 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.446606 29077 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.451879 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:39.449577 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.434266 29963 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.459059 21899 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.447604 29975 raft_consensus.cc:2804] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:39.430586 29967 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.447881 29932 raft_consensus.cc:2749] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:39.470068 29077 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:39.524703 29975 raft_consensus.cc:697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:39.457235 29952 raft_consensus.cc:2804] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:39.614562 22001 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 24: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:39.463071 29979 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:39.462553 29969 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.471932 29972 raft_consensus.cc:2804] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:39.473191 29963 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.628851 29789 raft_consensus.cc:493] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.630530 29789 raft_consensus.cc:515] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.632963 29789 leader_election.cc:290] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:39.640714 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 166825us
I20260709 13:08:39.626358 29944 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.629824 29963 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:39.652269 21742 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 259 similar messages]
I20260709 13:08:39.636401 29952 raft_consensus.cc:697] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:39.653138 20746 tablet_replica.cc:333] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.655047 20746 raft_consensus.cc:2243] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.650457 29988 ts_tablet_manager.cc:933] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:39.637171 29972 raft_consensus.cc:697] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:39.629652 29979 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.661021 20746 raft_consensus.cc:2272] T 7e16208fe6d441998baf1cb8b8546d10 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.629514 29969 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.665992 29979 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:39.669272 29978 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.663842 29952 consensus_queue.cc:237] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.670001 29978 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.677722 20746 tablet_replica.cc:333] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.678987 20746 raft_consensus.cc:2243] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.679785 20746 raft_consensus.cc:2272] T 2a91310e68464d64aa8324092182518f P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.668263 29975 consensus_queue.cc:237] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:39.666718 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 224011us
I20260709 13:08:39.664014 29972 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.698237 20746 tablet_replica.cc:333] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.699137 20746 raft_consensus.cc:2243] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.690923 29967 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:39.708885 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.709355 29077 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.709630 21928 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:39.710072 29077 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.652138 29944 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.721118 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:39.752491 21742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 100233us
I20260709 13:08:39.757099 21928 raft_consensus.cc:2393] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:39.763653 20746 raft_consensus.cc:2272] T caafc8397a4f4963abeff030a28f5c47 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.778674 29077 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:39.786139 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:39.805276 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:39.807052 29967 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:39.807513 29967 raft_consensus.cc:493] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.807876 29967 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:39.823475 20746 tablet_replica.cc:333] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
W20260709 13:08:39.832947 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 67736.9us
I20260709 13:08:39.832980 20746 raft_consensus.cc:2243] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.836982 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.838089 21933 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:39.840605 20746 raft_consensus.cc:2272] T 6622001ac78d44caa90b4ce7f5283aa1 P 8f3b35a35257416db3eaaf364cf6b52b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.815495 29963 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:39.855077 21972 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.860499 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958f1a2d78484a3bab2de506da912756" candidate_uuid: "8f3b35a35257416db3eaaf364cf6b52b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:39.869426 29944 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.876613 29969 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:39.885035 29944 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.892961 29944 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:39.894150 29978 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.901520 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:39.912001 22586 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 14.
I20260709 13:08:39.909562 29944 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.910904 29967 raft_consensus.cc:515] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:39.911389 22556 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.905535 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.915271 22001 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:39.919960 20746 tablet_replica.cc:333] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.926762 29967 leader_election.cc:290] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:39.927505 29988 tablet_copy_client.cc:323] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:08:39.935318 20746 raft_consensus.cc:2243] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.937126 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.936825 20746 raft_consensus.cc:2272] T a5e5e01f84ee40afb9b688bf0b446c6c P 8f3b35a35257416db3eaaf364cf6b52b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.930787 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:39.943215 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:39.950203 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 161806us
I20260709 13:08:39.951433 21899 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:39.951980 21825 leader_election.cc:304] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f3b35a35257416db3eaaf364cf6b52b; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:39.924546 29979 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.953610 29696 raft_consensus.cc:2749] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:39.954635 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.955281 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:39.955837 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:39.956902 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.957423 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:39.945600 29974 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:39.958551 29937 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:39.967268 29979 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:39.952793 29983 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:39.967489 20746 tablet_replica.cc:333] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
W20260709 13:08:39.959683 21972 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:39.973120 29983 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:39.973974 29983 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:39.975710 20746 raft_consensus.cc:2243] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.976440 20746 raft_consensus.cc:2272] T bc33f94111684d68a49eb643f8be2203 P 8f3b35a35257416db3eaaf364cf6b52b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.979770 20746 tablet_replica.cc:333] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:39.956614 29990 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:39.980615 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40a1bb4717e14331a67545a343ce1d87" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:40.003194 29990 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.003409 21933 raft_consensus.cc:3060] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:40.003713 29990 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:39.991871 22003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:39.992304 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:40.006157 22000 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25.
I20260709 13:08:39.992966 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:40.007186 22005 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:39.994832 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739311613bd748d68267b6f4478680c7" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:40.017442 22001 raft_consensus.cc:3060] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:40.017841 29990 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:39.997817 29983 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:39.958333 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.020689 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 70710.3us
W20260709 13:08:40.024780 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.029215 29983 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.029772 29990 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:40.029632 21750 catalog_manager.cc:5697] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 19 to 20, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 20 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:40.031106 21933 raft_consensus.cc:2468] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:40.034148 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:40.034921 20746 raft_consensus.cc:2243] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:40.036062 20746 raft_consensus.cc:2272] T 8f2ed3e1870148589d1bba70ce75f160 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:40.034965 22005 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 22: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:40.041338 22557 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.044737 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.045401 21935 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 14.
I20260709 13:08:40.045553 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.046208 21928 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:40.046293 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 676bc9491c5a4083bdd0b754d904c1d6 from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
W20260709 13:08:40.046789 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 46803.8us
I20260709 13:08:40.046877 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 676bc9491c5a4083bdd0b754d904c1d6 from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-676bc9491c5a4083bdd0b754d904c1d6
I20260709 13:08:40.049779 29944 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:40.054415 21972 leader_election.cc:304] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.055644 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.056900 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:40.057744 21926 raft_consensus.cc:2393] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:40.058797 29944 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:40.059227 29944 raft_consensus.cc:493] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.059609 29944 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:40.060649 29937 raft_consensus.cc:2804] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:40.061241 29937 raft_consensus.cc:697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:40.062237 29937 consensus_queue.cc:237] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.069767 20746 tablet_replica.cc:333] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:40.070612 20746 raft_consensus.cc:2243] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:40.056641 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.078688 21944 tablet_copy_source_session.cc:215] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:08:40.080487 21972 leader_election.cc:336] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.089406 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:40.089890 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 44966.5us
I20260709 13:08:40.083225 22001 raft_consensus.cc:2468] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:40.093432 21899 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.094548 20746 raft_consensus.cc:2272] T 958f1a2d78484a3bab2de506da912756 P 8f3b35a35257416db3eaaf364cf6b52b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:40.098543 29974 raft_consensus.cc:2804] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:08:40.204267 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 107114us
W20260709 13:08:40.205013 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 106516us
W20260709 13:08:40.205955 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 106097us
W20260709 13:08:40.208312 21972 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.214936 29077 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.209121 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.215492 29077 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.217953 20746 tablet_replica.cc:333] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b: stopping tablet replica
I20260709 13:08:40.218533 20746 raft_consensus.cc:2243] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:40.220573 29983 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:40.222354 29077 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.212575 29944 raft_consensus.cc:515] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.219657 20746 raft_consensus.cc:2272] T ec2c60ea913b4b8a9b1cbfd733880203 P 8f3b35a35257416db3eaaf364cf6b52b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:40.226029 22003 raft_consensus.cc:1275] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:08:40.241977 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.242656 21898 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.242743 29963 consensus_queue.cc:1048] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:40.245501 29944 leader_election.cc:290] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:40.243952 21898 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.248188 21898 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:40.252324 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:40.252972 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.254837 22556 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.209288 29974 raft_consensus.cc:697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:40.256814 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a226c66b13a04eb389353b8e2deb175c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:40.257464 22000 raft_consensus.cc:3060] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:08:40.253472 21898 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.258728 21898 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.259300 21898 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.270102 30005 raft_consensus.cc:2749] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:40.275243 29994 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:40.275753 29994 raft_consensus.cc:493] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.276108 29994 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:40.281791 22001 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:40.284097 22000 raft_consensus.cc:2468] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 26.
W20260709 13:08:40.293336 22556 leader_election.cc:336] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.257530 29963 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:40.284103 29988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676bc9491c5a4083bdd0b754d904c1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.303642 22557 leader_election.cc:304] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:40.307626 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 99342.9us
I20260709 13:08:40.309973 22000 raft_consensus.cc:1275] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:40.314441 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.311151 30007 raft_consensus.cc:2804] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:08:40.318435 29945 consensus_queue.cc:1048] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:40.318818 30007 raft_consensus.cc:697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 26 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:40.319808 30007 consensus_queue.cc:237] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:40.325387 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.294674 29969 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
W20260709 13:08:40.329353 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:40.348791 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:40.370937 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.371488 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:40.375804 21898 consensus_peers.cc:597] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:40.382707 21898 consensus_peers.cc:597] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:40.388041 29994 raft_consensus.cc:515] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.389314 21754 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51612 (request call id 294) took 1080 ms. Trace:
I20260709 13:08:40.389700 21754 rpcz_store.cc:276] 0709 13:08:39.309364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.309534 (+   170us) service_pool.cc:224] Handling call
0709 13:08:40.389252 (+1079718us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:39.399269 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:39.399833 (+   564us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:39.399862 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:08:39.399881 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:39.400207 (+   326us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:39.400276 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:39.400307 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:39.400480 (+   173us) tablet.cc:701] Row locks acquired
0709 13:08:39.400496 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:08:39.400622 (+   126us) write_op.cc:270] Start()
0709 13:08:39.400733 (+   111us) write_op.cc:276] Timestamp: P: 1783602519400582 usec, L: 0
0709 13:08:39.400751 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:08:39.556341 (+155590us) log.cc:844] Serialized 599 byte log entry
0709 13:08:40.025522 (+469181us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cae28 00007f3b74205581 00007f3b74204d58 00007f3b742099f1 00007f3b742099a9 00007f3b74209939 00007f3b74209901 00007f3b74208b2d 000000000038f354 000000000038f199
0709 13:08:40.026950 (+  1428us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54003edda0. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b70744dbe 00007f3b7074f930 00007f3b7074f8d8 00007f3b7074f858 00007f3b7074f813 00007f3b7074ea9c 00007f3b7663a626 00007f3b76634dd8 00007f3b7078b7cb 00007f3b7078e2c0
0709 13:08:40.071778 (+ 44828us) op_driver.cc:464] REPLICATION: finished
0709 13:08:40.212068 (+140290us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b54003ed854. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b70738571 00007f3b707cc32a 00007f3b70751f14 00007f3b70751ec3 00007f3b70751e38 00007f3b70751df3 00007f3b707510dc 00007f3b7077ae16 00007f3b70762cbb 00007f3b7074d39d 00007f3b70750d75
0709 13:08:40.245417 (+ 33349us) write_op.cc:301] APPLY: starting
0709 13:08:40.245526 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:40.247067 (+  1541us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:40.247102 (+    35us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:40.311631 (+ 64529us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:40.325299 (+ 13668us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:08:40.326796 (+  1497us) write_op.cc:312] APPLY: finished
0709 13:08:40.363119 (+ 36323us) log.cc:844] Serialized 25 byte log entry
0709 13:08:40.366703 (+  3584us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:40.372300 (+  5597us) write_op.cc:454] Released schema lock
0709 13:08:40.375086 (+  2786us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:40.389146 (+ 14060us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:40.389178 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":173363,"apply.run_cpu_time_us":4767,"apply.run_wall_time_us":139562,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8313,"num_ops":1,"prepare.queue_time_us":45757,"prepare.run_cpu_time_us":43007,"prepare.run_wall_time_us":626445,"raft.queue_time_us":144339,"raft.run_cpu_time_us":49216,"raft.run_wall_time_us":225891,"replication_time_us":670850,"spinlock_wait_cycles":220084736,"thread_start_us":448007,"threads_started":4,"wal-append.queue_time_us":365350}]]}
I20260709 13:08:40.379654 29973 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:40.415221 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:40.365512 29974 consensus_queue.cc:237] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.431864 21750 catalog_manager.cc:5697] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 25 to 26, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 26 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:40.451402 29994 leader_election.cc:290] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:40.461942 29973 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.462548 29973 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.466231 29973 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.474795 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:40.475472 21928 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:40.478808 21972 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.484786 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.485539 21935 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:40.486868 21926 raft_consensus.cc:1275] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:08:40.486872 22556 leader_election.cc:336] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.493273 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.494094 21753 catalog_manager.cc:5697] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 21 to 22, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 22 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:40.495747 21753 catalog_manager.cc:5697] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 22 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 23 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:40.494776 21928 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:40.506104 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2167f46d5e1049268b9ce66dbf46f781" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:40.506733 22001 raft_consensus.cc:3060] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:40.506784 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.497552 21753 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 22 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 23 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } }
I20260709 13:08:40.510661 29973 consensus_queue.cc:1048] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:40.517681 30017 log.cc:927] Time spent T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:08:40.526713 29973 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.531652 30013 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.532296 30013 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.534401 30013 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.539453 30001 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.540097 30001 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.543625 30001 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.544762 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:40.545775 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.549023 22001 raft_consensus.cc:2468] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:40.550158 22557 leader_election.cc:304] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.545435 22587 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 14.
I20260709 13:08:40.564780 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.561690 21974 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:40.572093 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.574313 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.559149 30001 raft_consensus.cc:2804] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:40.580111 29947 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:40.529762 29973 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.583249 29973 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.569941 29988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57102 (request call id 1823) took 1138 ms. Trace:
I20260709 13:08:40.586814 29988 rpcz_store.cc:276] 0709 13:08:39.420829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.421050 (+   221us) service_pool.cc:224] Handling call
0709 13:08:39.664977 (+243927us) ts_tablet_manager.cc:934] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
0709 13:08:40.262931 (+597954us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:08:40.414245 (+151314us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:40.418928 (+  4683us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:08:40.492605 (+ 73677us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:08:40.492889 (+   284us) tablet_replica.cc:206] Creating consensus instance
0709 13:08:40.554539 (+ 61650us) ts_tablet_manager.cc:989] Replica registered.
0709 13:08:40.557904 (+  3365us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":31824,"tablet-copy.queue_time_us":205401,"thread_start_us":150936,"threads_started":2}
I20260709 13:08:40.588521 29988 tablet_copy_client.cc:806] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:40.590065 29988 tablet_copy_client.cc:670] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:40.593917 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.594749 21926 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:40.595976 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.596881 29077 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:40.597332 29077 raft_consensus.cc:493] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.597699 29077 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:40.570480 29952 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.574069 29937 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:40.596683 30005 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.610877 30005 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.613235 30005 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.613471 29952 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.615696 29952 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:40.616423 29937 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.616982 29937 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.622044 29969 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.622742 29969 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.625124 29969 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:40.626524 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:40.628068 22005 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 20.
I20260709 13:08:40.629642 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:40.630266 22000 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:40.631283 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.597957 30001 raft_consensus.cc:697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:40.640431 30001 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.643440 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.644404 29963 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:40.644809 29963 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.645198 29963 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:40.649194 29988 tablet_copy_client.cc:538] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:40.653182 29974 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:40.653677 29974 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.654044 29974 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:40.665781 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:40.666628 22587 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 22.
W20260709 13:08:40.671371 21898 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.672333 21898 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.673864 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:40.673849 21898 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.678953 29077 raft_consensus.cc:515] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.684022 29952 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:40.684636 29952 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.685115 29952 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:40.671115 30005 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.690927 30005 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.693068 29969 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.693714 29969 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.695750 29969 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.682777 29937 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.701195 30005 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.701792 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:40.708263 29077 leader_election.cc:290] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.711406 29947 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.734422 29947 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.717090 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:40.736793 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:40.737130 22587 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:40.737455 22589 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.742297 29974 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.744028 30005 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.744552 30005 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.746127 29988 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:08:40.747459 30005 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:40.748260 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.748473 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:40.748284 22000 raft_consensus.cc:1275] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:08:40.750257 30001 consensus_queue.cc:1048] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:40.736796 29947 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.750816 22589 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
W20260709 13:08:40.751216 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:40.718088 29937 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.640887 21750 catalog_manager.cc:5697] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 20 to 21, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:40.718823 29963 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.755795 29963 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:40.727936 21898 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.756677 21898 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.757021 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:40.757121 21898 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.757674 22005 raft_consensus.cc:3060] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:40.758803 29974 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:40.759394 29974 raft_consensus.cc:2749] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:40.760879 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.761813 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7024293bd494d23afe75d0322f5445b" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:40.762403 22001 raft_consensus.cc:3060] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:40.763159 29969 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:40.763581 29969 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.763991 29969 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:40.766693 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.780642 29969 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.782279 21928 raft_consensus.cc:2393] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 24: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:40.784904 22005 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:40.786684 21899 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.789239 21754 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 23 to 24, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 24 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:40.792239 22001 raft_consensus.cc:2468] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 21.
I20260709 13:08:40.793424 21899 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.794288 30005 raft_consensus.cc:2804] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:40.794919 30005 raft_consensus.cc:697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:40.795781 30005 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.797971 29991 raft_consensus.cc:2804] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:40.798542 29991 raft_consensus.cc:697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:40.799443 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "951fdd0c64f542cbab511fb32cfa91cb" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:40.799402 29991 consensus_queue.cc:237] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.800469 22589 raft_consensus.cc:3060] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:40.803511 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.805279 30001 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.805863 30001 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.808120 30001 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:40.814272 22556 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.810484 21898 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.816864 21898 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.819806 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635368d86daf4188a9dcb9cc2f31ce10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
W20260709 13:08:40.820183 21898 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.820642 21928 raft_consensus.cc:3060] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:40.820878 22589 raft_consensus.cc:2468] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
W20260709 13:08:40.821185 21898 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.823073 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:40.825577 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.826370 21900 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.826419 21926 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:40.822212 29947 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:40.830143 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.822513 29969 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.837690 30001 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:40.838264 30001 raft_consensus.cc:493] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.827705 29990 raft_consensus.cc:2749] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:40.838825 30001 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:40.842172 21928 raft_consensus.cc:2468] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 26.
I20260709 13:08:40.846773 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc728770e5c4aa1ac36a662cebb8598" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:40.847425 21928 raft_consensus.cc:3060] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.840090 29969 raft_consensus.cc:2804] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:40.849486 29969 raft_consensus.cc:697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:40.849541 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.850432 29969 consensus_queue.cc:237] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.859680 30001 raft_consensus.cc:515] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:40.863942 21898 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.874109 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:40.874717 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.875155 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.875855 21928 raft_consensus.cc:2468] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 15.
I20260709 13:08:40.877475 29077 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:40.879573 21972 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.888401 21972 leader_election.cc:304] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.889353 29077 raft_consensus.cc:2804] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:08:40.889799 29077 raft_consensus.cc:697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:40.890666 29077 consensus_queue.cc:237] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.732875 29952 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:40.910175 21972 leader_election.cc:336] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.911144 21972 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:40.912047 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:40.918646 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.931149 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71a9145a148c4edebd5d8df1c9956298" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:40.931769 22587 raft_consensus.cc:3060] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:40.933220 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.931717 29952 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.949448 21972 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.954862 29966 raft_consensus.cc:2804] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:40.963905 21898 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.972173 22587 raft_consensus.cc:2468] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 23.
I20260709 13:08:40.970731 29966 raft_consensus.cc:697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:40.976016 29966 consensus_queue.cc:237] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.973569 21900 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:40.979413 29937 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.980013 29937 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:40.981197 30005 raft_consensus.cc:2804] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:40.982174 29937 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:40.980837 21754 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 23 to 24, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 24 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:40.983667 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:40.986162 21750 catalog_manager.cc:5697] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 25 to 26, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 26 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:40.987138 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:40.987751 21928 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:40.988847 29077 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.989351 29077 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.990224 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:40.991698 29077 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:40.992453 29937 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:40.992815 29937 raft_consensus.cc:493] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:40.993229 29937 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:40.982205 30005 raft_consensus.cc:697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
W20260709 13:08:40.993871 21972 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:40.994859 30005 consensus_queue.cc:237] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:40.996168 21754 catalog_manager.cc:5697] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 20 to 21, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:40.996841 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:41.003571 21928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
W20260709 13:08:41.006424 21972 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.006911 21972 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:41.019140 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): ChangeConfig:REMOVE_PEER RPC failed for tablet a5e5e01f84ee40afb9b688bf0b446c6c: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.015856 30024 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:41.027732 21898 consensus_peers.cc:597] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:41.031390 29937 raft_consensus.cc:515] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:41.033284 22556 leader_election.cc:336] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:41.036051 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.036911 30001 leader_election.cc:290] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:41.038829 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caafc8397a4f4963abeff030a28f5c47" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:41.039438 21935 raft_consensus.cc:3060] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:41.049376 21935 raft_consensus.cc:2468] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
W20260709 13:08:41.038841 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:41.051622 29937 leader_election.cc:290] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:41.056615 22000 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:08:41.059089 22005 raft_consensus.cc:1275] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:41.060636 29963 consensus_queue.cc:1048] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:41.060665 30007 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
W20260709 13:08:41.057401 21972 leader_election.cc:336] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.052629 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db8966fba10424a8df85ad63defc925" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:41.066188 21935 raft_consensus.cc:3060] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:41.066505 22556 leader_election.cc:304] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.075627 21935 raft_consensus.cc:2468] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:41.087762 21972 leader_election.cc:304] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.073824 29947 raft_consensus.cc:2804] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:41.103483 29947 raft_consensus.cc:697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:41.104527 29947 consensus_queue.cc:237] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.096714 29989 raft_consensus.cc:2804] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:41.149367 29989 raft_consensus.cc:697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:41.150321 29989 consensus_queue.cc:237] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.157373 30024 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.157903 30024 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.163342 21754 catalog_manager.cc:5697] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 23 to 24, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 24 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.166872 21754 catalog_manager.cc:5697] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 22 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 23 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.173246 30024 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:41.173367 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.175487 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:41.177647 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.179889 30024 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:41.183166 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.183468 21935 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:08:41.197045 21744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2167f46d5e1049268b9ce66dbf46f781 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:41.204913 30001 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:08:41.199246 21750 catalog_manager.cc:5697] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 14 to 15, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 15 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:41.292455 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 72142.4us
W20260709 13:08:41.296831 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.316352 30001 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.318356 30001 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.353785 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:41.355183 22001 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25.
I20260709 13:08:41.361112 30001 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:41.378584 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 49016.1us
I20260709 13:08:41.379964 21935 raft_consensus.cc:1275] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:08:41.388687 22005 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:08:41.390684 30005 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.391605 30005 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.394238 30005 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:41.395731 30024 consensus_queue.cc:1048] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:41.398082 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:41.399037 22008 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:41.403946 29963 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
W20260709 13:08:41.405714 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.405490 29963 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.410785 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:41.411306 29963 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:41.421952 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 122322us
W20260709 13:08:41.427711 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:41.439831 29963 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:41.447098 22589 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:41.462044 22587 raft_consensus.cc:1275] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:08:41.445168 29966 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.464931 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:41.466055 29966 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.467133 29974 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:41.467700 29974 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.468134 29974 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:41.469412 29937 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:41.474936 29952 consensus_queue.cc:1048] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:08:41.487041 21972 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.498243 22587 raft_consensus.cc:1275] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:08:41.505347 29947 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.506162 30005 consensus_queue.cc:1048] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:08:41.508633 29966 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:41.506225 29947 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.506695 21750 catalog_manager.cc:5697] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 22 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 23 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.511380 29947 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:41.559654 30007 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.560714 30007 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.563578 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:41.564468 21926 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:41.563078 30007 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:41.567468 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:41.579110 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.582458 29973 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:41.582924 29973 raft_consensus.cc:493] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.583436 29973 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:41.584452 29989 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.592804 29989 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.595393 29989 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:41.600350 21932 raft_consensus.cc:1275] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:41.602715 21754 catalog_manager.cc:5697] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 21 to 22, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 22 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:41.604424 21972 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.607226 29973 raft_consensus.cc:515] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.563068 29988 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:41.618467 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:41.619381 22584 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 17.
I20260709 13:08:41.624800 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.630120 29989 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:41.630596 29989 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.631062 29989 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:41.631603 29966 consensus_queue.cc:1048] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:08:41.634865 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 82346us
I20260709 13:08:41.646199 29963 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.646889 29963 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.665421 29963 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:41.670619 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:41.671554 22005 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 24.
I20260709 13:08:41.673557 29973 leader_election.cc:290] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:41.675843 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:41.676681 30005 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:41.677109 30005 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.677524 30005 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:41.692301 29988 tablet_bootstrap.cc:492] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:08:41.693580 29988 ts_tablet_manager.cc:1403] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.948s	user 0.052s	sys 0.006s
I20260709 13:08:41.693835 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2ed3e1870148589d1bba70ce75f160" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:41.694509 21926 raft_consensus.cc:3060] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:41.695453 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:41.705088 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:41.731025 21972 leader_election.cc:336] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.735880 21935 raft_consensus.cc:1275] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:08:41.744617 29988 raft_consensus.cc:359] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:41.745759 29988 raft_consensus.cc:740] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:08:41.747071 29974 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.706037 30005 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.755759 30005 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:41.757170 22001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d830a9119594381aa253bf173c99566" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
W20260709 13:08:41.757912 22556 leader_election.cc:336] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.758044 22001 raft_consensus.cc:3060] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:41.767431 22556 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.768301 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.770318 21926 raft_consensus.cc:2468] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:41.775038 21972 leader_election.cc:304] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:41.775894 30045 consensus_queue.cc:1048] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:08:41.777505 29937 raft_consensus.cc:2804] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:41.778100 29937 raft_consensus.cc:697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:41.779201 29937 consensus_queue.cc:237] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:41.787271 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.770002 29989 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:41.790300 29989 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:41.798491 29963 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:41.812424 30007 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:41.801581 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e16208fe6d441998baf1cb8b8546d10" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:41.819718 22000 raft_consensus.cc:3060] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:41.800284 21972 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:41.823844 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:41.805433 22005 raft_consensus.cc:1275] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:41.809629 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec2c60ea913b4b8a9b1cbfd733880203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:41.826184 22589 raft_consensus.cc:3060] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:41.824599 21932 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:41.821857 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:41.795334 29974 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:41.840380 22001 raft_consensus.cc:2468] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 25.
I20260709 13:08:41.849356 29952 consensus_queue.cc:1048] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:41.747103 29988 consensus_queue.cc:260] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:41.888788 21899 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:41.897408 30047 raft_consensus.cc:2804] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:08:41.885226 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.898074 30047 raft_consensus.cc:697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 25 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:41.869518 29963 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:41.925690 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:41.927965 22589 raft_consensus.cc:2468] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 18.
I20260709 13:08:41.930107 21933 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 25.
I20260709 13:08:41.930466 30047 consensus_queue.cc:237] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:41.951015 29963 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:41.952906 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:41.969122 21900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 48090.2us
I20260709 13:08:41.960918 22005 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 25.
I20260709 13:08:41.973542 22589 raft_consensus.cc:1217] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEARNER]: Deduplicated request from leader. Original: 21.12->[21.13-21.13]   Dedup: 21.13->[]
I20260709 13:08:41.981756 21974 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:42.004050 29077 raft_consensus.cc:2804] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:42.007418 22000 raft_consensus.cc:2468] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 25.
I20260709 13:08:42.019927 21933 raft_consensus.cc:1275] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:08:42.022109 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:42.039857 29988 ts_tablet_manager.cc:1434] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.346s	user 0.011s	sys 0.000s
I20260709 13:08:42.047379 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:42.048296 29947 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:42.049175 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:42.030934 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 25.
I20260709 13:08:42.048748 29947 raft_consensus.cc:493] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:42.052886 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:42.060659 30007 consensus_queue.cc:1048] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 13:08:42.065025 21899 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
W20260709 13:08:42.082621 21898 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:42.061931 29947 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:42.091658 29979 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:42.092180 29979 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:42.092576 29979 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:42.060469 30001 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:42.109157 29077 raft_consensus.cc:697] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:42.108740 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 108573us
W20260709 13:08:42.111562 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 61082.6us
I20260709 13:08:42.113258 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-676bc9491c5a4083bdd0b754d904c1d6 received from {username='slave'} at 127.0.0.1:37776
I20260709 13:08:42.113942 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-676bc9491c5a4083bdd0b754d904c1d6 on tablet 676bc9491c5a4083bdd0b754d904c1d6 with peer 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:42.126152 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:42.110095 30005 raft_consensus.cc:2804] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:42.110517 30001 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8f3b35a35257416db3eaaf364cf6b52b)
I20260709 13:08:42.165578 29077 consensus_queue.cc:237] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 17.17, Last appended by leader: 17, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:42.180915 30001 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:42.185277 30005 raft_consensus.cc:697] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:42.207442 30005 consensus_queue.cc:237] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:42.215579 21754 catalog_manager.cc:5697] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 24 to 25, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 25 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:42.221748 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 63593.9us
W20260709 13:08:42.245082 21898 leader_election.cc:336] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:42.252413 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 82059.1us
I20260709 13:08:42.254673 29966 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.255237 29966 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:42.257251 29966 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:42.247095 29979 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:42.249409 21898 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:42.276468 29990 raft_consensus.cc:2749] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:42.271323 21898 consensus_peers.cc:597] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.287743 29979 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:42.288144 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:42.288813 22008 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:08:42.294656 21823 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1553) could send its response
I20260709 13:08:42.295383 29947 raft_consensus.cc:515] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:42.296563 21823 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1554) could send its response
W20260709 13:08:42.279084 21898 consensus_peers.cc:597] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.304611 21823 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1555) could send its response
I20260709 13:08:42.309543 22008 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 26.
W20260709 13:08:42.310370 21823 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1556) could send its response
W20260709 13:08:42.312912 21898 consensus_peers.cc:597] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.315112 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
W20260709 13:08:42.311863 21823 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42922 (request call id 1557) could send its response
I20260709 13:08:42.303668 30001 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:42.323135 22587 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 821b3831b8a348c0826bd3a20a857f00 in current term 26: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
W20260709 13:08:42.324800 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 76759.5us
I20260709 13:08:42.330976 29947 leader_election.cc:290] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:42.331384 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:42.332002 21898 consensus_peers.cc:597] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.332494 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:42.333691 21898 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:42.334522 21898 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:42.335788 21898 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:42.345901 21926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:42.346716 21926 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 26: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:42.347476 21932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6622001ac78d44caa90b4ce7f5283aa1" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:42.348143 21932 raft_consensus.cc:3060] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:42.358047 30001 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:42.359606 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:42.360684 22005 raft_consensus.cc:2393] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 26: Already voted for candidate 821b3831b8a348c0826bd3a20a857f00 in this term.
I20260709 13:08:42.362108 22557 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:42.363606 30007 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:08:42.370640 21898 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:42.375296 21898 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:42.376076 30005 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:08:42.378244 22556 connection.cc:573] client connection to 127.20.66.129:38927 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:08:42.386898 21935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00" is_pre_election: true
I20260709 13:08:42.387686 21935 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:42.392987 21932 raft_consensus.cc:2468] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 22.
I20260709 13:08:42.399106 21927 raft_consensus.cc:1275] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:08:42.402983 20746 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1549) could send its response
W20260709 13:08:42.403645 20746 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1550) could send its response
W20260709 13:08:42.404131 20746 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1551) could send its response
W20260709 13:08:42.404558 20746 connection.cc:466] server connection from 127.0.0.1:42922 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42922 (request call id 1552) could send its response
I20260709 13:08:42.409082 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:42.410373 29989 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:42.410760 29989 raft_consensus.cc:493] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.411094 29989 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:42.417052 22556 leader_election.cc:336] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:42.417963 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:42.424886 22556 leader_election.cc:304] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:42.425793 30007 raft_consensus.cc:2804] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:42.426306 30007 raft_consensus.cc:697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:42.427310 30007 consensus_queue.cc:237] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:42.417271 29077 consensus_queue.cc:1048] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:08:42.456983 29989 raft_consensus.cc:515] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:42.457731 22005 raft_consensus.cc:1275] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:08:42.463022 21972 connection.cc:769] client connection to 127.20.66.129:38927 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:08:42.466506 30005 consensus_queue.cc:1048] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:08:42.479302 21972 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:08:42.484015 21750 catalog_manager.cc:5697] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 23 to 25, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 25 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.492933 30054 raft_consensus.cc:1064] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:08:42.493368 21753 catalog_manager.cc:5697] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 21 to 22, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 22 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:42.496027 30054 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:42.475930 29989 leader_election.cc:290] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:42.521705 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:42.522742 21898 consensus_peers.cc:597] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.534381 22587 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:42.540560 29979 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:42.552816 22000 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:42.554370 29963 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:42.576439 21933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09efb42cfd6142bb9f816328042aea38" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "821b3831b8a348c0826bd3a20a857f00"
I20260709 13:08:42.577164 21933 raft_consensus.cc:3060] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:42.596190 21754 catalog_manager.cc:5697] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 24 to 25, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 25 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.612861 22592 raft_consensus.cc:1275] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 17 index: 17. Preceding OpId from leader: term: 18 index: 18. (index mismatch)
I20260709 13:08:42.618546 29973 consensus_queue.cc:1048] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:08:42.627784 21898 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.658360 21933 raft_consensus.cc:2468] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
W20260709 13:08:42.592942 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:42.660105 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 67209.2us
W20260709 13:08:42.663282 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:42.671324 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:42.672051 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:42.672677 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.706975 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:42.756464 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 64571.1us
I20260709 13:08:42.755352 21750 catalog_manager.cc:5697] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 17 to 18, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 18 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.761823 21972 leader_election.cc:304] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 
I20260709 13:08:42.764402 29937 raft_consensus.cc:2804] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:42.788004 29937 raft_consensus.cc:697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:42.816648 29937 consensus_queue.cc:237] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:42.821249 30068 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:42.827286 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:42.853778 21972 leader_election.cc:336] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:42.873050 22592 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:42.897449 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 86069.1us
I20260709 13:08:42.910851 22000 raft_consensus.cc:1275] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 13:08:42.913918 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:42.928079 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:42.928839 21898 consensus_peers.cc:597] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:42.934576 22005 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:42.952028 21753 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 13 to 14, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } attrs { replace: true } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:42.981208 30088 consensus_queue.cc:1048] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:42.984534 21754 catalog_manager.cc:5697] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 21 to 23, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 23 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.016330 21926 raft_consensus.cc:1275] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 13:08:43.061672 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:43.067459 21926 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:43.068382 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 51658.6us
I20260709 13:08:43.081236 30066 consensus_queue.cc:1048] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:08:43.100914 22587 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:43.148018 22005 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:08:43.149117 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:43.154575 30068 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:08:43.144881 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.166123 21898 consensus_peers.cc:597] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:43.167435 30068 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:43.184767 22005 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:43.182495 30092 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:43.213538 22587 raft_consensus.cc:2955] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:43.270766 21753 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 14 to 15, VOTER 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) evicted. New cstate: current_term: 21 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:43.287838 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.301496 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:43.308418 21742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 174123us
I20260709 13:08:43.314755 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 676bc9491c5a4083bdd0b754d904c1d6 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:08:43.327486 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:43.402135 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:43.430783 21935 raft_consensus.cc:1275] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:08:43.434083 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:43.435842 30070 consensus_queue.cc:1048] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:08:43.454288 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.463896 20746 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21881)
W20260709 13:08:43.566088 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.571498 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.661429 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:43.683600 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.724712 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.761065 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.765149 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:43.765861 21898 consensus_peers.cc:597] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:43.923519 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:43.929888 21898 consensus_peers.cc:597] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:43.964123 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:44.011960 29979 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.012651 29979 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:44.015645 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:44.020732 21898 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:44.023517 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:44.024186 22005 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 26.
I20260709 13:08:44.026199 29979 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:44.027819 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:44.028520 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 26.
I20260709 13:08:44.031885 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:44.032704 30092 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:08:44.033111 30092 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:44.033490 30092 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:44.044006 30092 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:44.055392 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.057046 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:44.092885 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:44.093591 22000 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:44.106036 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e5e01f84ee40afb9b688bf0b446c6c" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:44.106873 22000 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 27.
I20260709 13:08:44.107987 22587 raft_consensus.cc:3060] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:44.102234 30092 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 27 election: Requested vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:44.124770 21898 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:44.128818 22587 raft_consensus.cc:2468] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 821b3831b8a348c0826bd3a20a857f00 in term 27.
I20260709 13:08:44.130566 21900 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00; no voters: 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:44.133026 30068 raft_consensus.cc:2804] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:08:44.133584 30068 raft_consensus.cc:697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 27 LEADER]: Becoming Leader. State: Replica: 821b3831b8a348c0826bd3a20a857f00, State: Running, Role: LEADER
I20260709 13:08:44.134622 30068 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:44.166244 21754 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: term changed from 25 to 27, leader changed from 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129) to 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130). New cstate: current_term: 27 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:44.239740 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.351918 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:44.400833 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.464439 20746 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 21881)
W20260709 13:08:44.727686 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:44.752564 21972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111) [suppressed 417 similar messages]
W20260709 13:08:44.799885 21898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 431415us
W20260709 13:08:44.859740 21742 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 48891us
W20260709 13:08:44.890450 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:44.896018 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:44.896864 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:44.897650 21898 consensus_peers.cc:597] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.898936 21898 consensus_peers.cc:597] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.902173 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:44.938457 21898 consensus_peers.cc:597] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.939026 21898 consensus_peers.cc:597] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.941790 21898 consensus_peers.cc:597] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.944156 21898 consensus_peers.cc:597] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:44.944180 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:44.965606 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:44.967980 22584 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:08:44.987604 30106 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:44.994508 22007 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:08:45.007699 30092 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:45.086334 22001 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:08:45.124790 21974 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ed3a64e6b0242fa9fda4766cc68016b. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.152532 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:45.155431 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.157200 21926 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 13:08:45.170517 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:45.179236 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:45.180707 30118 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:45.203111 30118 raft_consensus.cc:2955] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 LEADER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.213055 21926 raft_consensus.cc:2955] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.255131 21754 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 17, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 28 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:08:45.313364 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:45.325520 22584 consensus_queue.cc:237] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:45.352147 21928 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:45.373237 22001 raft_consensus.cc:1275] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:08:45.377823 30116 consensus_queue.cc:1048] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:08:45.387952 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:45.395071 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.410923 22001 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:08:45.412463 30108 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:08:45.414458 21900 consensus_peers.cc:597] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 309ab90abff84c36a30bd5a2a0363d2c. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.421430 21898 consensus_peers.cc:597] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.430222 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:45.462385 21956 debug-util.cc:398] Leaking SignalData structure 0x7b08001914e0 after lost signal to thread 20749
W20260709 13:08:45.464048 21956 debug-util.cc:398] Leaking SignalData structure 0x7b08002ccdc0 after lost signal to thread 21792
W20260709 13:08:45.465196 20746 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 21881)
W20260709 13:08:45.465525 21956 debug-util.cc:398] Leaking SignalData structure 0x7b08000942a0 after lost signal to thread 21959
W20260709 13:08:45.466374 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:45.466714 21956 debug-util.cc:398] Leaking SignalData structure 0x7b08000a91a0 after lost signal to thread 22033
W20260709 13:08:45.468011 21956 debug-util.cc:398] Leaking SignalData structure 0x7b0800145660 after lost signal to thread 22617
W20260709 13:08:45.477995 21898 consensus_peers.cc:597] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:45.487429 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:45.492986 30066 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index 12 to 16, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:45.499646 22001 raft_consensus.cc:2955] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 12 to 16, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:45.506623 21744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2167f46d5e1049268b9ce66dbf46f781 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:08:45.514003 30108 raft_consensus.cc:2955] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index -1 to 16, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.516185 22005 raft_consensus.cc:2955] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index -1 to 16, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.558326 21750 catalog_manager.cc:5697] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 12 to 16, VOTER 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) evicted. New cstate: current_term: 24 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:45.562915 22001 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
W20260709 13:08:45.564577 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:45.565222 21753 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 16, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 24 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:45.634336 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:45.646859 21920 tablet_service.cc:1558] Processing DeleteTablet for tablet 2167f46d5e1049268b9ce66dbf46f781 with delete_type TABLET_DATA_TOMBSTONED (TS 821b3831b8a348c0826bd3a20a857f00 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:41410
I20260709 13:08:45.699288 21928 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:08:45.704380 30107 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:08:45.705761 21974 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8594e4ce37964bf09c89143bed59f40e. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.709190 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.754732 30138 ts_tablet_manager.cc:933] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:45.754714 21898 consensus_peers.cc:597] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:45.769879 30137 tablet_replica.cc:333] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:45.771143 30137 raft_consensus.cc:2243] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.771826 30137 raft_consensus.cc:2272] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.782835 30138 tablet_copy_client.cc:323] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
W20260709 13:08:45.785473 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:45.784960 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 6ed3a64e6b0242fa9fda4766cc68016b from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:08:45.786412 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 6ed3a64e6b0242fa9fda4766cc68016b from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-6ed3a64e6b0242fa9fda4766cc68016b
I20260709 13:08:45.794394 30107 raft_consensus.cc:2955] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Committing config change with OpId 25.14: config changed from index -1 to 14, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.807020 21928 raft_consensus.cc:2955] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Committing config change with OpId 25.14: config changed from index -1 to 14, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.816061 30137 ts_tablet_manager.cc:1916] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:45.821712 22019 tablet_copy_source_session.cc:215] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:45.828461 30138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3a64e6b0242fa9fda4766cc68016b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:45.852375 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:45.854650 21754 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 14, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 25 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:45.889221 21927 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:45.928359 30137 ts_tablet_manager.cc:1929] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.15
I20260709 13:08:45.928906 30137 log.cc:1199] T 2167f46d5e1049268b9ce66dbf46f781 P 821b3831b8a348c0826bd3a20a857f00: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/wals/2167f46d5e1049268b9ce66dbf46f781
I20260709 13:08:45.933151 30138 tablet_copy_client.cc:806] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:45.933812 30138 tablet_copy_client.cc:670] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:45.947242 30138 tablet_copy_client.cc:538] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:45.949412 21742 catalog_manager.cc:5028] TS 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): tablet 2167f46d5e1049268b9ce66dbf46f781 (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:08:45.977645 30142 ts_tablet_manager.cc:933] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:46.008307 22005 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:08:46.009101 21900 consensus_peers.cc:597] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099): Couldn't send request to peer 34f3b83da84448bf8aac62b37a1486eb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 873e001f77374da4a034a00eaaea672f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:46.010526 30104 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:46.015666 30142 tablet_copy_client.cc:323] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
W20260709 13:08:46.018306 21898 consensus_peers.cc:597] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:46.021975 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 309ab90abff84c36a30bd5a2a0363d2c from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:08:46.022591 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 309ab90abff84c36a30bd5a2a0363d2c from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-309ab90abff84c36a30bd5a2a0363d2c
W20260709 13:08:46.029162 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:46.034883 21944 tablet_copy_source_session.cc:215] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:46.039538 30142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309ab90abff84c36a30bd5a2a0363d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:46.052330 30092 raft_consensus.cc:2955] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
I20260709 13:08:46.054428 22005 raft_consensus.cc:2955] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } } }
W20260709 13:08:46.083029 21742 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 45133.8us
I20260709 13:08:46.086903 21754 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index -1 to 15, NON_VOTER 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) added. New cstate: current_term: 23 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.090780 30138 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
W20260709 13:08:46.098630 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:46.136956 30142 tablet_copy_client.cc:806] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:46.153254 30142 tablet_copy_client.cc:670] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
W20260709 13:08:46.154320 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:46.161093 30142 tablet_copy_client.cc:538] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:46.172528 30142 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
W20260709 13:08:46.226361 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:46.227303 21898 consensus_peers.cc:597] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:46.264155 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:46.273929 30148 ts_tablet_manager.cc:933] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:46.290629 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:46.304950 30148 tablet_copy_client.cc:323] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.131:36521
I20260709 13:08:46.305367 30138 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:46.306298 30138 tablet_bootstrap.cc:492] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:08:46.307092 30138 ts_tablet_manager.cc:1403] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.260s	user 0.051s	sys 0.008s
I20260709 13:08:46.307135 22019 tablet_copy_service.cc:140] P 7d07aa0dcdd1484ab87925c9a4604b81: Received BeginTabletCopySession request for tablet 8594e4ce37964bf09c89143bed59f40e from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:32904)
I20260709 13:08:46.307905 22019 tablet_copy_service.cc:161] P 7d07aa0dcdd1484ab87925c9a4604b81: Beginning new tablet copy session on tablet 8594e4ce37964bf09c89143bed59f40e from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:32904: session id = 34f3b83da84448bf8aac62b37a1486eb-8594e4ce37964bf09c89143bed59f40e
W20260709 13:08:46.307279 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:46.316429 22019 tablet_copy_source_session.cc:215] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:46.320389 30138 raft_consensus.cc:359] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:46.321327 30138 raft_consensus.cc:740] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:08:46.322039 30138 consensus_queue.cc:260] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:46.324179 30148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8594e4ce37964bf09c89143bed59f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:46.327186 30138 ts_tablet_manager.cc:1434] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:08:46.338812 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-6ed3a64e6b0242fa9fda4766cc68016b received from {username='slave'} at 127.0.0.1:32904
I20260709 13:08:46.339313 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-6ed3a64e6b0242fa9fda4766cc68016b on tablet 6ed3a64e6b0242fa9fda4766cc68016b with peer 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:46.351254 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:46.442155 30142 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:46.443157 30142 tablet_bootstrap.cc:492] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:08:46.443877 30142 ts_tablet_manager.cc:1403] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.272s	user 0.042s	sys 0.014s
W20260709 13:08:46.448583 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:46.457144 30148 tablet_copy_client.cc:806] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:46.457834 30148 tablet_copy_client.cc:670] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:46.461650 30148 tablet_copy_client.cc:538] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:46.463804 30142 raft_consensus.cc:359] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:46.464869 30142 raft_consensus.cc:740] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
W20260709 13:08:46.466017 20746 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 21881)
I20260709 13:08:46.465621 30142 consensus_queue.cc:260] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:46.498086 30148 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
I20260709 13:08:46.507053 30142 ts_tablet_manager.cc:1434] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.001s
I20260709 13:08:46.509030 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-309ab90abff84c36a30bd5a2a0363d2c received from {username='slave'} at 127.0.0.1:37776
I20260709 13:08:46.509495 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-309ab90abff84c36a30bd5a2a0363d2c on tablet 309ab90abff84c36a30bd5a2a0363d2c with peer 34f3b83da84448bf8aac62b37a1486eb
W20260709 13:08:46.526648 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:46.563874 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:46.580708 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:46.584213 30142 ts_tablet_manager.cc:933] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: Initiating tablet copy from peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:46.597523 22584 raft_consensus.cc:1217] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 LEARNER]: Deduplicated request from leader. Original: 28.16->[28.17-28.17]   Dedup: 28.17->[]
I20260709 13:08:46.642794 30142 tablet_copy_client.cc:323] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.66.130:45191
I20260709 13:08:46.644534 21944 tablet_copy_service.cc:140] P 821b3831b8a348c0826bd3a20a857f00: Received BeginTabletCopySession request for tablet 873e001f77374da4a034a00eaaea672f from peer 34f3b83da84448bf8aac62b37a1486eb ({username='slave'} at 127.0.0.1:37776)
I20260709 13:08:46.645138 21944 tablet_copy_service.cc:161] P 821b3831b8a348c0826bd3a20a857f00: Beginning new tablet copy session on tablet 873e001f77374da4a034a00eaaea672f from peer 34f3b83da84448bf8aac62b37a1486eb at {username='slave'} at 127.0.0.1:37776: session id = 34f3b83da84448bf8aac62b37a1486eb-873e001f77374da4a034a00eaaea672f
I20260709 13:08:46.656366 21944 tablet_copy_source_session.cc:215] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:46.662108 30142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 873e001f77374da4a034a00eaaea672f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:46.673439 30148 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=3 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:46.674563 30148 tablet_bootstrap.cc:492] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:08:46.680483 30148 ts_tablet_manager.cc:1403] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.183s	user 0.058s	sys 0.003s
I20260709 13:08:46.684801 30148 raft_consensus.cc:359] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:46.690596 30148 raft_consensus.cc:740] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:08:46.691367 30148 consensus_queue.cc:260] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:46.704488 30148 ts_tablet_manager.cc:1434] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:08:46.706283 22019 tablet_copy_service.cc:342] P 7d07aa0dcdd1484ab87925c9a4604b81: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-8594e4ce37964bf09c89143bed59f40e received from {username='slave'} at 127.0.0.1:32904
I20260709 13:08:46.706673 22019 tablet_copy_service.cc:434] P 7d07aa0dcdd1484ab87925c9a4604b81: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-8594e4ce37964bf09c89143bed59f40e on tablet 8594e4ce37964bf09c89143bed59f40e with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:46.726929 30142 tablet_copy_client.cc:806] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:46.732046 30142 tablet_copy_client.cc:670] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:46.748620 30142 tablet_copy_client.cc:538] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:46.778694 30142 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap starting.
W20260709 13:08:46.867169 21898 consensus_peers.cc:597] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:46.867811 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:46.874476 22584 raft_consensus.cc:1217] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 LEARNER]: Deduplicated request from leader. Original: 25.13->[25.14-25.14]   Dedup: 25.14->[]
W20260709 13:08:46.892314 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:46.960156 22587 raft_consensus.cc:1217] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEARNER]: Deduplicated request from leader. Original: 24.15->[24.16-24.16]   Dedup: 24.16->[]
I20260709 13:08:47.036376 30142 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.038703 30142 tablet_bootstrap.cc:492] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: Bootstrap complete.
I20260709 13:08:47.039404 30142 ts_tablet_manager.cc:1403] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: Time spent bootstrapping tablet: real 0.261s	user 0.048s	sys 0.003s
W20260709 13:08:47.057400 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:47.069275 30142 raft_consensus.cc:359] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:47.070230 30142 raft_consensus.cc:740] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Initialized, Role: LEARNER
I20260709 13:08:47.070911 30142 consensus_queue.cc:260] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: NON_VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: true } }
I20260709 13:08:47.087626 30107 raft_consensus.cc:1064] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
W20260709 13:08:47.097218 21898 consensus_peers.cc:597] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:47.100005 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.102958 30107 consensus_queue.cc:237] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 15, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:47.106776 30142 ts_tablet_manager.cc:1434] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.001s
I20260709 13:08:47.110236 21944 tablet_copy_service.cc:342] P 821b3831b8a348c0826bd3a20a857f00: Request end of tablet copy session 34f3b83da84448bf8aac62b37a1486eb-873e001f77374da4a034a00eaaea672f received from {username='slave'} at 127.0.0.1:37776
I20260709 13:08:47.110684 21944 tablet_copy_service.cc:434] P 821b3831b8a348c0826bd3a20a857f00: ending tablet copy session 34f3b83da84448bf8aac62b37a1486eb-873e001f77374da4a034a00eaaea672f on tablet 873e001f77374da4a034a00eaaea672f with peer 34f3b83da84448bf8aac62b37a1486eb
I20260709 13:08:47.138685 22587 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 13:08:47.142748 30107 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:08:47.157632 21898 consensus_peers.cc:597] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.163295 21926 raft_consensus.cc:1275] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 13:08:47.173285 30070 consensus_queue.cc:1048] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:08:47.194065 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:47.196043 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:47.194674 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:47.202031 21898 consensus_peers.cc:597] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.210917 22589 raft_consensus.cc:1217] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
W20260709 13:08:47.245471 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:47.250283 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:47.258167 30107 raft_consensus.cc:2955] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.267066 21927 raft_consensus.cc:2955] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.277104 22587 raft_consensus.cc:2955] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.302284 21753 catalog_manager.cc:5697] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 17 to 18, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:47.328478 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:47.339068 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:47.346472 30117 raft_consensus.cc:1064] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:08:47.358753 30117 consensus_queue.cc:237] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 12, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:47.379642 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.388979 22587 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 LEARNER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:08:47.391122 21927 raft_consensus.cc:1275] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:08:47.391178 30117 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 13:08:47.416352 21898 consensus_peers.cc:597] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.436992 30117 consensus_queue.cc:1048] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:08:47.458715 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:47.466495 20746 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 21881)
I20260709 13:08:47.476327 21926 consensus_queue.cc:237] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:47.483768 21898 consensus_peers.cc:597] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.501858 30160 raft_consensus.cc:1064] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:08:47.513441 30160 consensus_queue.cc:237] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:47.601255 30107 raft_consensus.cc:2955] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Committing config change with OpId 25.15: config changed from index 14 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:47.610607 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:47.617429 21927 raft_consensus.cc:2955] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index 14 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.637907 22589 raft_consensus.cc:1275] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:08:47.641748 22005 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:47.643745 30104 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:08:47.647619 21898 consensus_peers.cc:597] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:47.647111 22587 raft_consensus.cc:2955] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index 14 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:47.652758 21898 consensus_peers.cc:597] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:47.656155 30104 consensus_queue.cc:1048] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:47.664139 21750 catalog_manager.cc:5697] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: config changed from index 14 to 15, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:47.671972 22584 raft_consensus.cc:1275] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:47.684684 30104 consensus_queue.cc:1048] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:08:47.714174 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:47.723273 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:47.772382 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 68317.7us
W20260709 13:08:47.805570 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:47.806617 30157 raft_consensus.cc:1064] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: attempting to promote NON_VOTER 34f3b83da84448bf8aac62b37a1486eb to VOTER
I20260709 13:08:47.809888 30157 consensus_queue.cc:237] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:47.823056 22593 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 4 to 17, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.795480 30104 raft_consensus.cc:2955] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 27 LEADER]: Committing config change with OpId 27.17: config changed from index 4 to 17, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.859494 30156 raft_consensus.cc:2955] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Committing config change with OpId 24.17: config changed from index 16 to 17, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.872629 21742 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a5e5e01f84ee40afb9b688bf0b446c6c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:08:47.887771 22586 raft_consensus.cc:2955] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 16 to 17, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.906629 21753 catalog_manager.cc:5697] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 4 to 17, VOTER 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131) evicted. New cstate: current_term: 27 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:47.914860 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:47.930480 21995 tablet_service.cc:1558] Processing DeleteTablet for tablet a5e5e01f84ee40afb9b688bf0b446c6c with delete_type TABLET_DATA_TOMBSTONED (TS 7d07aa0dcdd1484ab87925c9a4604b81 not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:49448
W20260709 13:08:47.945228 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:47.953197 22000 raft_consensus.cc:2955] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 16 to 17, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:47.958424 22001 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:08:47.960204 30156 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:47.968674 30182 tablet_replica.cc:333] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:47.970091 22586 raft_consensus.cc:1275] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEARNER]: Refusing update from remote peer 821b3831b8a348c0826bd3a20a857f00: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:08:47.973172 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:47.973757 21898 consensus_peers.cc:597] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:47.970082 30182 raft_consensus.cc:2243] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:47.977800 30182 raft_consensus.cc:2272] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:47.978200 30173 consensus_queue.cc:1048] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:08:47.987154 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:47.989085 21750 catalog_manager.cc:5697] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: config changed from index 16 to 17, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:47.998381 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:48.001564 30182 ts_tablet_manager.cc:1916] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:48.075091 30173 raft_consensus.cc:2955] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:48.089383 22586 raft_consensus.cc:2955] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
W20260709 13:08:48.106863 21898 consensus_peers.cc:597] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:48.115398 30182 ts_tablet_manager.cc:1929] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.16
W20260709 13:08:48.118695 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:48.128255 30182 log.cc:1199] T a5e5e01f84ee40afb9b688bf0b446c6c P 7d07aa0dcdd1484ab87925c9a4604b81: Deleting WAL directory at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/wals/a5e5e01f84ee40afb9b688bf0b446c6c
I20260709 13:08:48.130813 21743 catalog_manager.cc:5028] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): tablet a5e5e01f84ee40afb9b688bf0b446c6c (table test-workload [id=39a3ebfd1e154da8aa100f69cbe06c7a]) successfully deleted
I20260709 13:08:48.142861 22000 raft_consensus.cc:2955] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } } }
I20260709 13:08:48.165285 21753 catalog_manager.cc:5697] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 reported cstate change: config changed from index 15 to 16, 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "821b3831b8a348c0826bd3a20a857f00" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:48.168253 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:48.235563 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:48.248296 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:48.343925 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:48.350384 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:48.394552 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:48.607599 21898 consensus_peers.cc:597] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:48.652293 21898 consensus_peers.cc:597] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:48.861737 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:48.870306 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:48.919996 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.087424 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.107088 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.107373 21898 consensus_peers.cc:597] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.131102 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43729.2us
W20260709 13:08:49.236614 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.271647 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.278887 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:49.294936 21898 consensus_peers.cc:597] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.379125 21898 consensus_peers.cc:597] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.491832 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.520924 21898 consensus_peers.cc:597] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:49.531915 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
W20260709 13:08:49.538326 21898 consensus_peers.cc:597] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.649850 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:49.671808 21898 consensus_peers.cc:597] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.695995 21899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 67041.5us
W20260709 13:08:49.724272 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:49.736914 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:49.753400 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.759333 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.779953 22556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111) [suppressed 489 similar messages]
W20260709 13:08:49.840657 21898 consensus_peers.cc:597] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.847600 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.856879 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:49.933992 21898 consensus_peers.cc:597] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:49.989995 21898 consensus_peers.cc:597] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:49.990553 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:50.102761 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:50.127691 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
W20260709 13:08:50.147943 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.162683 21972 consensus_peers.cc:597] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.198100 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.201954 22556 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.203552 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:50.204429 21972 consensus_peers.cc:597] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.217324 22556 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.242497 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:50.243198 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:50.265897 21898 consensus_peers.cc:597] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:50.270349 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.277956 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.298259 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.321127 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.321797 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:50.325366 21898 consensus_peers.cc:597] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:50.325431 21972 consensus_peers.cc:597] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.330701 21898 consensus_peers.cc:597] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:50.332180 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.337492 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:50.345893 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.352061 21898 consensus_peers.cc:597] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:50.390369 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:50.404750 20746 tablet_replica.cc:333] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.406454 20746 raft_consensus.cc:2243] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:08:50.407577 21972 consensus_peers.cc:597] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.407763 20746 raft_consensus.cc:2272] T d059242e679f4cbaab1cc83ba8353d60 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.433322 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:50.450771 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.462096 20746 tablet_replica.cc:333] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.472111 20746 raft_consensus.cc:2243] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:08:50.473296 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.475075 20746 raft_consensus.cc:2272] T 873e001f77374da4a034a00eaaea672f P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.479908 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.486439 20746 tablet_replica.cc:333] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
W20260709 13:08:50.496107 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.488231 20746 raft_consensus.cc:2243] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.502485 20746 raft_consensus.cc:2272] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.509835 20746 tablet_replica.cc:333] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.521591 20746 raft_consensus.cc:2243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.524379 20746 raft_consensus.cc:2272] T 0b8c4da9f5144e2b8007c8ffc563a584 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.530365 20746 tablet_replica.cc:333] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
W20260709 13:08:50.530634 22556 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.534207 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.540422 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.547351 20746 raft_consensus.cc:2243] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.549065 20746 raft_consensus.cc:2272] T 430edce57310489986e4f739148461a6 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.552205 20746 tablet_replica.cc:333] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.553241 20746 raft_consensus.cc:2243] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.554070 20746 raft_consensus.cc:2272] T 635368d86daf4188a9dcb9cc2f31ce10 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.560184 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.566639 20746 tablet_replica.cc:333] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.571712 20746 raft_consensus.cc:2243] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 25 LEADER]: Raft consensus shutting down.
W20260709 13:08:50.570518 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:50.578480 20746 raft_consensus.cc:2272] T 1d830a9119594381aa253bf173c99566 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.581350 20746 tablet_replica.cc:333] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.594862 20746 raft_consensus.cc:2243] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.599610 20746 raft_consensus.cc:2272] T 40a1bb4717e14331a67545a343ce1d87 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.604687 20746 tablet_replica.cc:333] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.607064 20746 raft_consensus.cc:2243] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.611388 20746 raft_consensus.cc:2272] T 94e8056fad5248118e00e80f8a81538c P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.615010 20746 tablet_replica.cc:333] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
W20260709 13:08:50.638298 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.647581 20746 raft_consensus.cc:2243] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.649796 20746 raft_consensus.cc:2272] T 676bc9491c5a4083bdd0b754d904c1d6 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.677917 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.691663 20746 tablet_replica.cc:333] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.697695 20746 raft_consensus.cc:2243] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.700268 20746 raft_consensus.cc:2272] T 10076817fc0f46b4943de726ac0b508c P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.702947 21898 consensus_peers.cc:597] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:50.713179 20746 tablet_replica.cc:333] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.714174 20746 raft_consensus.cc:2243] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.714993 20746 raft_consensus.cc:2272] T 8594e4ce37964bf09c89143bed59f40e P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.719841 20746 tablet_replica.cc:333] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.720671 20746 raft_consensus.cc:2243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.721349 20746 raft_consensus.cc:2272] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.725311 20746 tablet_replica.cc:333] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.726749 20746 raft_consensus.cc:2243] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.728112 20746 raft_consensus.cc:2272] T f24c37d4e05d4b7eb71e234ac493bff5 P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.734892 20746 tablet_replica.cc:333] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.735826 20746 raft_consensus.cc:2243] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.736644 20746 raft_consensus.cc:2272] T 6ed3a64e6b0242fa9fda4766cc68016b P 821b3831b8a348c0826bd3a20a857f00 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.753779 21898 consensus_peers.cc:597] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:50.754374 21898 consensus_peers.cc:597] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:50.765565 20746 tablet_replica.cc:333] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.766563 20746 raft_consensus.cc:2243] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.775517 20746 raft_consensus.cc:2272] T 739311613bd748d68267b6f4478680c7 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.783655 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:50.786530 20746 tablet_replica.cc:333] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.787472 20746 raft_consensus.cc:2243] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.788275 20746 raft_consensus.cc:2272] T 634269b557d04cbe87cfab961aaf3f91 P 821b3831b8a348c0826bd3a20a857f00 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.791183 20746 tablet_replica.cc:333] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.792097 20746 raft_consensus.cc:2243] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.793428 20746 raft_consensus.cc:2272] T 4458afcc40954b359ea857cddbfdd681 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.798697 20746 tablet_replica.cc:333] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.799870 20746 raft_consensus.cc:2243] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:08:50.809649 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:50.812453 20746 raft_consensus.cc:2272] T 81cd5c4a1eef4004b3f1a9728e83a459 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.830655 20746 tablet_replica.cc:333] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.831583 20746 raft_consensus.cc:2243] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.841902 20746 raft_consensus.cc:2272] T 09efb42cfd6142bb9f816328042aea38 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.847527 20746 tablet_replica.cc:333] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.848448 20746 raft_consensus.cc:2243] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.849978 20746 raft_consensus.cc:2272] T 90e34f5775b8400e8e9bb1a13ff27479 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.861433 20746 tablet_replica.cc:333] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.863883 20746 raft_consensus.cc:2243] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.868032 20746 raft_consensus.cc:2272] T f6eae3eb9a4941fc88763a5da4ee7638 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.887714 20746 tablet_replica.cc:333] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.888631 20746 raft_consensus.cc:2243] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.889395 20746 raft_consensus.cc:2272] T b42af043129e482498af1e2c3882a042 P 821b3831b8a348c0826bd3a20a857f00 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.893055 20746 tablet_replica.cc:333] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.898336 20746 raft_consensus.cc:2243] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.902895 20746 raft_consensus.cc:2272] T c96543776baa4cdda2e6862202de4e52 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.905843 20746 tablet_replica.cc:333] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
W20260709 13:08:50.909432 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:50.914055 20746 raft_consensus.cc:2243] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.915680 20746 raft_consensus.cc:2272] T a5e5e01f84ee40afb9b688bf0b446c6c P 821b3831b8a348c0826bd3a20a857f00 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.927922 20746 tablet_replica.cc:333] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.928723 20746 raft_consensus.cc:2243] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.929294 20746 raft_consensus.cc:2272] T 6622001ac78d44caa90b4ce7f5283aa1 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.938673 20746 tablet_replica.cc:333] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.945335 20746 raft_consensus.cc:2243] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.948051 20746 raft_consensus.cc:2272] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 821b3831b8a348c0826bd3a20a857f00 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.963306 20746 tablet_replica.cc:333] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.977268 20746 raft_consensus.cc:2243] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.978680 20746 raft_consensus.cc:2272] T f938584aad7a4b76be81dca03c9a647f P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.982923 20746 tablet_replica.cc:333] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:50.993731 20746 raft_consensus.cc:2243] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.995445 20746 raft_consensus.cc:2272] T 309ab90abff84c36a30bd5a2a0363d2c P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.001986 20746 tablet_replica.cc:333] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.002878 20746 raft_consensus.cc:2243] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.003556 20746 raft_consensus.cc:2272] T 1db8966fba10424a8df85ad63defc925 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.006294 20746 tablet_replica.cc:333] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.014153 20746 raft_consensus.cc:2243] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.021471 20746 raft_consensus.cc:2272] T 6f466d510c1b46d481ab83686b150d61 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.030484 20746 tablet_replica.cc:333] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.031386 20746 raft_consensus.cc:2243] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.033068 20746 raft_consensus.cc:2272] T f7024293bd494d23afe75d0322f5445b P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.035974 20746 tablet_replica.cc:333] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.036963 20746 raft_consensus.cc:2243] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.038237 20746 raft_consensus.cc:2272] T 8a76564a0ee84b74a3af01c79300d4b7 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.041461 20746 tablet_replica.cc:333] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.042418 20746 raft_consensus.cc:2243] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.043067 20746 raft_consensus.cc:2272] T 9fc728770e5c4aa1ac36a662cebb8598 P 821b3831b8a348c0826bd3a20a857f00 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.045693 20746 tablet_replica.cc:333] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.046484 20746 raft_consensus.cc:2243] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.048154 20746 raft_consensus.cc:2272] T 71a9145a148c4edebd5d8df1c9956298 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.050995 20746 tablet_replica.cc:333] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.080518 20746 raft_consensus.cc:2243] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.081486 20746 raft_consensus.cc:2272] T 7d337058bd1f4df1bd938c43960820be P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.084223 20746 tablet_replica.cc:333] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.085083 20746 raft_consensus.cc:2243] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.086262 20746 raft_consensus.cc:2272] T 2ee2529af5954101bde1189a694cafa5 P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.091703 20746 tablet_replica.cc:333] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.092653 20746 raft_consensus.cc:2243] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.094381 20746 raft_consensus.cc:2272] T 951fdd0c64f542cbab511fb32cfa91cb P 821b3831b8a348c0826bd3a20a857f00 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.097338 20746 tablet_replica.cc:333] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.098209 20746 raft_consensus.cc:2243] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.098938 20746 raft_consensus.cc:2272] T 0103ac417f394657ac6cbe827df3b844 P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.104811 20746 tablet_replica.cc:333] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.106889 20746 raft_consensus.cc:2243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.108171 20746 raft_consensus.cc:2272] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 821b3831b8a348c0826bd3a20a857f00 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.112357 20746 tablet_replica.cc:333] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.113289 20746 raft_consensus.cc:2243] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.114085 20746 raft_consensus.cc:2272] T 958f1a2d78484a3bab2de506da912756 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.117683 20746 tablet_replica.cc:333] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.118558 20746 raft_consensus.cc:2243] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.119275 20746 raft_consensus.cc:2272] T caafc8397a4f4963abeff030a28f5c47 P 821b3831b8a348c0826bd3a20a857f00 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.121927 20746 tablet_replica.cc:333] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.122717 20746 raft_consensus.cc:2243] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.124619 20746 raft_consensus.cc:2272] T bc33f94111684d68a49eb643f8be2203 P 821b3831b8a348c0826bd3a20a857f00 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.127456 30157 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:08:51.127748 20746 tablet_replica.cc:333] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.128008 30157 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.129966 20746 raft_consensus.cc:2243] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.130916 20746 raft_consensus.cc:2272] T 8f2ed3e1870148589d1bba70ce75f160 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.131028 30157 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:51.134796 22000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:51.146889 20746 tablet_replica.cc:333] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.147907 20746 raft_consensus.cc:2243] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.143151 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bcf30627084357a4745445324a2430" candidate_uuid: "821b3831b8a348c0826bd3a20a857f00" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:51.148630 20746 raft_consensus.cc:2272] T 1e33356352ee4f06be9c7f962467110c P 821b3831b8a348c0826bd3a20a857f00 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.149958 21900 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 821b3831b8a348c0826bd3a20a857f00; no voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81
I20260709 13:08:51.151199 30157 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:51.151732 20746 tablet_replica.cc:333] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.152676 20746 raft_consensus.cc:2243] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:51.153977 20746 raft_consensus.cc:2272] T 7e16208fe6d441998baf1cb8b8546d10 P 821b3831b8a348c0826bd3a20a857f00 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:51.153165 22557 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36521, user_credentials={real_user=slave}} blocked reactor thread for 69468.2us
I20260709 13:08:51.163461 20746 tablet_replica.cc:333] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.166428 20746 raft_consensus.cc:2243] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.167246 20746 raft_consensus.cc:2272] T 2a91310e68464d64aa8324092182518f P 821b3831b8a348c0826bd3a20a857f00 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:51.175518 20746 tablet_replica.cc:333] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00: stopping tablet replica
I20260709 13:08:51.186802 20746 raft_consensus.cc:2243] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:51.188038 20746 raft_consensus.cc:2272] T d2bcf30627084357a4745445324a2430 P 821b3831b8a348c0826bd3a20a857f00 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:51.547277 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:51.584241 30189 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.584231 30070 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.586171 30070 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.586105 30189 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.589757 30070 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:51.593981 30189 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:51.615725 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:51.617120 30070 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.632102 30070 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:51.634977 30070 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:51.630882 21972 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:51.655689 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:51.685397 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:51.690428 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:51.693732 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:51.695148 30070 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:51.689282 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:51.696094 22556 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:51.697225 30189 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:51.734335 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:51.767503 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:51.768585 30189 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.769546 30189 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.774051 30189 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:51.776844 22556 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
==================
WARNING: ThreadSanitizer: data race (pid=20746)
  Write of size 4 at 0x7b40001690d8 by main thread:
    #0 kudu::DiagnosticSocket::Close() /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:96:7 (libkudu_util.so+0x49fe03)
    #1 kudu::rpc::AcceptorPool::Shutdown() /apache/dev/git/kudu/src/kudu/rpc/acceptor_pool.cc:212:3 (libkrpc.so+0x12ad33)
    #2 kudu::rpc::Messenger::ShutdownInternal(kudu::rpc::Messenger::ShutdownMode) /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:239:8 (libkrpc.so+0x16d257)
    #3 kudu::rpc::Messenger::Shutdown() /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:206:3 (libkrpc.so+0x16d80b)
    #4 kudu::kserver::KuduServer::Shutdown() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:192:17 (libkserver.so+0x13865)
    #5 kudu::tserver::TabletServer::ShutdownImpl() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:195:17 (libtserver.so+0x25c2ca)
    #6 kudu::tserver::TabletServer::Shutdown() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x261a79)
    #7 kudu::tserver::MiniTabletServer::Shutdown() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x31c692)
    #8 kudu::cluster::InternalMiniCluster::ShutdownNodes(kudu::cluster::ClusterNodes) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:242:22 (libmini_cluster.so+0xd7a87)
    #9 kudu::cluster::MiniCluster::Shutdown() /apache/dev/git/kudu/src/kudu/mini-cluster/mini_cluster.h:84:5 (auto_leader_rebalancer-test+0x37c6a4)
    #10 kudu::master::LeaderRebalancerTest::TearDown() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:116:17 (auto_leader_rebalancer-test+0x368432)
    #11 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #12 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #13 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2682:3 (libgtest.so.1.12.1+0x42a82)
    #14 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #15 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #16 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #17 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #18 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #19 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #20 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38f0bb)
    #21 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x38dcbc)

  Previous read of size 4 at 0x7b40001690d8 by thread T32 (mutexes: write M626461):
    #0 kudu::DiagnosticSocket::ReceiveResponse(std::__1::vector<kudu::DiagnosticSocket::TcpSocketInfo, std::__1::allocator<kudu::DiagnosticSocket::TcpSocketInfo> >*) const /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:264:3 (libkudu_util.so+0x4a08e1)
    #1 kudu::DiagnosticSocket::Query(kudu::Socket const&, kudu::DiagnosticSocket::TcpSocketInfo*) const /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:134:3 (libkudu_util.so+0x4a1274)
    #2 kudu::rpc::AcceptorPool::GetPendingConnectionsNum(unsigned int*) const /apache/dev/git/kudu/src/kudu/rpc/acceptor_pool.cc:241:3 (libkrpc.so+0x12b653)
    #3 kudu::rpc::Messenger::GetPendingConnectionsNum() /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:193:21 (libkrpc.so+0x16d4b1)
    #4 kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1::operator()() const /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:299:51 (libkrpc.so+0x171bb1)
    #5 decltype(std::__1::forward<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&>(fp)()) std::__1::__invoke<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&>(kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkrpc.so+0x171b69)
    #6 int std::__1::__invoke_void_return_wrapper<int>::__call<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&>(kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libkrpc.so+0x171af9)
    #7 std::__1::__function::__alloc_func<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1, std::__1::allocator<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1>, int ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkrpc.so+0x171ab1)
    #8 std::__1::__function::__func<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1, std::__1::allocator<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1>, int ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkrpc.so+0x170dad)
    #9 std::__1::__function::__value_func<int ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x4d3c74)
    #10 std::__1::function<int ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x4d3c19)
    #11 kudu::FunctionGauge<int>::value() const /apache/dev/git/kudu/src/kudu/util/metrics.h:1319:12 (libmaster.so+0x4d3b4a)
    #12 kudu::FunctionGauge<int>::WriteValue(kudu::JsonWriter*) const /apache/dev/git/kudu/src/kudu/util/metrics.h:1323:19 (libmaster.so+0x4d3830)
    #13 kudu::Gauge::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:856:3 (libkudu_util.so+0x380112)
    #14 void kudu::WriteMetricsToJson<std::__1::unordered_map<kudu::MetricPrototype const*, scoped_refptr<kudu::Metric>, std::__1::hash<kudu::MetricPrototype const*>, std::__1::equal_to<kudu::MetricPrototype const*>, std::__1::allocator<std::__1::pair<kudu::MetricPrototype const* const, scoped_refptr<kudu::Metric> > > > >(kudu::JsonWriter*, std::__1::unordered_map<kudu::MetricPrototype const*, scoped_refptr<kudu::Metric>, std::__1::hash<kudu::MetricPrototype const*>, std::__1::equal_to<kudu::MetricPrototype const*>, std::__1::allocator<std::__1::pair<kudu::MetricPrototype const* const, scoped_refptr<kudu::Metric> > > > const&, kudu::MetricJsonOptions const&) /apache/dev/git/kudu/src/kudu/util/metrics.cc:79:7 (libkudu_util.so+0x3851d0)
    #15 kudu::MetricEntity::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:414:3 (libkudu_util.so+0x37d847)
    #16 kudu::MetricRegistry::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:586:7 (libkudu_util.so+0x37eacc)
    #17 kudu::server::DiagnosticsLog::LogMetrics() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:348:3 (libserver_process.so+0x11db17)
    #18 kudu::server::DiagnosticsLog::RunThread() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:228:7 (libserver_process.so+0x11d7f6)
I20260709 13:08:51.925222 30189 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.926412 30189 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.929191 30189 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
    #19 kudu::server::DiagnosticsLog::Start()::$_0::operator()() const /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:145:46 (libserver_process.so+0x11f381)
    #20 decltype(std::__1::forward<kudu::server::DiagnosticsLog::Start()::$_0&>(fp)()) std::__1::__invoke<kudu::server::DiagnosticsLog::Start()::$_0&>(kudu::server::DiagnosticsLog::Start()::$_0&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libserver_process.so+0x11f339)
    #21 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::server::DiagnosticsLog::Start()::$_0&>(kudu::server::DiagnosticsLog::Start()::$_0&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libserver_process.so+0x11f2c9)
    #22 std::__1::__function::__alloc_func<kudu::server::DiagnosticsLog::Start()::$_0, std::__1::allocator<kudu::server::DiagnosticsLog::Start()::$_0>, void ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libserver_process.so+0x11f291)
    #23 std::__1::__function::__func<kudu::server::DiagnosticsLog::Start()::$_0, std::__1::allocator<kudu::server::DiagnosticsLog::Start()::$_0>, void ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libserver_process.so+0x11e58d)
    #24 std::__1::__function::__value_func<void ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (auto_leader_rebalancer-test+0x38f354)
    #25 std::__1::function<void ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (auto_leader_rebalancer-test+0x38f199)
    #26 kudu::Thread::SuperviseThread(void*) /apache/dev/git/kudu/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x40e83a)

  Location is heap block of size 248 at 0x7b4000169000 allocated by main thread:
    #0 operator new(unsigned long) /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64 (auto_leader_rebalancer-test+0x357897)
    #1 std::__1::__libcpp_allocate(unsigned long, unsigned long) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/new:253:10 (libkrpc.so+0x130946)
    #2 std::__1::allocator<std::__1::__shared_ptr_emplace<kudu::rpc::AcceptorPool, std::__1::allocator<kudu::rpc::AcceptorPool> > >::allocate(unsigned long) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/memory:1789:34 (libkrpc.so+0x17a5e1)
    #3 std::__1::enable_if<!(is_array<kudu::rpc::AcceptorPool>::value), std::__1::shared_ptr<kudu::rpc::AcceptorPool> >::type std::__1::make_shared<kudu::rpc::AcceptorPool, kudu::rpc::Messenger*, kudu::Socket*, kudu::Sockaddr&, int const&>(kudu::rpc::Messenger*&&, kudu::Socket*&&, kudu::Sockaddr&, int const&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/memory:4290:45 (libkrpc.so+0x172a25)
    #4 kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*) /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:288:34 (libkrpc.so+0x16da89)
    #5 kudu::RpcServer::Bind() /apache/dev/git/kudu/src/kudu/server/rpc_server.cc:262:5 (libserver_process.so+0x1393fb)
    #6 kudu::server::ServerBase::Init() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1030:23 (libserver_process.so+0x149419)
    #7 kudu::kserver::KuduServer::Init() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:144:3 (libkserver.so+0x12e81)
    #8 kudu::tserver::TabletServer::Init() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:107:3 (libtserver.so+0x25c752)
    #9 kudu::tserver::MiniTabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:106:3 (libtserver.so+0x31c79b)
    #10 kudu::cluster::InternalMiniCluster::AddTabletServer(kudu::HostPort const&) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:234:3 (libmini_cluster.so+0xd7889)
    #11 kudu::cluster::InternalMiniCluster::AddTabletServer() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:215:10 (libmini_cluster.so+0xd73a5)
    #12 kudu::cluster::InternalMiniCluster::Start() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:112:5 (libmini_cluster.so+0xd663a)
    #13 kudu::master::LeaderRebalancerTest::CreateAndStartCluster() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:104:22 (auto_leader_rebalancer-test+0x36536c)
    #14 kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:376:3 (auto_leader_rebalancer-test+0x359dff)
W20260709 13:08:51.935837 22556 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
    #15 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #16 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #17 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a11)
    #18 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #19 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #20 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #21 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #22 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #23 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #24 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38f0bb)
    #25 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x38dcbc)

  Mutex M626461 (0x7b200002a130) created at:
    #0 __tsan_atomic32_compare_exchange_strong /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interface_atomic.cpp:780 (auto_leader_rebalancer-test+0x3300f0)
    #1 base::subtle::Acquire_CompareAndSwap(int volatile*, int, int) /apache/dev/git/kudu/src/kudu/gutil/atomicops-internals-tsan.h:79:3 (auto_leader_rebalancer-test+0x36b037)
    #2 base::SpinLock::Lock() /apache/dev/git/kudu/src/kudu/gutil/spinlock.h:74:9 (libitest_util.so+0xe31b0)
    #3 kudu::simple_spinlock::lock() /apache/dev/git/kudu/src/kudu/util/locks.h:44:8 (libitest_util.so+0xe3169)
    #4 std::__1::lock_guard<kudu::simple_spinlock>::lock_guard(kudu::simple_spinlock&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/__mutex_base:91:27 (libmaster.so+0x36dcf8)
    #5 kudu::FunctionGauge<int>::value() const /apache/dev/git/kudu/src/kudu/util/metrics.h:1318:21 (libmaster.so+0x4d3b3e)
    #6 kudu::FunctionGauge<int>::WriteValue(kudu::JsonWriter*) const /apache/dev/git/kudu/src/kudu/util/metrics.h:1323:19 (libmaster.so+0x4d3830)
    #7 kudu::Gauge::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:856:3 (libkudu_util.so+0x380112)
    #8 void kudu::WriteMetricsToJson<std::__1::unordered_map<kudu::MetricPrototype const*, scoped_refptr<kudu::Metric>, std::__1::hash<kudu::MetricPrototype const*>, std::__1::equal_to<kudu::MetricPrototype const*>, std::__1::allocator<std::__1::pair<kudu::MetricPrototype const* const, scoped_refptr<kudu::Metric> > > > >(kudu::JsonWriter*, std::__1::unordered_map<kudu::MetricPrototype const*, scoped_refptr<kudu::Metric>, std::__1::hash<kudu::MetricPrototype const*>, std::__1::equal_to<kudu::MetricPrototype const*>, std::__1::allocator<std::__1::pair<kudu::MetricPrototype const* const, scoped_refptr<kudu::Metric> > > > const&, kudu::MetricJsonOptions const&) /apache/dev/git/kudu/src/kudu/util/metrics.cc:79:7 (libkudu_util.so+0x3851d0)
    #9 kudu::MetricEntity::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:414:3 (libkudu_util.so+0x37d847)
    #10 kudu::MetricRegistry::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:586:7 (libkudu_util.so+0x37eacc)
    #11 kudu::server::DiagnosticsLog::LogMetrics() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:348:3 (libserver_process.so+0x11db17)
    #12 kudu::server::DiagnosticsLog::RunThread() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:228:7 (libserver_process.so+0x11d7f6)
    #13 kudu::server::DiagnosticsLog::Start()::$_0::operator()() const /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:145:46 (libserver_process.so+0x11f381)
    #14 decltype(std::__1::forward<kudu::server::DiagnosticsLog::Start()::$_0&>(fp)()) std::__1::__invoke<kudu::server::DiagnosticsLog::Start()::$_0&>(kudu::server::DiagnosticsLog::Start()::$_0&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libserver_process.so+0x11f339)
    #15 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::server::DiagnosticsLog::Start()::$_0&>(kudu::server::DiagnosticsLog::Start()::$_0&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libserver_process.so+0x11f2c9)
    #16 std::__1::__function::__alloc_func<kudu::server::DiagnosticsLog::Start()::$_0, std::__1::allocator<kudu::server::DiagnosticsLog::Start()::$_0>, void ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libserver_process.so+0x11f291)
    #17 std::__1::__function::__func<kudu::server::DiagnosticsLog::Start()::$_0, std::__1::allocator<kudu::server::DiagnosticsLog::Start()::$_0>, void ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libserver_process.so+0x11e58d)
    #18 std::__1::__function::__value_func<void ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (auto_leader_rebalancer-test+0x38f354)
    #19 std::__1::function<void ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (auto_leader_rebalancer-test+0x38f199)
    #20 kudu::Thread::SuperviseThread(void*) /apache/dev/git/kudu/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x40e83a)

  Thread T32 'diag-logger-219' (tid=21956, running) created by main thread at:
    #0 pthread_create /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (auto_leader_rebalancer-test+0x2dbea5)
    #1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /apache/dev/git/kudu/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x40e0d8)
    #2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /apache/dev/git/kudu/src/kudu/util/thread.h:147:12 (libmaster.so+0x2e2ab9)
    #3 kudu::server::DiagnosticsLog::Start() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:144:14 (libserver_process.so+0x11d27d)
    #4 kudu::server::ServerBase::StartMetricsLogging() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1212:3 (libserver_process.so+0x14b6c4)
    #5 kudu::server::ServerBase::Start() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1352:3 (libserver_process.so+0x14bf64)
    #6 kudu::kserver::KuduServer::Start() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:179:22 (libkserver.so+0x13810)
    #7 kudu::tserver::TabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:161:3 (libtserver.so+0x25cf97)
    #8 kudu::tserver::MiniTabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:107:3 (libtserver.so+0x31c7d9)
    #9 kudu::cluster::InternalMiniCluster::AddTabletServer(kudu::HostPort const&) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:234:3 (libmini_cluster.so+0xd7889)
    #10 kudu::cluster::InternalMiniCluster::AddTabletServer() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:215:10 (libmini_cluster.so+0xd73a5)
    #11 kudu::cluster::InternalMiniCluster::Start() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:112:5 (libmini_cluster.so+0xd663a)
    #12 kudu::master::LeaderRebalancerTest::CreateAndStartCluster() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:104:22 (auto_leader_rebalancer-test+0x36536c)
    #13 kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:376:3 (auto_leader_rebalancer-test+0x359dff)
    #14 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #15 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #16 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a11)
    #17 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #18 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #19 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #20 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #21 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #22 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #23 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38f0bb)
    #24 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x38dcbc)

SUMMARY: ThreadSanitizer: data race /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:96:7 in kudu::DiagnosticSocket::Close()
==================
I20260709 13:08:51.972436 30189 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.974509 30194 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:51.981259 30194 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.983737 30194 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:51.984501 30189 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:51.987052 30189 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:51.993283 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:52.013903 30198 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.014779 30198 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.013057 30070 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.017642 30070 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.019860 30070 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.026376 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:52.027462 22586 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
W20260709 13:08:52.031853 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:52.048213 30197 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.048622 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:52.049377 22586 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 21.
I20260709 13:08:52.049168 30197 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.052349 30198 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.053047 30117 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:52.053175 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.053779 30117 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.055084 30070 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:52.053781 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.055567 30169 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.068027 30169 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.075340 30117 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.069386 30200 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.057439 30197 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.056061 30196 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.075480 30169 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.069084 30070 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.077721 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.078133 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:52.082495 30200 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.085889 30200 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:52.082898 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.088802 30203 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.088316 30196 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.089558 30203 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.091689 30196 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.093269 30070 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.081730 30117 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.094971 30117 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.097039 30117 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.105151 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.105952 22008 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
I20260709 13:08:52.107916 22007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.108672 22007 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:52.091724 30203 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.109799 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.110487 22004 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 17.
W20260709 13:08:52.114140 21974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:46099, user_credentials={real_user=slave}} blocked reactor thread for 49964us
I20260709 13:08:52.115347 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:52.116946 30203 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:52.117544 30203 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.117929 30203 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:52.119390 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
W20260709 13:08:52.121402 22556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 43702.1us
I20260709 13:08:52.121454 30199 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:52.122164 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.122378 21974 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.122591 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.123181 30117 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:52.123692 30117 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.124059 30117 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:52.129452 30198 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:52.131521 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.135609 22557 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:52.136951 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.138190 30200 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:52.138618 30200 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.139020 30200 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:52.142112 30189 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:52.142599 30189 raft_consensus.cc:493] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.143011 30189 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:52.150434 21972 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.162812 30117 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.172503 30199 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.173380 30199 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.175769 30194 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.176506 30194 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.178129 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:52.178668 30194 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:52.175801 30199 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.184674 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:52.185420 22587 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:52.187435 30200 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.188172 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:52.195945 30117 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.197644 30200 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:52.189610 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.190747 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:52.192360 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:52.192816 30189 raft_consensus.cc:515] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.204023 22584 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 22: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:52.204694 22586 raft_consensus.cc:2393] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d07aa0dcdd1484ab87925c9a4604b81 in current term 18: Already voted for candidate 34f3b83da84448bf8aac62b37a1486eb in this term.
I20260709 13:08:52.206596 30189 leader_election.cc:290] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:52.198652 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.184842 30203 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.207314 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.207857 22005 raft_consensus.cc:2393] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 22: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:52.201325 30199 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:52.202096 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.211627 30199 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.212801 22557 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:52.213634 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.214072 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.215283 30117 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.215592 30189 raft_consensus.cc:3060] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:52.216325 30116 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:08:52.217692 22556 leader_election.cc:336] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.219288 30117 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.220043 30117 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.202982 22004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:52.223431 22004 raft_consensus.cc:2393] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f3b83da84448bf8aac62b37a1486eb in current term 22: Already voted for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in this term.
I20260709 13:08:52.226519 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24c37d4e05d4b7eb71e234ac493bff5" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:52.227259 22005 raft_consensus.cc:3060] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.233117 30189 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.234269 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:52.236701 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.237877 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.238379 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:52.238831 30203 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.214028 30199 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.241484 30203 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.242148 30203 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.244721 30203 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.246068 30117 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.250324 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d059242e679f4cbaab1cc83ba8353d60" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:52.259554 22005 raft_consensus.cc:2468] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 18.
W20260709 13:08:52.263242 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.265471 21972 leader_election.cc:336] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.269166 21972 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:52.282289 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.292277 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.292869 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:52.283311 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.293705 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:52.294656 21972 leader_election.cc:336] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.282936 30117 raft_consensus.cc:2749] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:52.279211 22586 raft_consensus.cc:2468] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
I20260709 13:08:52.295136 30203 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:52.290405 22557 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.295607 21972 leader_election.cc:304] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:52.298722 30169 raft_consensus.cc:2749] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:52.298929 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.299448 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.300246 30199 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.300366 30189 raft_consensus.cc:2804] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:52.300691 21974 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.300987 30189 raft_consensus.cc:697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:52.302220 30189 consensus_queue.cc:237] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.303023 30117 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:52.302130 30203 raft_consensus.cc:2804] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:52.305984 30203 raft_consensus.cc:697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:52.307935 30203 consensus_queue.cc:237] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.315361 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:52.322829 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.323455 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.323872 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.324663 30117 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.327639 30200 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.328477 30200 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.331060 30200 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.332413 30200 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.333240 30200 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.337436 30200 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:52.337896 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.338586 22005 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
I20260709 13:08:52.337286 30169 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.340029 30169 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.342635 30169 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.342464 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.343835 30116 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.344575 30116 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.346101 21972 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.346570 21972 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:52.349206 30117 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.349799 30116 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:52.350466 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.350593 30199 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:52.358500 30199 raft_consensus.cc:493] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.359048 30199 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:52.353364 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.361923 22005 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 24.
I20260709 13:08:52.368512 21750 catalog_manager.cc:5697] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 21 to 22, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 22 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:52.369808 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:52.370755 21972 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.378597 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.377911 21753 catalog_manager.cc:5697] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 17 to 18, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 18 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:52.379237 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.381770 30203 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:52.382184 22556 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.385329 30117 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.388701 30169 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:52.388794 30117 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:52.391148 30117 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:52.391851 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:08:52.392642 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.393757 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.393621 30189 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:52.387727 30199 raft_consensus.cc:515] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.394742 22556 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.399850 22556 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:08:52.400735 30189 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:08:52.401957 22556 leader_election.cc:336] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.402761 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.402827 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6eae3eb9a4941fc88763a5da4ee7638" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81"
I20260709 13:08:52.403522 22005 raft_consensus.cc:3060] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:52.404002 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.412387 22005 raft_consensus.cc:2468] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 21.
I20260709 13:08:52.416482 22557 leader_election.cc:304] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.404454 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.413506 30199 leader_election.cc:290] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:52.417778 30200 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:52.417776 30189 raft_consensus.cc:2804] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:52.419281 30189 raft_consensus.cc:697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Becoming Leader. State: Replica: 34f3b83da84448bf8aac62b37a1486eb, State: Running, Role: LEADER
I20260709 13:08:52.420442 30189 consensus_queue.cc:237] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.441640 30203 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
W20260709 13:08:52.441820 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.442396 30203 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.444694 30203 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.447882 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:52.448546 22586 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:52.449734 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.450608 30203 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:52.450999 30203 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.451347 30203 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:52.457273 30189 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.458117 30189 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.460322 30189 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:52.465364 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.465850 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.466549 30117 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.475582 30203 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.480340 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f938584aad7a4b76be81dca03c9a647f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:52.480401 30203 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.480952 22586 raft_consensus.cc:3060] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:52.497453 30117 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.498193 30117 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.500288 22586 raft_consensus.cc:2468] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 25.
I20260709 13:08:52.503427 21974 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:52.504639 30204 raft_consensus.cc:2804] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:52.505304 30204 raft_consensus.cc:697] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
I20260709 13:08:52.506310 30117 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:52.506390 30204 consensus_queue.cc:237] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.509613 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.510329 22556 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.514302 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.514945 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.525445 30189 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:52.526643 22556 leader_election.cc:336] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.527428 22556 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.527866 22556 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.518231 22586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
W20260709 13:08:52.515661 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.528661 30189 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:52.529969 22556 leader_election.cc:336] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.528909 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.540195 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.540867 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:52.553712 21750 catalog_manager.cc:5697] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 24 to 25, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 25 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:52.553719 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.563886 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.564199 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.568830 30117 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:52.568984 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:52.569998 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.570421 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.566226 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.571832 30189 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:52.581285 30117 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:52.584649 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:52.589636 21972 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.590571 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:52.598637 21750 catalog_manager.cc:5697] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb reported cstate change: term changed from 20 to 21, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132). New cstate: current_term: 21 leader_uuid: "34f3b83da84448bf8aac62b37a1486eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:52.603508 30189 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.604259 30189 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:52.606725 30189 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:52.620059 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:52.625324 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.627449 30189 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:52.628126 30189 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:52.631650 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:52.632311 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:52.632666 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:52.633430 22005 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
W20260709 13:08:52.634402 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.632582 30189 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:52.632726 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 34f3b83da84448bf8aac62b37a1486eb, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.634933 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.636557 30189 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:52.638540 30117 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:52.645408 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:52.648461 21972 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.651491 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.654017 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:52.653447 22556 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:52.660498 22556 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:52.661980 22556 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:52.666479 30189 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:52.690477 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:52.691486 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:52.733839 22005 raft_consensus.cc:1275] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:08:52.736042 30189 consensus_queue.cc:1048] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:08:52.733958 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.742050 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.762322 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:52.771062 22586 raft_consensus.cc:1275] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:08:52.772729 30117 consensus_queue.cc:1048] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:08:52.798125 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:52.801254 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.828934 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:52.836923 22556 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:52.843425 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:52.875015 22004 raft_consensus.cc:1275] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Refusing update from remote peer 34f3b83da84448bf8aac62b37a1486eb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:52.877632 30199 consensus_queue.cc:1048] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:08:52.887136 22556 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.900480 21972 consensus_peers.cc:597] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:52.903970 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:52.917865 22584 raft_consensus.cc:1275] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:08:52.919816 30204 consensus_queue.cc:1048] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:08:52.976413 20746 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21956)
W20260709 13:08:52.978432 22556 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:52.979256 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.980263 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:53.017735 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.025373 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.025688 22556 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.054546 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:53.055647 21972 consensus_peers.cc:597] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:53.056260 21972 consensus_peers.cc:597] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.087466 21972 consensus_peers.cc:597] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.088305 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.094645 21972 consensus_peers.cc:597] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.102664 22556 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.105664 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.114185 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.257174 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.301654 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.436744 21972 consensus_peers.cc:597] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.480775 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:53.574152 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:53.600278 30117 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:53.600898 30117 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:53.608557 30117 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:53.618328 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.637303 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:53.638170 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:53.638731 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:53.639672 30117 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:53.648664 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.679507 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:53.703205 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.720405 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:53.976985 20746 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 21956)
I20260709 13:08:53.980077 30200 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:53.980754 30200 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:53.983423 30200 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:53.984687 30117 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.985244 30117 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:53.988647 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:53.989403 22587 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 22.
W20260709 13:08:53.990648 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:53.991003 21974 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
W20260709 13:08:53.991555 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:53.992009 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:53.992017 30203 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:53.992759 30203 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.992945 30200 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:53.993211 30203 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:54.008648 30117 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:54.015090 30203 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.026950 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676bc9491c5a4083bdd0b754d904c1d6" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb"
I20260709 13:08:54.027696 22587 raft_consensus.cc:3060] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:54.049604 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:54.066578 22587 raft_consensus.cc:2468] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
I20260709 13:08:54.077271 30203 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Requested vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:54.097303 21972 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.097455 30200 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.110808 30200 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.113622 30200 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:54.098706 21974 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 
I20260709 13:08:54.114993 30203 raft_consensus.cc:2804] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:54.115633 30203 raft_consensus.cc:697] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Becoming Leader. State: Replica: 7d07aa0dcdd1484ab87925c9a4604b81, State: Running, Role: LEADER
W20260709 13:08:54.123432 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.124404 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.125248 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.126405 30203 consensus_queue.cc:237] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.127549 30200 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:08:54.128232 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:54.138757 21972 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.145357 30200 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.146163 30200 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.151823 30200 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:54.154376 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:54.157871 22005 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
W20260709 13:08:54.156386 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.167629 21750 catalog_manager.cc:5697] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 reported cstate change: term changed from 21 to 23, leader changed from 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130) to 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131). New cstate: current_term: 23 leader_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:54.170742 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.184926 22556 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.198431 30200 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:54.210559 30197 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.211272 30197 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.214819 30197 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.219342 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.222671 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:54.226178 30227 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.226990 30227 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:54.230609 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.231241 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.231920 30227 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:08:54.232061 30197 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:54.234401 22005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:54.235126 22005 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 20.
W20260709 13:08:54.245934 22556 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:54.254024 22556 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.254966 22556 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.256137 30200 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:54.259078 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:54.256052 30227 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.262156 30227 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.263263 30203 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.263917 30203 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.264605 30227 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:54.270510 22587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:54.271255 22587 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
I20260709 13:08:54.272823 30203 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
W20260709 13:08:54.281570 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.278039 30203 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.282868 30203 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.290588 30203 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.293958 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.294546 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.295362 30200 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:54.286571 30227 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.296468 30227 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:54.284726 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:54.298869 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:54.303668 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.298918 30227 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:54.313663 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.314738 30203 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:54.317551 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.318348 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.318771 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.319571 30227 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:54.336164 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:54.339977 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.344719 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.345943 30203 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:54.352288 30227 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.352959 30227 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.365206 30227 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.367726 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.370900 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:54.375595 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:54.385953 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.386546 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.387378 30227 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:54.452060 30200 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.452767 30200 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.455093 30200 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.473400 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:54.487500 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.488914 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.489398 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.501274 30234 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:54.513581 30197 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.514237 30197 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.516739 30197 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.521700 21972 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:54.528237 30197 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.528920 30197 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.532662 30197 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.533710 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:54.538797 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.539485 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.540366 30197 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:54.541596 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:54.562665 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.564929 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.573267 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.574383 30197 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:54.665063 30200 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.666031 30200 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.665117 30234 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.672595 30234 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:54.679531 30234 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:54.679925 30197 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.680558 30197 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.682880 30197 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:54.669477 30200 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.695708 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:54.716522 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.717151 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.719645 30234 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:54.731748 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.732239 22587 raft_consensus.cc:1275] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Refusing update from remote peer 7d07aa0dcdd1484ab87925c9a4604b81: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:08:54.735348 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:54.736073 30197 consensus_queue.cc:1048] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:54.750049 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.751157 30197 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:54.756379 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:54.763865 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:54.769361 30197 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.765547 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:54.770733 30234 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:54.770941 30197 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.776324 30197 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:54.784286 21972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111) [suppressed 636 similar messages]
I20260709 13:08:54.780658 30197 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:54.789878 30197 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:54.793995 30197 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:55.211404 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:45191, user_credentials={real_user=slave}} blocked reactor thread for 427124us
W20260709 13:08:55.322233 21972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:38927, user_credentials={real_user=slave}} blocked reactor thread for 107010us
W20260709 13:08:55.481299 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:55.482681 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:55.483119 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:55.483455 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:55.484164 30238 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:55.488704 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:55.490121 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:55.490538 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:55.491900 30238 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:55.498772 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:55.552181 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:55.552834 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:55.567945 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:55.586586 20746 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 21956)
W20260709 13:08:55.661563 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:55.662504 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:55.666121 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:55.666673 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:55.681897 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:55.686817 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:55.711472 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:55.716415 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:55.742559 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:55.942502 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:55.943747 30252 raft_consensus.cc:493] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:55.944443 30252 raft_consensus.cc:515] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:55.946967 30252 leader_election.cc:290] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:55.963853 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:55.964762 21972 leader_election.cc:336] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:55.965257 21972 leader_election.cc:304] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:55.966229 30252 raft_consensus.cc:2749] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:55.972716 30252 raft_consensus.cc:493] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:55.973600 30252 raft_consensus.cc:515] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
W20260709 13:08:55.982430 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:55.986181 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:55.989699 22556 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:56.005540 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:55.985915 30252 leader_election.cc:290] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:56.063438 21972 leader_election.cc:336] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.073092 21972 leader_election.cc:304] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.075456 30239 raft_consensus.cc:2749] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:56.082759 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:56.085840 30253 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.086645 30253 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:56.091897 22008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "34f3b83da84448bf8aac62b37a1486eb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" is_pre_election: true
I20260709 13:08:56.092854 22008 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f3b83da84448bf8aac62b37a1486eb in term 23.
W20260709 13:08:56.103186 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:56.104210 30253 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:56.105818 22556 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.113303 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:56.119158 22556 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.122005 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:56.130703 21972 consensus_peers.cc:597] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:56.139192 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:56.139756 22556 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.140620 30244 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:56.143378 21972 consensus_peers.cc:597] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.144091 21972 consensus_peers.cc:597] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:56.149566 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:56.160635 30240 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.161379 30240 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:56.164785 30240 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:56.167657 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309ab90abff84c36a30bd5a2a0363d2c" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:56.168526 22584 raft_consensus.cc:2468] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 24.
W20260709 13:08:56.183089 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.193668 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:56.197927 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:56.201198 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:56.201649 21972 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.202415 21972 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.203539 30240 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:56.212380 22556 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:56.223069 22556 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.236799 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:56.249120 30240 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.250101 30240 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:56.257447 30240 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:56.257681 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:56.259253 22584 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
W20260709 13:08:56.276716 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:56.277344 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:56.278373 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.278986 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.279965 30240 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:56.284689 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.287961 21972 consensus_peers.cc:597] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.311919 21972 consensus_peers.cc:597] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:56.511670 30240 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.512331 30240 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:56.517596 30244 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.518392 30244 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:56.520831 30244 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:56.530315 30240 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:56.546927 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:56.552214 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.552831 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.553719 30253 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:56.555514 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:56.557601 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.558451 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.559828 30240 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:56.583597 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:56.587146 20746 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 21956)
W20260709 13:08:56.626673 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.631232 22614 debug-util.cc:398] Leaking SignalData structure 0x7b080031d820 after lost signal to thread 20749
W20260709 13:08:56.633172 22614 debug-util.cc:398] Leaking SignalData structure 0x7b0800223160 after lost signal to thread 21792
W20260709 13:08:56.633261 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.635179 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6940 after lost signal to thread 22033
W20260709 13:08:56.636886 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6040 after lost signal to thread 22617
W20260709 13:08:56.666332 21972 consensus_peers.cc:597] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:56.681731 30244 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.682385 30244 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:56.682973 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:56.684640 30244 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:56.689874 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:56.696691 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:56.698132 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.705477 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.706337 30253 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:56.722500 30244 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:56.723145 30244 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:56.725399 30244 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:56.736734 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.751827 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:56.753146 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:56.754076 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:56.754550 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:56.755481 30253 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:56.864130 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:56.885911 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:57.007731 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:57.044881 30249 raft_consensus.cc:493] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.045560 30249 raft_consensus.cc:515] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:57.047662 30249 leader_election.cc:290] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.121963 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:57.123296 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:57.127384 21972 leader_election.cc:336] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:57.127942 21972 leader_election.cc:304] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.128834 30249 raft_consensus.cc:2749] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:57.140833 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:57.145780 30244 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.146509 30244 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:57.148869 30244 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.151716 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:57.159284 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:57.162771 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.163300 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.164069 30253 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:57.197898 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:57.296059 30253 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.296643 30253 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:57.298806 30253 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.308097 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:57.308723 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.309089 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.310138 30253 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:57.334952 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:57.349207 30249 raft_consensus.cc:493] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.349988 30249 raft_consensus.cc:515] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:57.352137 30249 leader_election.cc:290] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.370743 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:57.371727 21972 leader_election.cc:336] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.377175 21972 leader_election.cc:304] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.378113 30249 raft_consensus.cc:2749] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:57.411775 30253 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.412448 30253 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:57.419814 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:57.420989 30253 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:57.424952 30244 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.427124 30244 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:57.429167 30244 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.433867 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.434520 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.435402 30253 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:57.442126 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:57.442934 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:57.445737 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:57.445165 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:57.446557 30253 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:08:57.452865 30253 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.453433 30253 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:57.455696 30253 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.470633 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:57.478632 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.479105 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.480513 30253 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:57.604015 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:57.656167 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:57.802398 30249 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.803074 30249 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:08:57.810909 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:57.816781 22589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc33f94111684d68a49eb643f8be2203" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:57.817070 30249 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:57.817500 22589 raft_consensus.cc:2468] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 20.
W20260709 13:08:57.821184 21972 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.821846 21972 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.820657 30249 raft_consensus.cc:493] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:57.822860 30249 raft_consensus.cc:515] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:57.824297 30240 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:57.827207 30249 leader_election.cc:290] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:57.839661 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:57.850803 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:57.860188 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:57.861536 21972 leader_election.cc:336] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:57.867079 21972 leader_election.cc:304] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:57.867914 30262 raft_consensus.cc:2749] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:57.889714 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:57.890197 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:57.893286 21972 consensus_peers.cc:597] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:57.902168 21972 consensus_peers.cc:597] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:57.941622 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:57.956319 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:57.961611 21972 consensus_peers.cc:597] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:58.022521 30249 raft_consensus.cc:493] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:58.023126 30249 raft_consensus.cc:515] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:58.025266 30249 leader_election.cc:290] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:58.050686 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:58.054201 21972 leader_election.cc:336] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:58.054942 21972 leader_election.cc:304] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:58.055778 30249 raft_consensus.cc:2749] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:58.071411 21972 consensus_peers.cc:597] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.106870 21972 consensus_peers.cc:597] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:58.167214 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:58.202018 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:58.244419 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:58.302093 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:58.337553 21972 consensus_peers.cc:597] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:58.338878 30249 raft_consensus.cc:493] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:58.340004 30249 raft_consensus.cc:515] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:58.342196 30249 leader_election.cc:290] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:58.372747 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:08:58.378031 21972 leader_election.cc:336] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:58.378643 21972 leader_election.cc:304] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:58.382856 30263 raft_consensus.cc:2749] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:58.417886 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.441689 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.453925 21972 consensus_peers.cc:597] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:58.469980 21972 consensus_peers.cc:597] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.514726 21972 consensus_peers.cc:597] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.517168 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:58.520960 22556 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:58.541172 21972 consensus_peers.cc:597] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.580142 21972 consensus_peers.cc:597] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:58.600539 21972 consensus_peers.cc:597] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:58.624818 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
W20260709 13:08:58.658082 22556 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.667727 21972 consensus_peers.cc:597] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:58.724843 30265 raft_consensus.cc:493] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:58.725425 30265 raft_consensus.cc:515] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } }
I20260709 13:08:58.737716 30265 leader_election.cc:290] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:58.751032 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:58.753147 22556 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:58.753707 22556 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.755290 21972 consensus_peers.cc:597] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.755678 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:58.765411 21972 leader_election.cc:336] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:58.765940 21972 leader_election.cc:304] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:58.766844 30266 raft_consensus.cc:2749] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:58.767827 21972 consensus_peers.cc:597] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:58.768373 21972 consensus_peers.cc:597] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.770258 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:58.786465 30266 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:58.787073 30266 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:58.790123 30266 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 34f3b83da84448bf8aac62b37a1486eb (127.20.66.132:46099)
I20260709 13:08:58.791157 22584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "873e001f77374da4a034a00eaaea672f" candidate_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "34f3b83da84448bf8aac62b37a1486eb" is_pre_election: true
I20260709 13:08:58.791842 22584 raft_consensus.cc:2468] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d07aa0dcdd1484ab87925c9a4604b81 in term 23.
W20260709 13:08:58.798708 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:58.804844 21972 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:58.805337 21972 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb, 7d07aa0dcdd1484ab87925c9a4604b81; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:58.806034 30266 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:58.817497 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:58.859166 21972 consensus_peers.cc:597] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.888206 21972 consensus_peers.cc:597] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:58.899418 22556 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:58.944715 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:08:58.995507 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:59.034539 30253 raft_consensus.cc:493] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:59.037984 30253 raft_consensus.cc:515] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:59.040073 30253 leader_election.cc:290] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:59.047386 22557 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:59.053649 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:59.054517 22556 leader_election.cc:336] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:59.058888 22556 leader_election.cc:304] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:59.060328 30253 raft_consensus.cc:2749] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:59.079284 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:59.080420 20746 tablet_replica.cc:333] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.081566 20746 raft_consensus.cc:2243] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.083153 20746 raft_consensus.cc:2272] T 635368d86daf4188a9dcb9cc2f31ce10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.086050 21972 consensus_peers.cc:597] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:59.086251 20746 tablet_replica.cc:333] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.092412 20746 raft_consensus.cc:2243] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.093668 20746 raft_consensus.cc:2272] T d059242e679f4cbaab1cc83ba8353d60 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.096493 20746 tablet_replica.cc:333] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
W20260709 13:08:59.096805 22557 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.097425 20746 raft_consensus.cc:2243] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.101202 20746 raft_consensus.cc:2272] T 8f3ae8d7d5514b8eb2514ea857b270c1 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.104341 20746 tablet_replica.cc:333] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.105314 20746 raft_consensus.cc:2243] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.107218 20746 raft_consensus.cc:2272] T 5f588420255042dda396ba6419abf6bc P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.112562 20746 tablet_replica.cc:333] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.113520 20746 raft_consensus.cc:2243] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.114923 20746 raft_consensus.cc:2272] T 873e001f77374da4a034a00eaaea672f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.118876 20746 tablet_replica.cc:333] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.119772 20746 raft_consensus.cc:2243] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.121176 20746 raft_consensus.cc:2272] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.124893 20746 tablet_replica.cc:333] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.125432 30253 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:59.125955 30253 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:59.126250 20746 raft_consensus.cc:2243] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.127488 20746 raft_consensus.cc:2272] T 1d830a9119594381aa253bf173c99566 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.128242 30253 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:08:59.131469 20746 tablet_replica.cc:333] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.132473 20746 raft_consensus.cc:2243] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.133287 20746 raft_consensus.cc:2272] T 739311613bd748d68267b6f4478680c7 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.136291 20746 tablet_replica.cc:333] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.137305 20746 raft_consensus.cc:2243] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.137975 20746 raft_consensus.cc:2272] T 5ef7ed88974a427b9cab09af73283182 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.141940 20746 tablet_replica.cc:333] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
W20260709 13:08:59.146171 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:59.146915 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:59.147289 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:08:59.149941 22557 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.148582 20746 raft_consensus.cc:2243] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.152137 20746 raft_consensus.cc:2272] T 40a1bb4717e14331a67545a343ce1d87 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.154844 20746 tablet_replica.cc:333] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.155699 20746 raft_consensus.cc:2243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.157121 20746 raft_consensus.cc:2272] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.160384 20746 tablet_replica.cc:333] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.161266 30253 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:59.161509 20746 raft_consensus.cc:2243] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.162360 20746 raft_consensus.cc:2272] T be9ae799c8f043d7b4fc384aa1edf5ce P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.165099 20746 tablet_replica.cc:333] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.165959 20746 raft_consensus.cc:2243] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.166731 20746 raft_consensus.cc:2272] T d41480ad8e164cf2aa88df180060345c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.172314 20746 tablet_replica.cc:333] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.173281 20746 raft_consensus.cc:2243] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:59.173841 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:59.173997 20746 raft_consensus.cc:2272] T 90e34f5775b8400e8e9bb1a13ff27479 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.179287 20746 tablet_replica.cc:333] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.180249 20746 raft_consensus.cc:2243] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.180898 20746 raft_consensus.cc:2272] T 94e8056fad5248118e00e80f8a81538c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.183934 20746 tablet_replica.cc:333] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.184762 20746 raft_consensus.cc:2243] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.188957 20746 raft_consensus.cc:2272] T 676bc9491c5a4083bdd0b754d904c1d6 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.191872 20746 tablet_replica.cc:333] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.194096 20746 raft_consensus.cc:2243] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.197732 20746 raft_consensus.cc:2272] T 10076817fc0f46b4943de726ac0b508c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.200667 20746 tablet_replica.cc:333] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.201567 20746 raft_consensus.cc:2243] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Raft consensus shutting down.
W20260709 13:08:59.213462 22557 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.214259 20746 raft_consensus.cc:2272] T 8594e4ce37964bf09c89143bed59f40e P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.217443 20746 tablet_replica.cc:333] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.219673 20746 raft_consensus.cc:2243] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.224632 20746 raft_consensus.cc:2272] T f6eae3eb9a4941fc88763a5da4ee7638 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.226568 21972 consensus_peers.cc:597] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:59.228694 20746 tablet_replica.cc:333] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.229882 20746 raft_consensus.cc:2243] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.230573 20746 raft_consensus.cc:2272] T b42af043129e482498af1e2c3882a042 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.233577 20746 tablet_replica.cc:333] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
W20260709 13:08:59.242395 22557 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.245703 20746 raft_consensus.cc:2243] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:59.245746 21972 consensus_peers.cc:597] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:59.246703 20746 raft_consensus.cc:2272] T 81cd5c4a1eef4004b3f1a9728e83a459 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.249565 20746 tablet_replica.cc:333] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.250588 20746 raft_consensus.cc:2243] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.251377 20746 raft_consensus.cc:2272] T f24c37d4e05d4b7eb71e234ac493bff5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.261555 22557 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:59.265976 22557 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.267036 20746 tablet_replica.cc:333] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
W20260709 13:08:59.267769 21972 consensus_peers.cc:597] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:59.276636 22557 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.277722 20746 raft_consensus.cc:2243] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.280217 20746 raft_consensus.cc:2272] T 6ed3a64e6b0242fa9fda4766cc68016b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.282492 21972 consensus_peers.cc:597] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:59.284324 20746 tablet_replica.cc:333] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.286798 20746 raft_consensus.cc:2243] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.288352 20746 raft_consensus.cc:2272] T a226c66b13a04eb389353b8e2deb175c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.296833 22557 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.298106 20746 tablet_replica.cc:333] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
W20260709 13:08:59.302556 22557 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.303727 20746 raft_consensus.cc:2243] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.305059 20746 raft_consensus.cc:2272] T 09efb42cfd6142bb9f816328042aea38 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.308005 20746 tablet_replica.cc:333] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.308887 20746 raft_consensus.cc:2243] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.309777 20746 raft_consensus.cc:2272] T 7e16208fe6d441998baf1cb8b8546d10 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.312552 20746 tablet_replica.cc:333] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.313373 20746 raft_consensus.cc:2243] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.314764 20746 raft_consensus.cc:2272] T 1e33356352ee4f06be9c7f962467110c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.319159 20746 tablet_replica.cc:333] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
W20260709 13:08:59.324640 22557 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.325304 20746 raft_consensus.cc:2243] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.327308 20746 raft_consensus.cc:2272] T 2ee2529af5954101bde1189a694cafa5 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.330356 20746 tablet_replica.cc:333] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.331302 20746 raft_consensus.cc:2243] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.332427 20746 raft_consensus.cc:2272] T 7d337058bd1f4df1bd938c43960820be P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.335222 20746 tablet_replica.cc:333] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.336122 20746 raft_consensus.cc:2243] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.338227 20746 raft_consensus.cc:2272] T 634269b557d04cbe87cfab961aaf3f91 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.342841 20746 tablet_replica.cc:333] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.344951 20746 raft_consensus.cc:2243] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.346493 20746 raft_consensus.cc:2272] T 0ecc321cb1e14e99a3838b646f82e71a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.349941 20746 tablet_replica.cc:333] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.350795 20746 raft_consensus.cc:2243] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.353703 20746 raft_consensus.cc:2272] T 2167f46d5e1049268b9ce66dbf46f781 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.356941 20746 tablet_replica.cc:333] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.357923 20746 raft_consensus.cc:2243] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.358942 20746 raft_consensus.cc:2272] T c96543776baa4cdda2e6862202de4e52 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.362046 20746 tablet_replica.cc:333] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.363387 20746 raft_consensus.cc:2243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.364341 20746 raft_consensus.cc:2272] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.367960 20746 tablet_replica.cc:333] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.370929 20746 raft_consensus.cc:2243] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.373701 30244 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:59.374408 30244 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:59.373715 20746 raft_consensus.cc:2272] T f938584aad7a4b76be81dca03c9a647f P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.377506 30244 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:08:59.383772 22557 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:59.386103 22556 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:08:59.386725 20746 tablet_replica.cc:333] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.386597 22556 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:59.387625 30244 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:59.387676 22556 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:59.389572 20746 raft_consensus.cc:2243] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.391469 20746 raft_consensus.cc:2272] T 9fc728770e5c4aa1ac36a662cebb8598 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.395551 20746 tablet_replica.cc:333] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.396466 20746 raft_consensus.cc:2243] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.398147 20746 raft_consensus.cc:2272] T 6f466d510c1b46d481ab83686b150d61 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.401376 20746 tablet_replica.cc:333] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.402431 20746 raft_consensus.cc:2243] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.403936 20746 raft_consensus.cc:2272] T 97cf25498fcd4e4594d809b889d236d3 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.407864 20746 tablet_replica.cc:333] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.409674 20746 raft_consensus.cc:2243] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.410640 20746 raft_consensus.cc:2272] T f7024293bd494d23afe75d0322f5445b P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.413832 22557 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.416908 20746 tablet_replica.cc:333] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.417943 20746 raft_consensus.cc:2243] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.419488 20746 raft_consensus.cc:2272] T 0103ac417f394657ac6cbe827df3b844 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.424400 20746 tablet_replica.cc:333] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.425385 20746 raft_consensus.cc:2243] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.427920 20746 raft_consensus.cc:2272] T 1db8966fba10424a8df85ad63defc925 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.431066 20746 tablet_replica.cc:333] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.431870 20746 raft_consensus.cc:2243] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.433106 20746 raft_consensus.cc:2272] T 309ab90abff84c36a30bd5a2a0363d2c P 7d07aa0dcdd1484ab87925c9a4604b81 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.435973 20746 tablet_replica.cc:333] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.436949 20746 raft_consensus.cc:2243] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.438265 20746 raft_consensus.cc:2272] T 4021839a030d44ab886833307930ad33 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.441205 20746 tablet_replica.cc:333] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.442003 20746 raft_consensus.cc:2243] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.443619 20746 raft_consensus.cc:2272] T 8f2ed3e1870148589d1bba70ce75f160 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.446756 20746 tablet_replica.cc:333] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.447556 20746 raft_consensus.cc:2243] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.448851 20746 raft_consensus.cc:2272] T d2bcf30627084357a4745445324a2430 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.452021 20746 tablet_replica.cc:333] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.452801 20746 raft_consensus.cc:2243] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.454185 20746 raft_consensus.cc:2272] T ec2c60ea913b4b8a9b1cbfd733880203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.457176 20746 tablet_replica.cc:333] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81: stopping tablet replica
I20260709 13:08:59.457914 20746 raft_consensus.cc:2243] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.459152 20746 raft_consensus.cc:2272] T bc33f94111684d68a49eb643f8be2203 P 7d07aa0dcdd1484ab87925c9a4604b81 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.470453 22557 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.516242 30253 raft_consensus.cc:493] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:59.516860 30253 raft_consensus.cc:515] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:59.519238 30253 leader_election.cc:290] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:59.538173 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:59.538990 22556 leader_election.cc:336] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:59.539405 22556 leader_election.cc:304] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:59.540199 30253 raft_consensus.cc:2749] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:59.555678 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:59.577328 30244 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:08:59.578042 30244 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:08:59.580422 30244 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:08:59.586198 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:08:59.586884 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:08:59.587294 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:08:59.597095 30244 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:08:59.618783 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:59.691310 22030 debug-util.cc:398] Leaking SignalData structure 0x7b0800156d40 after lost signal to thread 20749
W20260709 13:08:59.692507 22030 debug-util.cc:398] Leaking SignalData structure 0x7b080005a900 after lost signal to thread 21792
W20260709 13:08:59.693931 22030 debug-util.cc:398] Leaking SignalData structure 0x7b0800368980 after lost signal to thread 22033
W20260709 13:08:59.695227 22030 debug-util.cc:398] Leaking SignalData structure 0x7b08002c2c80 after lost signal to thread 22617
W20260709 13:08:59.788712 22556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111) [suppressed 550 similar messages]
W20260709 13:08:59.813690 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:59.933389 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:09:00.304280 30244 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.304904 30244 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.312597 30244 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:09:00.315006 21747 maintenance_manager.cc:419] P 85f464a9537546118c8c1817bd5574c1: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033383
W20260709 13:09:00.317770 22556 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:00.318684 22556 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.319128 22556 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:00.320183 30244 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:09:00.323292 22557 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.361181 30244 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.361732 30244 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:00.370091 22556 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:00.380978 22557 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.382776 22557 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.383906 30253 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:00.384667 30244 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.395601 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:09:00.411464 30244 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.412129 30244 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.414244 30244 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.421505 22557 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.432257 30253 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.432883 30253 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.434764 30253 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.443441 22557 leader_election.cc:336] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:00.457858 22556 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:00.458660 22556 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:00.477257 30253 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.483167 30253 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.483745 30253 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:00.491432 22557 leader_election.cc:336] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.492750 30253 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.510131 22556 leader_election.cc:336] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.510694 22556 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.513651 30253 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:00.525962 30253 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.526561 30253 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.529779 30253 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.540288 22557 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.543329 30253 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.543929 30253 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.546073 30253 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.552748 22557 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.556852 30253 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.557425 30253 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.559466 30253 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:09:00.568848 30253 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.569590 30253 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.571903 30253 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.572549 22557 leader_election.cc:336] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.577333 21746 maintenance_manager.cc:643] P 85f464a9537546118c8c1817bd5574c1: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.261s	user 0.041s	sys 0.007s Metrics: {"bytes_written":8233,"delete_count":0,"lbm_write_time_us":1195,"lbm_writes_lt_1ms":7,"reinsert_count":0,"thread_start_us":35046,"threads_started":1,"update_count":306,"wal-append.queue_time_us":8574}
W20260709 13:09:00.580585 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:00.591101 22556 leader_election.cc:336] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:00.591580 22556 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:00.591749 21747 maintenance_manager.cc:419] P 85f464a9537546118c8c1817bd5574c1: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.384602
W20260709 13:09:00.593187 22557 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.593976 30244 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.619727 30253 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.620309 30253 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:00.621471 20746 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22030)
I20260709 13:09:00.622444 30253 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.625867 22556 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.626360 22556 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:09:00.632103 22556 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.632589 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.633955 30244 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:00.634698 30244 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:00.637439 22556 leader_election.cc:336] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.637956 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
W20260709 13:09:00.638820 22557 leader_election.cc:336] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.640604 30253 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:00.662060 22556 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:00.680575 22556 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.682096 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.683696 30253 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:00.692283 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:00.695932 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:00.711130 30253 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.711696 30253 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.713887 30253 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.719230 22557 leader_election.cc:336] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:00.722818 22556 leader_election.cc:336] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.723747 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.725066 30253 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:00.743844 30253 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.744335 30253 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.746186 30253 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.758327 22557 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:00.765539 22556 leader_election.cc:336] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.773743 22556 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.781498 30253 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:00.784554 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:00.790257 22556 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:00.791530 22556 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:00.792126 30253 raft_consensus.cc:493] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:00.792661 30253 raft_consensus.cc:515] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:00.800602 22557 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:00.801503 30253 leader_election.cc:290] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:09:00.802567 30253 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:00.842862 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:00.843421 22556 leader_election.cc:304] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:00.844210 30253 raft_consensus.cc:2749] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:00.845379 22556 leader_election.cc:336] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:00.861837 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:00.878376 30253 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:00.878886 30253 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:00.881491 30253 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:00.895480 22557 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:00.899170 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:00.900055 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:00.904331 30244 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:00.903847 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:00.913923 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:00.927271 22556 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:01.040436 21746 maintenance_manager.cc:643] P 85f464a9537546118c8c1817bd5574c1: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.447s	user 0.109s	sys 0.015s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":29299,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":3393,"lbm_read_time_us":2453,"lbm_reads_lt_1ms":26,"lbm_write_time_us":1920,"lbm_writes_lt_1ms":12,"peak_mem_usage":313875,"reinsert_count":0,"thread_start_us":2259,"threads_started":2,"update_count":313}
I20260709 13:09:01.043859 21747 maintenance_manager.cc:419] P 85f464a9537546118c8c1817bd5574c1: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:09:01.044840 21746 maintenance_manager.cc:643] P 85f464a9537546118c8c1817bd5574c1: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 13:09:01.120553 30253 raft_consensus.cc:493] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:01.121332 30253 raft_consensus.cc:515] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:01.124439 30253 leader_election.cc:290] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:09:01.135546 30253 raft_consensus.cc:493] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:01.136269 30253 raft_consensus.cc:515] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:01.176508 22556 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:01.177323 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:01.177968 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:01.186100 22556 leader_election.cc:336] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:01.187894 22556 leader_election.cc:304] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:01.189970 30277 raft_consensus.cc:2749] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:01.187021 30253 leader_election.cc:290] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:01.207455 22557 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:01.209594 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:01.210245 22556 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:01.210621 22556 leader_election.cc:304] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:01.214520 30280 raft_consensus.cc:2749] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:01.224942 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:01.239003 22556 leader_election.cc:336] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:01.243064 22556 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:01.244102 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:01.322279 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:01.324124 30280 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:01.325673 30280 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:01.331219 30280 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:09:01.348160 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:01.349148 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:01.349634 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:01.350617 30280 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:01.352406 22556 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:01.455696 22557 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.464123 22557 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.475389 22556 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:01.567343 22557 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.682466 22557 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.684809 22556 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:01.706573 22557 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.719522 22557 consensus_peers.cc:597] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.755617 22557 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.773576 22557 consensus_peers.cc:597] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:01.833956 30244 raft_consensus.cc:493] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:01.835628 30244 raft_consensus.cc:515] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:01.840130 30244 leader_election.cc:290] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:01.850005 22556 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:01.851107 22556 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:01.851630 22556 leader_election.cc:304] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:09:01.853181 22557 leader_election.cc:336] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:01.853875 30244 raft_consensus.cc:2749] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:09:01.858244 22557 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:01.916433 22557 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:01.927223 30244 raft_consensus.cc:493] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:01.931628 30244 raft_consensus.cc:515] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:01.940608 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:01.942378 30244 leader_election.cc:290] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:09:01.972409 22556 leader_election.cc:336] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:01.974711 22556 leader_election.cc:304] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:01.976277 30244 raft_consensus.cc:2749] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:01.978204 22556 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:01.987289 22557 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:02.019449 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388a00 after lost signal to thread 20749
W20260709 13:09:02.022455 21789 debug-util.cc:398] Leaking SignalData structure 0x7b0800388b40 after lost signal to thread 21792
W20260709 13:09:02.029369 21789 debug-util.cc:398] Leaking SignalData structure 0x7b080008a980 after lost signal to thread 22617
I20260709 13:09:02.048162 30244 raft_consensus.cc:493] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:02.049837 30244 raft_consensus.cc:515] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.055068 30244 leader_election.cc:290] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:09:02.055810 22557 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:02.068830 30244 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.069902 30244 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.074434 30244 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.090380 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:02.092259 22556 leader_election.cc:336] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:02.093185 22556 leader_election.cc:304] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
W20260709 13:09:02.091176 22557 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.094961 30244 raft_consensus.cc:2749] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:09:02.111433 22556 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.112634 22556 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.120653 30244 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:02.163769 22556 consensus_peers.cc:597] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:02.206156 22557 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:02.219122 30253 raft_consensus.cc:493] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.222378 30253 raft_consensus.cc:515] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.226029 30253 leader_election.cc:290] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.238943 22556 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:02.240237 22556 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.240808 22556 leader_election.cc:304] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.241828 30253 raft_consensus.cc:2749] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:02.249863 30253 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.250562 30253 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.253300 30253 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.264780 22557 leader_election.cc:336] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:02.279335 22557 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:02.286513 22556 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.287438 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.288462 30253 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:02.300297 30253 raft_consensus.cc:493] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:02.301333 30253 raft_consensus.cc:515] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.304118 30253 leader_election.cc:290] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
I20260709 13:09:02.326932 30253 raft_consensus.cc:493] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.328176 30253 raft_consensus.cc:515] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:02.337994 22557 leader_election.cc:336] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.338649 30253 leader_election.cc:290] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.340003 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:02.340973 22556 leader_election.cc:336] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.341459 22556 leader_election.cc:304] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.342571 30253 raft_consensus.cc:2749] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:02.352767 22556 leader_election.cc:336] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.353569 22556 leader_election.cc:304] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.354503 30253 raft_consensus.cc:2749] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:02.377060 30253 raft_consensus.cc:493] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.377990 30253 raft_consensus.cc:515] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.381615 30253 leader_election.cc:290] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.388860 22557 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:02.390570 22556 consensus_peers.cc:597] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:09:02.394047 30253 raft_consensus.cc:493] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.394903 30253 raft_consensus.cc:515] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.397472 30253 leader_election.cc:290] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.404531 22557 leader_election.cc:336] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:02.411586 22556 leader_election.cc:336] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.412909 22556 leader_election.cc:304] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.414224 30253 raft_consensus.cc:2749] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:02.420843 22556 leader_election.cc:336] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.421729 22556 leader_election.cc:304] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.423022 30253 raft_consensus.cc:2749] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:02.561137 30253 raft_consensus.cc:493] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.561986 30253 raft_consensus.cc:515] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } attrs { replace: true } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:02.578876 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:02.593236 30282 raft_consensus.cc:493] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
W20260709 13:09:02.604790 22556 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:02.605778 22556 leader_election.cc:304] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.610556 30244 raft_consensus.cc:2749] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:02.564913 30253 leader_election.cc:290] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.619024 22557 leader_election.cc:336] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.598605 30282 raft_consensus.cc:515] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.623796 30282 leader_election.cc:290] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:09:02.626346 30244 raft_consensus.cc:493] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.627666 30244 raft_consensus.cc:515] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.634388 30244 leader_election.cc:290] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.636246 22556 leader_election.cc:336] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:02.637414 22557 leader_election.cc:336] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.637998 22557 leader_election.cc:304] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.639849 30244 raft_consensus.cc:2749] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:09:02.641788 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:02.642823 22556 leader_election.cc:336] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:02.649634 22557 leader_election.cc:336] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.650331 22557 leader_election.cc:304] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:02.661422 30282 raft_consensus.cc:2749] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:02.672403 30282 raft_consensus.cc:493] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.673666 30282 raft_consensus.cc:515] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.676656 30282 leader_election.cc:290] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.690573 22556 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:02.703162 22557 leader_election.cc:336] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.704067 22557 leader_election.cc:304] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.705670 30284 raft_consensus.cc:2749] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:02.713085 30284 raft_consensus.cc:493] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.714202 30284 raft_consensus.cc:515] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.719944 30244 raft_consensus.cc:493] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:02.720715 30244 raft_consensus.cc:515] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.723001 30284 leader_election.cc:290] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:09:02.723742 30244 leader_election.cc:290] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
I20260709 13:09:02.740386 30282 raft_consensus.cc:493] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:02.741151 30282 raft_consensus.cc:515] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.744426 30282 leader_election.cc:290] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:09:02.750028 22556 leader_election.cc:336] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:02.751300 22556 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:02.752488 22557 leader_election.cc:336] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.758464 22557 leader_election.cc:304] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.759598 30244 raft_consensus.cc:2749] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:02.761649 22557 leader_election.cc:336] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:02.766742 22557 leader_election.cc:304] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.770201 30244 raft_consensus.cc:2749] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:02.788002 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:02.789070 22556 leader_election.cc:336] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:02.789592 22556 leader_election.cc:304] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.790464 30244 raft_consensus.cc:2749] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:02.827371 30244 raft_consensus.cc:493] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:02.828622 30244 raft_consensus.cc:515] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } attrs { replace: true } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:02.832396 30244 leader_election.cc:290] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:02.839206 22557 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:02.857311 22556 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
I20260709 13:09:02.858047 22556 leader_election.cc:304] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:02.859006 30244 raft_consensus.cc:2749] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:02.860208 22556 leader_election.cc:336] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
W20260709 13:09:03.158291 22556 consensus_peers.cc:597] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:03.232414 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:03.261860 22556 consensus_peers.cc:597] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:09:03.275036 22556 consensus_peers.cc:597] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:03.304013 22556 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:03.337306 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:09:03.416406 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
W20260709 13:09:03.515338 22556 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:03.552672 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:03.577855 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6b20 after lost signal to thread 20749
W20260709 13:09:03.579546 22614 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6860 after lost signal to thread 21792
W20260709 13:09:03.584134 22614 debug-util.cc:398] Leaking SignalData structure 0x7b0800321440 after lost signal to thread 22617
I20260709 13:09:03.647203 20746 tablet_server.cc:179] TabletServer@127.20.66.132:0 shutting down...
W20260709 13:09:03.694977 22556 consensus_peers.cc:597] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb -> Peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Couldn't send request to peer 8f3b35a35257416db3eaaf364cf6b52b. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:03.696118 22556 consensus_peers.cc:597] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:03.742528 22556 consensus_peers.cc:597] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:03.758961 22556 consensus_peers.cc:597] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:03.760617 30282 raft_consensus.cc:493] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:03.761395 30282 raft_consensus.cc:515] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:03.761675 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:03.767975 30282 leader_election.cc:290] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:03.770622 22557 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:03.773262 20746 tablet_replica.cc:333] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.774811 20746 raft_consensus.cc:2243] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.776221 20746 raft_consensus.cc:2272] T 873e001f77374da4a034a00eaaea672f P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:03.785266 22556 leader_election.cc:336] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:03.787565 20746 tablet_replica.cc:333] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.789291 20746 raft_consensus.cc:2243] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.792833 20746 raft_consensus.cc:2272] T 8594e4ce37964bf09c89143bed59f40e P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.785936 22556 leader_election.cc:304] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:03.801177 30282 raft_consensus.cc:2749] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:03.801826 20746 tablet_replica.cc:333] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.810815 20746 raft_consensus.cc:2243] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:09:03.802894 22556 consensus_peers.cc:597] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:03.818113 20746 raft_consensus.cc:2272] T 309ab90abff84c36a30bd5a2a0363d2c P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:03.824600 22556 consensus_peers.cc:597] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:03.829821 20746 tablet_replica.cc:333] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.834950 20746 raft_consensus.cc:2243] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.841846 20746 raft_consensus.cc:2272] T 4458afcc40954b359ea857cddbfdd681 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.865427 20746 tablet_replica.cc:333] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.866727 20746 raft_consensus.cc:2243] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.867919 20746 raft_consensus.cc:2272] T 9fc728770e5c4aa1ac36a662cebb8598 P 34f3b83da84448bf8aac62b37a1486eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.877349 20746 tablet_replica.cc:333] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.881765 20746 raft_consensus.cc:2243] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.883168 20746 raft_consensus.cc:2272] T 7d337058bd1f4df1bd938c43960820be P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.888123 20746 tablet_replica.cc:333] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.889518 20746 raft_consensus.cc:2243] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.892199 20746 raft_consensus.cc:2272] T 676bc9491c5a4083bdd0b754d904c1d6 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.896438 20746 tablet_replica.cc:333] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.897713 20746 raft_consensus.cc:2243] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.899600 20746 raft_consensus.cc:2272] T 6622001ac78d44caa90b4ce7f5283aa1 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.903322 20746 tablet_replica.cc:333] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.904326 20746 raft_consensus.cc:2243] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.906100 20746 raft_consensus.cc:2272] T b42af043129e482498af1e2c3882a042 P 34f3b83da84448bf8aac62b37a1486eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.909791 20746 tablet_replica.cc:333] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.910745 20746 raft_consensus.cc:2243] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.919288 30282 raft_consensus.cc:493] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7d07aa0dcdd1484ab87925c9a4604b81)
I20260709 13:09:03.919901 30282 raft_consensus.cc:515] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "7d07aa0dcdd1484ab87925c9a4604b81" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36521 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
I20260709 13:09:03.922513 30282 leader_election.cc:290] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191), 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521)
W20260709 13:09:03.927734 22556 consensus_peers.cc:597] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Couldn't send request to peer 821b3831b8a348c0826bd3a20a857f00. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:03.930604 22557 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:03.934371 22556 leader_election.cc:336] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:03.934916 22556 leader_election.cc:304] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 7d07aa0dcdd1484ab87925c9a4604b81, 821b3831b8a348c0826bd3a20a857f00
I20260709 13:09:03.936213 30282 raft_consensus.cc:2749] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:03.937345 20746 raft_consensus.cc:2272] T 2a91310e68464d64aa8324092182518f P 34f3b83da84448bf8aac62b37a1486eb [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.941062 20746 tablet_replica.cc:333] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.942148 20746 raft_consensus.cc:2243] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.942993 20746 raft_consensus.cc:2272] T d2bcf30627084357a4745445324a2430 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.946691 20746 tablet_replica.cc:333] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.947760 20746 raft_consensus.cc:2243] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.950760 20746 raft_consensus.cc:2272] T 8b5d11192c5d4d72b0bb9f3ebe9b715a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.954541 20746 tablet_replica.cc:333] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.955538 20746 raft_consensus.cc:2243] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.957279 20746 raft_consensus.cc:2272] T 958f1a2d78484a3bab2de506da912756 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.961381 20746 tablet_replica.cc:333] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.962514 20746 raft_consensus.cc:2243] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.964357 20746 raft_consensus.cc:2272] T 8a76564a0ee84b74a3af01c79300d4b7 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.968415 20746 tablet_replica.cc:333] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.969602 20746 raft_consensus.cc:2243] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.970966 20746 raft_consensus.cc:2272] T bc33f94111684d68a49eb643f8be2203 P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.975131 20746 tablet_replica.cc:333] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.976152 20746 raft_consensus.cc:2243] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.978792 20746 raft_consensus.cc:2272] T a5e5e01f84ee40afb9b688bf0b446c6c P 34f3b83da84448bf8aac62b37a1486eb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.982786 20746 tablet_replica.cc:333] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.983757 20746 raft_consensus.cc:2243] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.986955 20746 raft_consensus.cc:2272] T caafc8397a4f4963abeff030a28f5c47 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.990501 20746 tablet_replica.cc:333] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.991549 20746 raft_consensus.cc:2243] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:09:03.993494 20746 raft_consensus.cc:2272] T f6eae3eb9a4941fc88763a5da4ee7638 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:03.997155 20746 tablet_replica.cc:333] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:03.998397 20746 raft_consensus.cc:2243] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:03.999651 20746 raft_consensus.cc:2272] T 6ed3a64e6b0242fa9fda4766cc68016b P 34f3b83da84448bf8aac62b37a1486eb [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.003785 20746 tablet_replica.cc:333] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.004724 20746 raft_consensus.cc:2243] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.006985 20746 raft_consensus.cc:2272] T a226c66b13a04eb389353b8e2deb175c P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.011152 20746 tablet_replica.cc:333] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.012476 20746 raft_consensus.cc:2243] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.014495 20746 raft_consensus.cc:2272] T f24c37d4e05d4b7eb71e234ac493bff5 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.018486 20746 tablet_replica.cc:333] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.019508 20746 raft_consensus.cc:2243] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.021167 20746 raft_consensus.cc:2272] T 71a9145a148c4edebd5d8df1c9956298 P 34f3b83da84448bf8aac62b37a1486eb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.024873 20746 tablet_replica.cc:333] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.025933 20746 raft_consensus.cc:2243] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.028838 20746 raft_consensus.cc:2272] T 2167f46d5e1049268b9ce66dbf46f781 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.033192 20746 tablet_replica.cc:333] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.034569 20746 raft_consensus.cc:2243] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.036111 20746 raft_consensus.cc:2272] T 0ecc321cb1e14e99a3838b646f82e71a P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.046332 20746 tablet_replica.cc:333] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
W20260709 13:09:04.056031 22557 consensus_peers.cc:597] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:04.059500 30282 raft_consensus.cc:493] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 821b3831b8a348c0826bd3a20a857f00)
I20260709 13:09:04.060218 30282 raft_consensus.cc:515] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8f3b35a35257416db3eaaf364cf6b52b" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 38927 } } peers { permanent_uuid: "821b3831b8a348c0826bd3a20a857f00" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 45191 } } peers { permanent_uuid: "34f3b83da84448bf8aac62b37a1486eb" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 46099 } attrs { promote: false } }
W20260709 13:09:04.072553 22557 consensus_peers.cc:597] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:04.073163 30282 leader_election.cc:290] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927), 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191)
W20260709 13:09:04.076006 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:04.081213 22556 leader_election.cc:336] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 821b3831b8a348c0826bd3a20a857f00 (127.20.66.130:45191): Network error: Client connection negotiation failed: client connection to 127.20.66.130:45191: connect: Connection refused (error 111)
I20260709 13:09:04.082619 22556 leader_election.cc:304] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f3b83da84448bf8aac62b37a1486eb; no voters: 821b3831b8a348c0826bd3a20a857f00, 8f3b35a35257416db3eaaf364cf6b52b
I20260709 13:09:04.085616 30244 raft_consensus.cc:2749] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:04.087054 20746 raft_consensus.cc:2243] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.090322 20746 raft_consensus.cc:2272] T 5f588420255042dda396ba6419abf6bc P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.095371 20746 tablet_replica.cc:333] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.096702 20746 raft_consensus.cc:2243] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.098085 20746 raft_consensus.cc:2272] T 6f466d510c1b46d481ab83686b150d61 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.102053 20746 tablet_replica.cc:333] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.103276 20746 raft_consensus.cc:2243] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.106125 20746 raft_consensus.cc:2272] T 97cf25498fcd4e4594d809b889d236d3 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.119066 20746 tablet_replica.cc:333] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.120416 20746 raft_consensus.cc:2243] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.121435 20746 raft_consensus.cc:2272] T 0c70dbf50c904384a2ab23b1d7a9dbfd P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:04.122799 22557 consensus_peers.cc:597] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb -> Peer 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): Couldn't send request to peer 7d07aa0dcdd1484ab87925c9a4604b81. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:04.125531 20746 tablet_replica.cc:333] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.126678 20746 raft_consensus.cc:2243] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.127569 20746 raft_consensus.cc:2272] T 4021839a030d44ab886833307930ad33 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.131508 20746 tablet_replica.cc:333] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.132848 20746 raft_consensus.cc:2243] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.133911 20746 raft_consensus.cc:2272] T 951fdd0c64f542cbab511fb32cfa91cb P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.137509 20746 tablet_replica.cc:333] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.138571 20746 raft_consensus.cc:2243] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.140883 20746 raft_consensus.cc:2272] T c96543776baa4cdda2e6862202de4e52 P 34f3b83da84448bf8aac62b37a1486eb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.144949 20746 tablet_replica.cc:333] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.146018 20746 raft_consensus.cc:2243] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.147742 20746 raft_consensus.cc:2272] T 1cf1fd218a894ed39170b8db4ecc6ee8 P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.151412 20746 tablet_replica.cc:333] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.152714 20746 raft_consensus.cc:2243] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.154122 20746 raft_consensus.cc:2272] T 0b8c4da9f5144e2b8007c8ffc563a584 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.157891 20746 tablet_replica.cc:333] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.158938 20746 raft_consensus.cc:2243] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.159912 20746 raft_consensus.cc:2272] T d059242e679f4cbaab1cc83ba8353d60 P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.163523 20746 tablet_replica.cc:333] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.164469 20746 raft_consensus.cc:2243] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.165434 20746 raft_consensus.cc:2272] T ec2c60ea913b4b8a9b1cbfd733880203 P 34f3b83da84448bf8aac62b37a1486eb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.168820 20746 tablet_replica.cc:333] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.169780 20746 raft_consensus.cc:2243] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.171463 20746 raft_consensus.cc:2272] T 5ef7ed88974a427b9cab09af73283182 P 34f3b83da84448bf8aac62b37a1486eb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.175231 20746 tablet_replica.cc:333] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.176175 20746 raft_consensus.cc:2243] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.177039 20746 raft_consensus.cc:2272] T 430edce57310489986e4f739148461a6 P 34f3b83da84448bf8aac62b37a1486eb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.180454 20746 tablet_replica.cc:333] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.181424 20746 raft_consensus.cc:2243] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.183259 20746 raft_consensus.cc:2272] T d41480ad8e164cf2aa88df180060345c P 34f3b83da84448bf8aac62b37a1486eb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.188016 20746 tablet_replica.cc:333] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.188910 20746 raft_consensus.cc:2243] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.189781 20746 raft_consensus.cc:2272] T f938584aad7a4b76be81dca03c9a647f P 34f3b83da84448bf8aac62b37a1486eb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.193553 20746 tablet_replica.cc:333] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.194401 20746 raft_consensus.cc:2243] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:04.195381 20746 raft_consensus.cc:2272] T 55c6a16f96334a6a8bd95d99ea2ffc7e P 34f3b83da84448bf8aac62b37a1486eb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.198859 20746 tablet_replica.cc:333] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb: stopping tablet replica
I20260709 13:09:04.199667 20746 raft_consensus.cc:2243] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.201412 20746 raft_consensus.cc:2272] T be9ae799c8f043d7b4fc384aa1edf5ce P 34f3b83da84448bf8aac62b37a1486eb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:05.205796 20746 tablet_server.cc:196] TabletServer@127.20.66.132:0 shutdown complete.
I20260709 13:09:05.310269 20746 master.cc:562] Master@127.20.66.190:46167 shutting down...
W20260709 13:09:05.362041 21806 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:09:05.362459 21806 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:09:05.362735 21806 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:09:17.125967 21742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111) [suppressed 370 similar messages]
W20260709 13:09:17.130354 21742 catalog_manager.cc:4755] TS 8f3b35a35257416db3eaaf364cf6b52b (127.20.66.129:38927): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676bc9491c5a4083bdd0b754d904c1d6: Network error: Client connection negotiation failed: client connection to 127.20.66.129:38927: connect: Connection refused (error 111)
W20260709 13:09:37.765496 21743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
W20260709 13:09:37.769660 21743 catalog_manager.cc:4755] TS 7d07aa0dcdd1484ab87925c9a4604b81 (127.20.66.131:36521): ChangeConfig:REMOVE_PEER RPC failed for tablet 9fc728770e5c4aa1ac36a662cebb8598: Network error: Client connection negotiation failed: client connection to 127.20.66.131:36521: connect: Connection refused (error 111)
I20260709 13:09:38.546769 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:38.548043 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:38.548492 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P 85f464a9537546118c8c1817bd5574c1: stopping tablet replica
I20260709 13:09:38.715189 20746 master.cc:584] Master@127.20.66.190:46167 shutdown complete.
I20260709 13:09:38.778316 20746 test_util.cc:182] -----------------------------------------------
I20260709 13:09:38.778590 20746 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602258489487-20746-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (296490 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:09:38.782737 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:41433
I20260709 13:09:38.784080 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:38.791169 30295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:38.791664 30298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:38.791664 30296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:38.793423 20746 server_base.cc:1061] running on GCE node
I20260709 13:09:38.798614 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:38.798884 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:38.799104 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602578799080 us; error 0 us; skew 500 ppm
I20260709 13:09:38.799728 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:38.802834 20746 webserver.cc:533] Webserver started at http://127.20.66.190:38577/ using document root <none> and password file <none>
I20260709 13:09:38.803426 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:38.803628 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:38.803972 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:38.805696 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "2230c368179f4041943f07933371a9ad"
format_stamp: "Formatted at 2026-07-09 13:09:38 on dist-test-slave-rgrd"
I20260709 13:09:38.811182 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:09:38.815517 30303 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:38.816561 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:09:38.816949 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "2230c368179f4041943f07933371a9ad"
format_stamp: "Formatted at 2026-07-09 13:09:38 on dist-test-slave-rgrd"
I20260709 13:09:38.817370 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:38.841972 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:38.843446 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:38.888877 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:41433
I20260709 13:09:38.889096 30354 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:41433 every 8 connection(s)
I20260709 13:09:38.893913 30355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:38.909237 30355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad: Bootstrap starting.
I20260709 13:09:38.915411 30355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:38.920739 30355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad: No bootstrap required, opened a new log
I20260709 13:09:38.923316 30355 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2230c368179f4041943f07933371a9ad" member_type: VOTER }
I20260709 13:09:38.923874 30355 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:38.924067 30355 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2230c368179f4041943f07933371a9ad, State: Initialized, Role: FOLLOWER
I20260709 13:09:38.924629 30355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2230c368179f4041943f07933371a9ad" member_type: VOTER }
I20260709 13:09:38.925165 30355 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:38.925395 30355 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:38.925618 30355 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:38.931424 30355 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2230c368179f4041943f07933371a9ad" member_type: VOTER }
I20260709 13:09:38.932006 30355 leader_election.cc:304] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2230c368179f4041943f07933371a9ad; no voters: 
I20260709 13:09:38.933182 30355 leader_election.cc:290] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:38.933553 30358 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:38.934945 30358 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 1 LEADER]: Becoming Leader. State: Replica: 2230c368179f4041943f07933371a9ad, State: Running, Role: LEADER
I20260709 13:09:38.935720 30358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2230c368179f4041943f07933371a9ad" member_type: VOTER }
I20260709 13:09:38.936235 30355 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:38.943233 30359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2230c368179f4041943f07933371a9ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2230c368179f4041943f07933371a9ad" member_type: VOTER } }
I20260709 13:09:38.943394 30360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2230c368179f4041943f07933371a9ad. Latest consensus state: current_term: 1 leader_uuid: "2230c368179f4041943f07933371a9ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2230c368179f4041943f07933371a9ad" member_type: VOTER } }
I20260709 13:09:38.944032 30359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:38.944173 30360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:38.946188 30368 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:38.951488 30368 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:38.952734 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:38.960801 30368 catalog_manager.cc:1383] Generated new cluster ID: d9b6135f937a433d9bc1b83853363527
I20260709 13:09:38.961163 30368 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:39.001164 30368 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:39.002460 30368 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:39.015482 30368 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad: Generated new TSK 0
I20260709 13:09:39.016186 30368 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:39.084344 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:39.090893 30376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.091959 30377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.101269 30379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.101893 20746 server_base.cc:1061] running on GCE node
I20260709 13:09:39.104460 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:39.104656 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:39.104869 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602579104836 us; error 0 us; skew 500 ppm
I20260709 13:09:39.105517 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:39.108078 20746 webserver.cc:533] Webserver started at http://127.20.66.129:36493/ using document root <none> and password file <none>
I20260709 13:09:39.108606 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:39.108769 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:39.109133 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:39.111132 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "5825cf9d14b4469c9960cca89b0010f1"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.117574 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:09:39.121068 30384 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.121901 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:09:39.122232 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "5825cf9d14b4469c9960cca89b0010f1"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.122546 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:39.135226 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:39.136793 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:39.139019 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:39.141557 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:39.141745 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.142053 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:39.142226 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.184304 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:34915
I20260709 13:09:39.184423 30446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:34915 every 8 connection(s)
I20260709 13:09:39.189648 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:39.198016 30451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.200147 30452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.201632 30454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.202311 20746 server_base.cc:1061] running on GCE node
I20260709 13:09:39.204941 30447 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41433
I20260709 13:09:39.205420 30447 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:39.205952 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:39.206218 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:39.206316 30447 heartbeater.cc:507] Master 127.20.66.190:41433 requested a full tablet report, sending...
I20260709 13:09:39.206406 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602579206390 us; error 0 us; skew 500 ppm
I20260709 13:09:39.207221 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:39.208839 30320 ts_manager.cc:194] Registered new tserver with Master: 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:39.210393 20746 webserver.cc:533] Webserver started at http://127.20.66.130:37163/ using document root <none> and password file <none>
I20260709 13:09:39.210935 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:39.211171 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:39.211007 30320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36812
I20260709 13:09:39.211520 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:39.212990 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "19b1445cece541d3b2abe177283fef44"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.217579 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:39.221556 30459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.222409 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:09:39.222844 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "19b1445cece541d3b2abe177283fef44"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.223201 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:39.235462 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:39.236739 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:39.238390 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:39.240644 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:39.240870 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.241123 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:39.241302 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.285995 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:35751
I20260709 13:09:39.286103 30521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:35751 every 8 connection(s)
I20260709 13:09:39.291695 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:39.299305 30525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.299922 30526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.303550 30522 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41433
I20260709 13:09:39.304091 30522 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:39.304492 20746 server_base.cc:1061] running on GCE node
W20260709 13:09:39.304795 30528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.305254 30522 heartbeater.cc:507] Master 127.20.66.190:41433 requested a full tablet report, sending...
I20260709 13:09:39.305913 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:39.306180 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:39.306377 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602579306361 us; error 0 us; skew 500 ppm
I20260709 13:09:39.307098 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:39.307643 30320 ts_manager.cc:194] Registered new tserver with Master: 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:39.309597 30320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36826
I20260709 13:09:39.310369 20746 webserver.cc:533] Webserver started at http://127.20.66.131:43509/ using document root <none> and password file <none>
I20260709 13:09:39.311084 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:39.311269 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:39.311604 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:39.312834 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "955d2e588381448981f85fadf225aa05"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.317668 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:39.321473 30533 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.322396 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:09:39.322808 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "955d2e588381448981f85fadf225aa05"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.323107 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:39.348315 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:39.349704 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:39.351332 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:39.353804 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:39.354058 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.354310 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:39.354487 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.399777 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:36261
I20260709 13:09:39.399909 30595 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:36261 every 8 connection(s)
I20260709 13:09:39.404809 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:39.413390 30599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.415345 30600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.418236 20746 server_base.cc:1061] running on GCE node
W20260709 13:09:39.418856 30602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.420091 30596 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41433
I20260709 13:09:39.420174 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:39.420608 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:39.420670 30596 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:39.420903 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602579420883 us; error 0 us; skew 500 ppm
I20260709 13:09:39.421813 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:39.421743 30596 heartbeater.cc:507] Master 127.20.66.190:41433 requested a full tablet report, sending...
I20260709 13:09:39.424798 30320 ts_manager.cc:194] Registered new tserver with Master: 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:39.425565 20746 webserver.cc:533] Webserver started at http://127.20.66.132:35481/ using document root <none> and password file <none>
I20260709 13:09:39.426278 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:39.426525 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:39.426844 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:39.426947 30320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36828
I20260709 13:09:39.428481 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-3-root/instance:
uuid: "b897660686b3465390a5bad32efea13c"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.433557 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:09:39.437337 30607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.438464 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:09:39.438894 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-3-root
uuid: "b897660686b3465390a5bad32efea13c"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:39.439249 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:39.460952 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:39.462486 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:39.463996 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:39.466598 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:39.466856 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.467137 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:39.467351 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.510926 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.132:45099
I20260709 13:09:39.511108 30669 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.132:45099 every 8 connection(s)
I20260709 13:09:39.526937 30670 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41433
I20260709 13:09:39.527474 30670 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:39.528436 30670 heartbeater.cc:507] Master 127.20.66.190:41433 requested a full tablet report, sending...
I20260709 13:09:39.530591 30320 ts_manager.cc:194] Registered new tserver with Master: b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:39.530763 20746 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016676971s
I20260709 13:09:39.532032 30320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36842
I20260709 13:09:39.559108 30320 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36858:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:09:39.815788 30411 tablet_service.cc:1511] Processing CreateTablet for tablet 8391e1197cab41039a7daf01b4cfb72b (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:09:39.817185 30412 tablet_service.cc:1511] Processing CreateTablet for tablet d5b071d1d18845b8828943fac633def7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:09:39.818229 30410 tablet_service.cc:1511] Processing CreateTablet for tablet 1213ccbc173c43f683d18d933293e4de (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:09:39.819501 30412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b071d1d18845b8828943fac633def7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.819938 30408 tablet_service.cc:1511] Processing CreateTablet for tablet 6664fcb5ba4f487b957d5cc98ec559de (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:09:39.820696 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 06e3f31e48cd40e890350bcb72063bf7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:09:39.821611 30408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6664fcb5ba4f487b957d5cc98ec559de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.822600 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8391e1197cab41039a7daf01b4cfb72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.829476 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06e3f31e48cd40e890350bcb72063bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.830603 30407 tablet_service.cc:1511] Processing CreateTablet for tablet c09468889ceb4dd490e2cbd2c245b3d9 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:09:39.832270 30407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c09468889ceb4dd490e2cbd2c245b3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.833500 30410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1213ccbc173c43f683d18d933293e4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.852659 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 730e52dc15a842e8a606385755e5558e (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:09:39.854338 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 730e52dc15a842e8a606385755e5558e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.877817 30487 tablet_service.cc:1511] Processing CreateTablet for tablet d5b071d1d18845b8828943fac633def7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:09:39.879932 30405 tablet_service.cc:1511] Processing CreateTablet for tablet 289ca1e3324041acb6ac3410039210be (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:09:39.880635 30487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b071d1d18845b8828943fac633def7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.881500 30405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 289ca1e3324041acb6ac3410039210be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.893054 30404 tablet_service.cc:1511] Processing CreateTablet for tablet 78e63f8484d24f8f9bc5c94fef442265 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:09:39.894693 30404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78e63f8484d24f8f9bc5c94fef442265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.896559 30484 tablet_service.cc:1511] Processing CreateTablet for tablet 06e3f31e48cd40e890350bcb72063bf7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:09:39.897099 30485 tablet_service.cc:1511] Processing CreateTablet for tablet 309754d072834ef381a0c8a6121b05cb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:09:39.898294 30484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06e3f31e48cd40e890350bcb72063bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.898423 30486 tablet_service.cc:1511] Processing CreateTablet for tablet 8391e1197cab41039a7daf01b4cfb72b (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:09:39.899246 30485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309754d072834ef381a0c8a6121b05cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.900223 30486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8391e1197cab41039a7daf01b4cfb72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.904661 30483 tablet_service.cc:1511] Processing CreateTablet for tablet 6664fcb5ba4f487b957d5cc98ec559de (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:09:39.906591 30483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6664fcb5ba4f487b957d5cc98ec559de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.909651 30482 tablet_service.cc:1511] Processing CreateTablet for tablet bd0df1d226d24b358d6d328eb07552ba (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:09:39.911522 30482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0df1d226d24b358d6d328eb07552ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.931192 30690 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:39.950219 30560 tablet_service.cc:1511] Processing CreateTablet for tablet 1213ccbc173c43f683d18d933293e4de (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:09:39.952549 30560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1213ccbc173c43f683d18d933293e4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.954923 30561 tablet_service.cc:1511] Processing CreateTablet for tablet 8391e1197cab41039a7daf01b4cfb72b (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:09:39.957453 30561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8391e1197cab41039a7daf01b4cfb72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.960708 30559 tablet_service.cc:1511] Processing CreateTablet for tablet 309754d072834ef381a0c8a6121b05cb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:09:39.962424 30559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309754d072834ef381a0c8a6121b05cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.964905 30558 tablet_service.cc:1511] Processing CreateTablet for tablet 06e3f31e48cd40e890350bcb72063bf7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:09:39.966562 30558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06e3f31e48cd40e890350bcb72063bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.968488 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 730e52dc15a842e8a606385755e5558e (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:09:39.970075 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 730e52dc15a842e8a606385755e5558e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.969959 30557 tablet_service.cc:1511] Processing CreateTablet for tablet c09468889ceb4dd490e2cbd2c245b3d9 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:09:39.970283 30556 tablet_service.cc:1511] Processing CreateTablet for tablet bd0df1d226d24b358d6d328eb07552ba (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
W20260709 13:09:39.968928 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:39.971868 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c09468889ceb4dd490e2cbd2c245b3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.973731 30556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0df1d226d24b358d6d328eb07552ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.974277 30555 tablet_service.cc:1511] Processing CreateTablet for tablet 730e52dc15a842e8a606385755e5558e (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:09:39.975728 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 730e52dc15a842e8a606385755e5558e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.978328 30554 tablet_service.cc:1511] Processing CreateTablet for tablet 78e63f8484d24f8f9bc5c94fef442265 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:09:39.979796 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78e63f8484d24f8f9bc5c94fef442265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.983620 30553 tablet_service.cc:1511] Processing CreateTablet for tablet e003633f9bf24b09b3789ddc3ac48cc1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:09:39.985265 30553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e003633f9bf24b09b3789ddc3ac48cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:39.994447 30480 tablet_service.cc:1511] Processing CreateTablet for tablet 289ca1e3324041acb6ac3410039210be (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:09:39.996044 30480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 289ca1e3324041acb6ac3410039210be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.002779 30479 tablet_service.cc:1511] Processing CreateTablet for tablet e003633f9bf24b09b3789ddc3ac48cc1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:09:40.004252 30479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e003633f9bf24b09b3789ddc3ac48cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.014743 30635 tablet_service.cc:1511] Processing CreateTablet for tablet d5b071d1d18845b8828943fac633def7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:09:40.016409 30635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b071d1d18845b8828943fac633def7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.022982 30634 tablet_service.cc:1511] Processing CreateTablet for tablet 1213ccbc173c43f683d18d933293e4de (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:09:40.023582 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 9035baa441924d8b9218b26855598f19 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:09:40.024677 30634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1213ccbc173c43f683d18d933293e4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.025085 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9035baa441924d8b9218b26855598f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.034979 30633 tablet_service.cc:1511] Processing CreateTablet for tablet 309754d072834ef381a0c8a6121b05cb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:09:40.036554 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309754d072834ef381a0c8a6121b05cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.042889 30632 tablet_service.cc:1511] Processing CreateTablet for tablet 6664fcb5ba4f487b957d5cc98ec559de (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:09:40.042943 30692 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:40.044497 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6664fcb5ba4f487b957d5cc98ec559de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.051801 30631 tablet_service.cc:1511] Processing CreateTablet for tablet c09468889ceb4dd490e2cbd2c245b3d9 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:09:40.054131 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c09468889ceb4dd490e2cbd2c245b3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.060675 30692 tablet_bootstrap.cc:654] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.063112 30478 tablet_service.cc:1511] Processing CreateTablet for tablet 9035baa441924d8b9218b26855598f19 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:09:40.064558 30478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9035baa441924d8b9218b26855598f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.068742 30411 tablet_service.cc:1511] Processing CreateTablet for tablet 6f17363c79974c6a863b51a45eef6e0e (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:09:40.070391 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f17363c79974c6a863b51a45eef6e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.077589 30557 tablet_service.cc:1511] Processing CreateTablet for tablet d4871109603643fe996c7bfd323672c7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:09:40.078645 30630 tablet_service.cc:1511] Processing CreateTablet for tablet bd0df1d226d24b358d6d328eb07552ba (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:09:40.079463 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4871109603643fe996c7bfd323672c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.080049 30630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0df1d226d24b358d6d328eb07552ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.090889 30406 tablet_service.cc:1511] Processing CreateTablet for tablet d4871109603643fe996c7bfd323672c7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:09:40.091233 30629 tablet_service.cc:1511] Processing CreateTablet for tablet 289ca1e3324041acb6ac3410039210be (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:09:40.092344 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4871109603643fe996c7bfd323672c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.092896 30629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 289ca1e3324041acb6ac3410039210be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.094004 30556 tablet_service.cc:1511] Processing CreateTablet for tablet 845d59ad223a4f6c9c388017a170527b (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:09:40.095305 30556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845d59ad223a4f6c9c388017a170527b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.102983 30628 tablet_service.cc:1511] Processing CreateTablet for tablet 78e63f8484d24f8f9bc5c94fef442265 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:09:40.104647 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78e63f8484d24f8f9bc5c94fef442265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.114867 30627 tablet_service.cc:1511] Processing CreateTablet for tablet e003633f9bf24b09b3789ddc3ac48cc1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:09:40.116542 30627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e003633f9bf24b09b3789ddc3ac48cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.126891 30626 tablet_service.cc:1511] Processing CreateTablet for tablet 6f17363c79974c6a863b51a45eef6e0e (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:09:40.128424 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f17363c79974c6a863b51a45eef6e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.141582 30560 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4e2ca1d8a742e1a0beb1710464b3bf (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:09:40.143371 30560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4e2ca1d8a742e1a0beb1710464b3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.159466 30690 tablet_bootstrap.cc:654] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.183655 30558 tablet_service.cc:1511] Processing CreateTablet for tablet 9035baa441924d8b9218b26855598f19 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:09:40.193894 30558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9035baa441924d8b9218b26855598f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.214780 30447 heartbeater.cc:499] Master 127.20.66.190:41433 was elected leader, sending a full tablet report...
I20260709 13:09:40.238415 30554 tablet_service.cc:1511] Processing CreateTablet for tablet 47885caab3e14ff2ae88840a9dfc0944 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:09:40.240025 30557 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad2905df5e84f04bcc21f6c9713c744 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:09:40.250312 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 6f17363c79974c6a863b51a45eef6e0e (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:09:40.251680 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f17363c79974c6a863b51a45eef6e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.257256 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad2905df5e84f04bcc21f6c9713c744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.263214 30692 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:40.263792 30692 ts_tablet_manager.cc:1403] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.221s	user 0.013s	sys 0.035s
I20260709 13:09:40.267313 30692 raft_consensus.cc:359] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:40.268091 30692 raft_consensus.cc:385] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.268572 30692 raft_consensus.cc:740] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.269542 30478 tablet_service.cc:1511] Processing CreateTablet for tablet 845d59ad223a4f6c9c388017a170527b (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:09:40.271415 30628 tablet_service.cc:1511] Processing CreateTablet for tablet d4871109603643fe996c7bfd323672c7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:09:40.267829 30626 tablet_service.cc:1511] Processing CreateTablet for tablet 845d59ad223a4f6c9c388017a170527b (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:09:40.275429 30411 tablet_service.cc:1511] Processing CreateTablet for tablet d7ea982486944f2fbb7ce66208002c94 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:09:40.275616 30405 tablet_service.cc:1511] Processing CreateTablet for tablet 47885caab3e14ff2ae88840a9dfc0944 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:09:40.280231 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7ea982486944f2fbb7ce66208002c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.282315 30691 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:40.284409 30405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47885caab3e14ff2ae88840a9dfc0944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.277100 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 95d00bccf5ce41f4b601bba7d235ffbb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:09:40.291278 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95d00bccf5ce41f4b601bba7d235ffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.294360 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47885caab3e14ff2ae88840a9dfc0944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.294669 30484 tablet_service.cc:1511] Processing CreateTablet for tablet d7ea982486944f2fbb7ce66208002c94 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:09:40.260875 30487 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4e2ca1d8a742e1a0beb1710464b3bf (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:09:40.299113 30691 tablet_bootstrap.cc:654] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.274492 30408 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4e2ca1d8a742e1a0beb1710464b3bf (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:09:40.302395 30408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4e2ca1d8a742e1a0beb1710464b3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.302986 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845d59ad223a4f6c9c388017a170527b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.307966 30631 tablet_service.cc:1511] Processing CreateTablet for tablet d7ea982486944f2fbb7ce66208002c94 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:09:40.308280 30629 tablet_service.cc:1511] Processing CreateTablet for tablet 47885caab3e14ff2ae88840a9dfc0944 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:09:40.310030 30629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47885caab3e14ff2ae88840a9dfc0944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.312016 30635 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad2905df5e84f04bcc21f6c9713c744 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:09:40.312717 30522 heartbeater.cc:499] Master 127.20.66.190:41433 was elected leader, sending a full tablet report...
I20260709 13:09:40.284397 30692 consensus_queue.cc:260] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:40.323293 30690 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:40.323889 30690 ts_tablet_manager.cc:1403] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.393s	user 0.012s	sys 0.011s
I20260709 13:09:40.323501 30479 tablet_service.cc:1511] Processing CreateTablet for tablet 95d00bccf5ce41f4b601bba7d235ffbb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:09:40.327037 30690 raft_consensus.cc:359] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:40.327790 30690 raft_consensus.cc:385] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.328117 30690 raft_consensus.cc:740] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.335240 30561 tablet_service.cc:1511] Processing CreateTablet for tablet 95d00bccf5ce41f4b601bba7d235ffbb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:09:40.336678 30561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95d00bccf5ce41f4b601bba7d235ffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.341197 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7ea982486944f2fbb7ce66208002c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.341502 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 24a265170a904a89a2801145c1a61172 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:09:40.342782 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24a265170a904a89a2801145c1a61172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.345366 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4871109603643fe996c7bfd323672c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.346809 30635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad2905df5e84f04bcc21f6c9713c744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.349686 30478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845d59ad223a4f6c9c388017a170527b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.350826 30484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7ea982486944f2fbb7ce66208002c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.358101 30487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4e2ca1d8a742e1a0beb1710464b3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.362030 30627 tablet_service.cc:1511] Processing CreateTablet for tablet 24a265170a904a89a2801145c1a61172 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:09:40.363416 30627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24a265170a904a89a2801145c1a61172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.365319 30479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95d00bccf5ce41f4b601bba7d235ffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.369077 30486 tablet_service.cc:1511] Processing CreateTablet for tablet 24a265170a904a89a2801145c1a61172 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:09:40.351785 30485 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad2905df5e84f04bcc21f6c9713c744 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:09:40.372310 30690 consensus_queue.cc:260] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:40.390487 30486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24a265170a904a89a2801145c1a61172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.392956 30695 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:40.399236 30557 tablet_service.cc:1511] Processing CreateTablet for tablet bbc578fa661c4935aae4aece586f4271 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:09:40.399420 30632 tablet_service.cc:1511] Processing CreateTablet for tablet bbc578fa661c4935aae4aece586f4271 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:09:40.400791 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc578fa661c4935aae4aece586f4271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.401355 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc578fa661c4935aae4aece586f4271. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:40.405611 30597 tablet.cc:2406] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:09:40.406224 30596 heartbeater.cc:499] Master 127.20.66.190:41433 was elected leader, sending a full tablet report...
I20260709 13:09:40.402370 30692 ts_tablet_manager.cc:1434] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.138s	user 0.011s	sys 0.007s
I20260709 13:09:40.409443 30692 tablet_bootstrap.cc:492] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:40.412730 30695 tablet_bootstrap.cc:654] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.437867 30692 tablet_bootstrap.cc:654] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.445807 30411 tablet_service.cc:1511] Processing CreateTablet for tablet bbc578fa661c4935aae4aece586f4271 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:09:40.454717 30633 tablet_service.cc:1511] Processing CreateTablet for tablet 107dd7c115d0402a9fd4e3312f621930 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:09:40.437906 30485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad2905df5e84f04bcc21f6c9713c744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.454782 30690 ts_tablet_manager.cc:1434] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.131s	user 0.019s	sys 0.018s
I20260709 13:09:40.475601 30690 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:40.471627 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc578fa661c4935aae4aece586f4271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.471627 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 107dd7c115d0402a9fd4e3312f621930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.477274 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1c1fe97f07496c92182b3065536c15 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:09:40.479857 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1c1fe97f07496c92182b3065536c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.486534 30483 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1c1fe97f07496c92182b3065536c15 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:09:40.489384 30554 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1c1fe97f07496c92182b3065536c15 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:09:40.490685 30483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1c1fe97f07496c92182b3065536c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.490688 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1c1fe97f07496c92182b3065536c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.500387 30631 tablet_service.cc:1511] Processing CreateTablet for tablet fdb6f3b113b34e3c8bce234b33434313 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:09:40.501751 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb6f3b113b34e3c8bce234b33434313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.485316 30690 tablet_bootstrap.cc:654] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.451927 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 107dd7c115d0402a9fd4e3312f621930 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:09:40.506225 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 107dd7c115d0402a9fd4e3312f621930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.517390 30695 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:40.518149 30695 ts_tablet_manager.cc:1403] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.129s	user 0.009s	sys 0.009s
I20260709 13:09:40.453140 30558 tablet_service.cc:1511] Processing CreateTablet for tablet 107dd7c115d0402a9fd4e3312f621930 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:09:40.527494 30558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 107dd7c115d0402a9fd4e3312f621930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.531068 30484 tablet_service.cc:1511] Processing CreateTablet for tablet fdb6f3b113b34e3c8bce234b33434313 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:09:40.535703 30670 heartbeater.cc:499] Master 127.20.66.190:41433 was elected leader, sending a full tablet report...
I20260709 13:09:40.547209 30408 tablet_service.cc:1511] Processing CreateTablet for tablet fdb6f3b113b34e3c8bce234b33434313 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:09:40.548635 30408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb6f3b113b34e3c8bce234b33434313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.532892 30484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb6f3b113b34e3c8bce234b33434313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.564273 30561 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2c6fa20b2b4325804ef77b1d2172bf (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:09:40.566154 30561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2c6fa20b2b4325804ef77b1d2172bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.576869 30697 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.577636 30697 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:40.585035 30696 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.585575 30696 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:40.588097 30696 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:40.603060 30691 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:40.603762 30691 ts_tablet_manager.cc:1403] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.322s	user 0.011s	sys 0.016s
I20260709 13:09:40.604869 30697 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:40.605136 30695 raft_consensus.cc:359] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.606138 30695 raft_consensus.cc:385] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.606528 30695 raft_consensus.cc:740] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.607486 30695 consensus_queue.cc:260] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.623696 30632 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2c6fa20b2b4325804ef77b1d2172bf (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:09:40.624459 30690 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:40.625196 30690 ts_tablet_manager.cc:1403] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.150s	user 0.015s	sys 0.002s
I20260709 13:09:40.625619 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2c6fa20b2b4325804ef77b1d2172bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.627225 30692 tablet_bootstrap.cc:492] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:40.627737 30692 ts_tablet_manager.cc:1403] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.219s	user 0.037s	sys 0.064s
I20260709 13:09:40.628595 30690 raft_consensus.cc:359] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.629392 30690 raft_consensus.cc:385] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.629693 30690 raft_consensus.cc:740] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.632479 30690 consensus_queue.cc:260] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.633226 30692 raft_consensus.cc:359] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:40.634008 30692 raft_consensus.cc:385] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.634295 30692 raft_consensus.cc:740] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.634728 30690 ts_tablet_manager.cc:1434] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:09:40.635021 30692 consensus_queue.cc:260] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:40.647248 30691 raft_consensus.cc:359] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:40.624573 30486 tablet_service.cc:1511] Processing CreateTablet for tablet dc9c275f1b334c7db06956d0941b04ed (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:09:40.656343 30709 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.659405 30631 tablet_service.cc:1511] Processing CreateTablet for tablet dc9c275f1b334c7db06956d0941b04ed (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:09:40.656422 30690 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:40.664077 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2c6fa20b2b4325804ef77b1d2172bf (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:09:40.666392 30692 ts_tablet_manager.cc:1434] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 13:09:40.669713 30692 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:40.669795 30486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc9c275f1b334c7db06956d0941b04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.668720 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc9c275f1b334c7db06956d0941b04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.675031 30405 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac46eb166db448cb0c6f504819da525 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:09:40.666875 30691 raft_consensus.cc:385] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.677333 30691 raft_consensus.cc:740] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.681211 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac46eb166db448cb0c6f504819da525 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:09:40.682771 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac46eb166db448cb0c6f504819da525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.685307 30691 consensus_queue.cc:260] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:40.682859 30695 ts_tablet_manager.cc:1434] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.164s	user 0.016s	sys 0.011s
I20260709 13:09:40.690019 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2c6fa20b2b4325804ef77b1d2172bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.693619 30405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac46eb166db448cb0c6f504819da525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.701246 30633 tablet_service.cc:1511] Processing CreateTablet for tablet 86637a5e467340ae9ac4f57d22c68bf6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:09:40.703753 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86637a5e467340ae9ac4f57d22c68bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.708516 30692 tablet_bootstrap.cc:654] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.711277 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:40.714003 30536 leader_election.cc:343] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.718093 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 86637a5e467340ae9ac4f57d22c68bf6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:09:40.719434 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86637a5e467340ae9ac4f57d22c68bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.726969 30485 tablet_service.cc:1511] Processing CreateTablet for tablet 86637a5e467340ae9ac4f57d22c68bf6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:09:40.728366 30485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86637a5e467340ae9ac4f57d22c68bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.735975 30691 ts_tablet_manager.cc:1434] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.132s	user 0.014s	sys 0.003s
I20260709 13:09:40.688416 30709 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.744755 30632 tablet_service.cc:1511] Processing CreateTablet for tablet ec6ae4a76cb84d41a9dcc0c1cce9cba2 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:09:40.746333 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6ae4a76cb84d41a9dcc0c1cce9cba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.755942 30695 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:40.742697 30691 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:40.766469 30709 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:40.767059 30690 tablet_bootstrap.cc:654] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.767731 30561 tablet_service.cc:1511] Processing CreateTablet for tablet ec6ae4a76cb84d41a9dcc0c1cce9cba2 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:09:40.767880 30558 tablet_service.cc:1511] Processing CreateTablet for tablet dc9c275f1b334c7db06956d0941b04ed (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:09:40.789903 30554 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac46eb166db448cb0c6f504819da525 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:09:40.791791 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac46eb166db448cb0c6f504819da525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.794669 30561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6ae4a76cb84d41a9dcc0c1cce9cba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.801836 30558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc9c275f1b334c7db06956d0941b04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.804878 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:40.807358 30388 leader_election.cc:343] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.825115 30710 raft_consensus.cc:493] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.825776 30710 raft_consensus.cc:515] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:40.826573 30411 tablet_service.cc:1511] Processing CreateTablet for tablet ec6ae4a76cb84d41a9dcc0c1cce9cba2 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:09:40.828385 30710 leader_election.cc:290] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:40.828536 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6ae4a76cb84d41a9dcc0c1cce9cba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.844046 30557 tablet_service.cc:1511] Processing CreateTablet for tablet e78ade21dd6d46a3a18152943df69389 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:09:40.844241 30633 tablet_service.cc:1511] Processing CreateTablet for tablet e78ade21dd6d46a3a18152943df69389 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:09:40.844549 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 93a6f0d4b4c848deb76000430c8bb0a5 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:09:40.849886 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a6f0d4b4c848deb76000430c8bb0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.852270 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:40.853955 30695 tablet_bootstrap.cc:654] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:40.854264 30386 leader_election.cc:343] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.854815 30692 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:40.854923 30386 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:40.855410 30692 ts_tablet_manager.cc:1403] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.186s	user 0.008s	sys 0.009s
I20260709 13:09:40.856331 30697 raft_consensus.cc:2749] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:40.857122 30690 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:40.857177 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ade21dd6d46a3a18152943df69389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.860881 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:40.863130 30692 raft_consensus.cc:359] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:40.864073 30692 raft_consensus.cc:385] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.864452 30692 raft_consensus.cc:740] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
W20260709 13:09:40.864810 30535 leader_election.cc:343] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.865633 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ade21dd6d46a3a18152943df69389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.867615 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:40.869334 30535 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:40.870558 30696 raft_consensus.cc:2749] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:40.868091 30691 tablet_bootstrap.cc:654] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:40.887759 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 93a6f0d4b4c848deb76000430c8bb0a5 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:09:40.888921 30696 raft_consensus.cc:493] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.889158 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a6f0d4b4c848deb76000430c8bb0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.889470 30696 raft_consensus.cc:515] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:40.891886 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 726e825b67364fccb3b417c19afe5e14 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:09:40.893251 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 726e825b67364fccb3b417c19afe5e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.893977 30555 tablet_service.cc:1511] Processing CreateTablet for tablet 93a6f0d4b4c848deb76000430c8bb0a5 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:09:40.896500 30628 tablet_service.cc:1511] Processing CreateTablet for tablet 726e825b67364fccb3b417c19afe5e14 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260709 13:09:40.897621 30610 leader_election.cc:343] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.897956 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 726e825b67364fccb3b417c19afe5e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.902063 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a6f0d4b4c848deb76000430c8bb0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.904289 30697 raft_consensus.cc:493] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.904798 30697 raft_consensus.cc:515] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.884840 30484 tablet_service.cc:1511] Processing CreateTablet for tablet e78ade21dd6d46a3a18152943df69389 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:09:40.906934 30484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78ade21dd6d46a3a18152943df69389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.880825 30690 ts_tablet_manager.cc:1403] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.231s	user 0.007s	sys 0.032s
I20260709 13:09:40.916616 30690 raft_consensus.cc:359] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.917294 30690 raft_consensus.cc:385] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:40.917558 30690 raft_consensus.cc:740] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:40.918282 30690 consensus_queue.cc:260] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:40.911407 30697 leader_election.cc:290] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:40.920490 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309754d072834ef381a0c8a6121b05cb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:40.920785 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:40.922005 30535 leader_election.cc:343] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:40.922425 30386 leader_election.cc:343] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.891920 30696 leader_election.cc:290] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:40.932585 30485 tablet_service.cc:1511] Processing CreateTablet for tablet 726e825b67364fccb3b417c19afe5e14 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:09:40.934027 30485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 726e825b67364fccb3b417c19afe5e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.937064 30692 consensus_queue.cc:260] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:40.939085 30557 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca8dcce43e444f8ada68dae5eecc1f6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:09:40.940469 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca8dcce43e444f8ada68dae5eecc1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.947440 30691 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:40.948024 30691 ts_tablet_manager.cc:1403] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.211s	user 0.012s	sys 0.018s
I20260709 13:09:40.962206 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:40.944877 30692 ts_tablet_manager.cc:1434] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.003s
I20260709 13:09:40.971540 30486 tablet_service.cc:1511] Processing CreateTablet for tablet 7a582514dd6e459fa48675d41a9ff078 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:09:40.966912 30632 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca8dcce43e444f8ada68dae5eecc1f6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 13:09:40.973692 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:40.974149 30486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a582514dd6e459fa48675d41a9ff078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:40.974498 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca8dcce43e444f8ada68dae5eecc1f6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:40.965392 30611 leader_election.cc:343] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:40.975281 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06e3f31e48cd40e890350bcb72063bf7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:40.975610 30611 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:40.976055 30571 raft_consensus.cc:2468] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:40.976586 30709 raft_consensus.cc:2749] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:40.978240 30463 leader_election.cc:304] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:40.980154 30710 raft_consensus.cc:2804] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:40.980561 30710 raft_consensus.cc:493] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.980929 30710 raft_consensus.cc:3060] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:40.982304 30558 tablet_service.cc:1511] Processing CreateTablet for tablet 7a582514dd6e459fa48675d41a9ff078 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:09:41.003315 30558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a582514dd6e459fa48675d41a9ff078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.003407 30631 tablet_service.cc:1511] Processing CreateTablet for tablet 7a582514dd6e459fa48675d41a9ff078 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:09:41.004907 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a582514dd6e459fa48675d41a9ff078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.008977 30487 tablet_service.cc:1511] Processing CreateTablet for tablet c678eac1173e4cdaa75a8279bc764f29 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:09:41.010241 30690 ts_tablet_manager.cc:1434] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.096s	user 0.005s	sys 0.001s
I20260709 13:09:41.010640 30487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c678eac1173e4cdaa75a8279bc764f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.011533 30690 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:41.011191 30409 tablet_service.cc:1511] Processing CreateTablet for tablet c678eac1173e4cdaa75a8279bc764f29 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:09:41.014957 30692 tablet_bootstrap.cc:492] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:41.015285 30554 tablet_service.cc:1511] Processing CreateTablet for tablet c678eac1173e4cdaa75a8279bc764f29 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:09:41.016605 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c678eac1173e4cdaa75a8279bc764f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.018879 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c678eac1173e4cdaa75a8279bc764f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.021623 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309754d072834ef381a0c8a6121b05cb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:41.024799 30537 leader_election.cc:343] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.025928 30537 leader_election.cc:304] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:41.027953 30406 tablet_service.cc:1511] Processing CreateTablet for tablet c69a30e3a4a8473a9f1f8652ea9463d8 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:09:41.028808 30696 raft_consensus.cc:2749] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.029310 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a30e3a4a8473a9f1f8652ea9463d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.025944 30405 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca8dcce43e444f8ada68dae5eecc1f6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:09:41.031997 30405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca8dcce43e444f8ada68dae5eecc1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.033674 30483 tablet_service.cc:1511] Processing CreateTablet for tablet c69a30e3a4a8473a9f1f8652ea9463d8 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:09:41.035007 30483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a30e3a4a8473a9f1f8652ea9463d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.039315 30691 raft_consensus.cc:359] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.040156 30691 raft_consensus.cc:385] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.040457 30691 raft_consensus.cc:740] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.048441 30553 tablet_service.cc:1511] Processing CreateTablet for tablet ba4cf9aa835b48038aa80c194df56bd6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:09:41.050616 30553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4cf9aa835b48038aa80c194df56bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.053532 30691 consensus_queue.cc:260] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.053931 30628 tablet_service.cc:1511] Processing CreateTablet for tablet c69a30e3a4a8473a9f1f8652ea9463d8 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:09:41.055275 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69a30e3a4a8473a9f1f8652ea9463d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.064481 30626 tablet_service.cc:1511] Processing CreateTablet for tablet ba4cf9aa835b48038aa80c194df56bd6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:09:41.066004 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4cf9aa835b48038aa80c194df56bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.066373 30692 tablet_bootstrap.cc:654] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.069875 30710 raft_consensus.cc:515] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.070577 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:41.071807 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06e3f31e48cd40e890350bcb72063bf7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:41.072141 30388 leader_election.cc:343] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.072623 30388 leader_election.cc:304] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
W20260709 13:09:41.073369 30462 leader_election.cc:343] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.074471 30723 raft_consensus.cc:2749] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.075625 30690 tablet_bootstrap.cc:654] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.091738 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06e3f31e48cd40e890350bcb72063bf7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
W20260709 13:09:41.093878 30462 leader_election.cc:343] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.096586 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 37455107e57b47c8978a434d17f3c76c (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:09:41.096747 30411 tablet_service.cc:1511] Processing CreateTablet for tablet ba4cf9aa835b48038aa80c194df56bd6 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:09:41.097998 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37455107e57b47c8978a434d17f3c76c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.097998 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4cf9aa835b48038aa80c194df56bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.103374 30633 tablet_service.cc:1511] Processing CreateTablet for tablet 37455107e57b47c8978a434d17f3c76c (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:09:41.103449 30408 tablet_service.cc:1511] Processing CreateTablet for tablet a15b8193c82e4caeb06f0da37159665c (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:09:41.104911 30408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a15b8193c82e4caeb06f0da37159665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.106447 30557 tablet_service.cc:1511] Processing CreateTablet for tablet 37455107e57b47c8978a434d17f3c76c (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:09:41.107810 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37455107e57b47c8978a434d17f3c76c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.104833 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37455107e57b47c8978a434d17f3c76c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.121057 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06e3f31e48cd40e890350bcb72063bf7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:41.121717 30571 raft_consensus.cc:3060] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.131894 30571 raft_consensus.cc:2468] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:41.131778 30484 tablet_service.cc:1511] Processing CreateTablet for tablet a15b8193c82e4caeb06f0da37159665c (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:09:41.133430 30484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a15b8193c82e4caeb06f0da37159665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.135076 30695 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:41.135625 30695 ts_tablet_manager.cc:1403] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.380s	user 0.011s	sys 0.015s
I20260709 13:09:41.118106 30710 leader_election.cc:290] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:41.139480 30463 leader_election.cc:304] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:41.140527 30725 raft_consensus.cc:2804] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:41.148051 30710 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.148550 30710 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.151346 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:41.152766 30462 leader_election.cc:343] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.153774 30710 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:41.155284 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:41.156975 30463 leader_election.cc:343] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.157435 30463 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:41.158187 30710 raft_consensus.cc:2749] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.159973 30695 raft_consensus.cc:359] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.167268 30690 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:41.167841 30690 ts_tablet_manager.cc:1403] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.157s	user 0.009s	sys 0.009s
I20260709 13:09:41.171175 30690 raft_consensus.cc:359] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.171967 30690 raft_consensus.cc:385] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.172302 30690 raft_consensus.cc:740] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.173234 30690 consensus_queue.cc:260] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.160857 30695 raft_consensus.cc:385] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.176743 30695 raft_consensus.cc:740] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.177691 30557 tablet_service.cc:1511] Processing CreateTablet for tablet a15b8193c82e4caeb06f0da37159665c (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:09:41.178505 30695 consensus_queue.cc:260] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.179199 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a15b8193c82e4caeb06f0da37159665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.183471 30409 tablet_service.cc:1511] Processing CreateTablet for tablet b59eef71464c40618e5e26de6a40aa90 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:09:41.186048 30481 tablet_service.cc:1511] Processing CreateTablet for tablet b59eef71464c40618e5e26de6a40aa90 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:09:41.186795 30555 tablet_service.cc:1511] Processing CreateTablet for tablet 6e52d69d8f9044099d83595faf8a7c88 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:09:41.187419 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59eef71464c40618e5e26de6a40aa90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.188230 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e52d69d8f9044099d83595faf8a7c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.189520 30691 ts_tablet_manager.cc:1434] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.241s	user 0.022s	sys 0.004s
I20260709 13:09:41.190583 30691 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:41.192747 30690 ts_tablet_manager.cc:1434] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 13:09:41.194509 30690 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:41.193498 30725 raft_consensus.cc:697] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:41.197763 30725 consensus_queue.cc:237] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.187326 30632 tablet_service.cc:1511] Processing CreateTablet for tablet b59eef71464c40618e5e26de6a40aa90 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:09:41.202021 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59eef71464c40618e5e26de6a40aa90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.184944 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59eef71464c40618e5e26de6a40aa90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.207283 30692 tablet_bootstrap.cc:492] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:41.207839 30692 ts_tablet_manager.cc:1403] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.193s	user 0.017s	sys 0.001s
I20260709 13:09:41.208700 30691 tablet_bootstrap.cc:654] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.213279 30626 tablet_service.cc:1511] Processing CreateTablet for tablet 6e52d69d8f9044099d83595faf8a7c88 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:09:41.214319 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 6e52d69d8f9044099d83595faf8a7c88 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:09:41.215619 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e52d69d8f9044099d83595faf8a7c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.216476 30690 tablet_bootstrap.cc:654] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.214732 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e52d69d8f9044099d83595faf8a7c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.216830 30695 ts_tablet_manager.cc:1434] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.003s
I20260709 13:09:41.217725 30692 raft_consensus.cc:359] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.219143 30692 raft_consensus.cc:385] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.219628 30692 raft_consensus.cc:740] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.220592 30692 consensus_queue.cc:260] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.231874 30695 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:41.233019 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4b70f2baf64ae8a943a8bb8aa24385 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:09:41.234390 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4b70f2baf64ae8a943a8bb8aa24385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.234447 30557 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4b70f2baf64ae8a943a8bb8aa24385 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:09:41.235684 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4b70f2baf64ae8a943a8bb8aa24385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.249460 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 040a6253e94b4c19952ed01cb9d8c6a5 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:09:41.250725 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 040a6253e94b4c19952ed01cb9d8c6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.255080 30626 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4b70f2baf64ae8a943a8bb8aa24385 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:09:41.255566 30484 tablet_service.cc:1511] Processing CreateTablet for tablet 040a6253e94b4c19952ed01cb9d8c6a5 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:09:41.256544 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4b70f2baf64ae8a943a8bb8aa24385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.256845 30484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 040a6253e94b4c19952ed01cb9d8c6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.256618 30320 catalog_manager.cc:5697] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:41.271152 30557 tablet_service.cc:1511] Processing CreateTablet for tablet 040a6253e94b4c19952ed01cb9d8c6a5 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:09:41.272610 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 040a6253e94b4c19952ed01cb9d8c6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.273203 30628 tablet_service.cc:1511] Processing CreateTablet for tablet 366b9fff4017464a9b99ab4878dbb119 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:09:41.274487 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 366b9fff4017464a9b99ab4878dbb119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.271149 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 366b9fff4017464a9b99ab4878dbb119 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:09:41.279503 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 366b9fff4017464a9b99ab4878dbb119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.279685 30483 tablet_service.cc:1511] Processing CreateTablet for tablet 366b9fff4017464a9b99ab4878dbb119 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:09:41.280944 30483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 366b9fff4017464a9b99ab4878dbb119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.283167 30633 tablet_service.cc:1511] Processing CreateTablet for tablet 679bf6d4a8654a72b47b318645ab6fc0 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:09:41.284276 30411 tablet_service.cc:1511] Processing CreateTablet for tablet 679bf6d4a8654a72b47b318645ab6fc0 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:09:41.282445 30554 tablet_service.cc:1511] Processing CreateTablet for tablet 679bf6d4a8654a72b47b318645ab6fc0 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:09:41.285356 30692 ts_tablet_manager.cc:1434] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.002s
I20260709 13:09:41.285565 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679bf6d4a8654a72b47b318645ab6fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.285691 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679bf6d4a8654a72b47b318645ab6fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.285948 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679bf6d4a8654a72b47b318645ab6fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.296599 30695 tablet_bootstrap.cc:654] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.299299 30691 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:41.299963 30691 ts_tablet_manager.cc:1403] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.110s	user 0.009s	sys 0.004s
I20260709 13:09:41.287284 30487 tablet_service.cc:1511] Processing CreateTablet for tablet 92c56097e8594d298daed39079975dd1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:09:41.305151 30487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c56097e8594d298daed39079975dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.306941 30690 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:41.307651 30690 ts_tablet_manager.cc:1403] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.113s	user 0.009s	sys 0.007s
I20260709 13:09:41.303591 30691 raft_consensus.cc:359] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.310388 30691 raft_consensus.cc:385] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.310803 30691 raft_consensus.cc:740] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.313033 30691 consensus_queue.cc:260] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.286501 30692 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:41.321487 30691 ts_tablet_manager.cc:1434] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 13:09:41.329149 30725 raft_consensus.cc:493] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.329797 30725 raft_consensus.cc:515] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.330690 30690 raft_consensus.cc:359] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.331462 30690 raft_consensus.cc:385] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.331761 30690 raft_consensus.cc:740] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.332667 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "730e52dc15a842e8a606385755e5558e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:41.332578 30690 consensus_queue.cc:260] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.333431 30422 raft_consensus.cc:2468] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:41.334896 30690 ts_tablet_manager.cc:1434] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:09:41.334365 30725 leader_election.cc:290] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:41.334702 30632 tablet_service.cc:1511] Processing CreateTablet for tablet 92c56097e8594d298daed39079975dd1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:09:41.335942 30690 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:41.336681 30723 raft_consensus.cc:493] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.337226 30723 raft_consensus.cc:515] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.338438 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "730e52dc15a842e8a606385755e5558e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:41.339401 30723 leader_election.cc:290] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:41.336664 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c56097e8594d298daed39079975dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.333223 30406 tablet_service.cc:1511] Processing CreateTablet for tablet d204ad91a1c74eb9b3335582ee6717c7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:09:41.331126 30555 tablet_service.cc:1511] Processing CreateTablet for tablet 92c56097e8594d298daed39079975dd1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:09:41.343236 30462 leader_election.cc:304] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
W20260709 13:09:41.344900 30463 leader_election.cc:343] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.340909 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:41.347824 30388 leader_election.cc:343] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.343863 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d204ad91a1c74eb9b3335582ee6717c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.350319 30692 tablet_bootstrap.cc:654] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.354111 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:41.359480 30388 leader_election.cc:343] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.359920 30481 tablet_service.cc:1511] Processing CreateTablet for tablet d204ad91a1c74eb9b3335582ee6717c7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:09:41.361378 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d204ad91a1c74eb9b3335582ee6717c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.360077 30388 leader_election.cc:304] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:41.362869 30557 tablet_service.cc:1511] Processing CreateTablet for tablet d204ad91a1c74eb9b3335582ee6717c7 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:09:41.364346 30723 raft_consensus.cc:2749] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.364544 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d204ad91a1c74eb9b3335582ee6717c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.355614 30725 raft_consensus.cc:2804] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:41.367118 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c56097e8594d298daed39079975dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.367311 30725 raft_consensus.cc:493] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.367833 30725 raft_consensus.cc:3060] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.383167 30691 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:41.396098 30626 tablet_service.cc:1511] Processing CreateTablet for tablet 44a0e227c0ca4247b0ba8fbf2d005fec (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:09:41.398994 30725 raft_consensus.cc:515] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.400887 30690 tablet_bootstrap.cc:654] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.405269 30486 tablet_service.cc:1511] Processing CreateTablet for tablet 44a0e227c0ca4247b0ba8fbf2d005fec (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:09:41.409020 30486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44a0e227c0ca4247b0ba8fbf2d005fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.414417 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 44a0e227c0ca4247b0ba8fbf2d005fec (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:09:41.415773 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44a0e227c0ca4247b0ba8fbf2d005fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.424124 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "730e52dc15a842e8a606385755e5558e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:41.430413 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "730e52dc15a842e8a606385755e5558e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:41.431023 30422 raft_consensus.cc:3060] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:41.432009 30463 leader_election.cc:343] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.436776 30557 tablet_service.cc:1511] Processing CreateTablet for tablet efba0c8d26eb438ea2ff27b4bb66b20a (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:09:41.437986 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efba0c8d26eb438ea2ff27b4bb66b20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.421322 30725 leader_election.cc:290] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:41.450958 30422 raft_consensus.cc:2468] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:41.451926 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44a0e227c0ca4247b0ba8fbf2d005fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.452570 30462 leader_election.cc:304] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:41.454622 30690 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:41.455127 30690 ts_tablet_manager.cc:1403] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.119s	user 0.009s	sys 0.005s
I20260709 13:09:41.459224 30633 tablet_service.cc:1511] Processing CreateTablet for tablet efba0c8d26eb438ea2ff27b4bb66b20a (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:09:41.464977 30411 tablet_service.cc:1511] Processing CreateTablet for tablet efba0c8d26eb438ea2ff27b4bb66b20a (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:09:41.465858 30690 raft_consensus.cc:359] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.466475 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efba0c8d26eb438ea2ff27b4bb66b20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.466624 30690 raft_consensus.cc:385] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.467038 30690 raft_consensus.cc:740] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.466887 30633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efba0c8d26eb438ea2ff27b4bb66b20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.463128 30695 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:41.468096 30695 ts_tablet_manager.cc:1403] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.236s	user 0.008s	sys 0.007s
I20260709 13:09:41.467798 30690 consensus_queue.cc:260] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.469172 30725 raft_consensus.cc:2804] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:41.469738 30725 raft_consensus.cc:697] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:41.470541 30725 consensus_queue.cc:237] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.472913 30690 ts_tablet_manager.cc:1434] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:09:41.474010 30690 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:41.471060 30481 tablet_service.cc:1511] Processing CreateTablet for tablet a5ed6aaa36704114809147b7efabcaee (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:09:41.480752 30692 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:41.481329 30692 ts_tablet_manager.cc:1403] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.195s	user 0.016s	sys 0.036s
I20260709 13:09:41.481467 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ed6aaa36704114809147b7efabcaee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.464061 30691 tablet_bootstrap.cc:654] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.484643 30692 raft_consensus.cc:359] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.485731 30692 raft_consensus.cc:385] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.486120 30692 raft_consensus.cc:740] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.487107 30692 consensus_queue.cc:260] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.488893 30628 tablet_service.cc:1511] Processing CreateTablet for tablet a5ed6aaa36704114809147b7efabcaee (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:09:41.489964 30695 raft_consensus.cc:359] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.506904 30695 raft_consensus.cc:385] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.507445 30695 raft_consensus.cc:740] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.504726 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ed6aaa36704114809147b7efabcaee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.505203 30555 tablet_service.cc:1511] Processing CreateTablet for tablet a5ed6aaa36704114809147b7efabcaee (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:09:41.509348 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ed6aaa36704114809147b7efabcaee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.510349 30692 ts_tablet_manager.cc:1434] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.003s
I20260709 13:09:41.511471 30692 tablet_bootstrap.cc:492] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:41.515050 30409 tablet_service.cc:1511] Processing CreateTablet for tablet ea53b9ee66e343b18d301fe650b12554 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:09:41.516288 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea53b9ee66e343b18d301fe650b12554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.516521 30487 tablet_service.cc:1511] Processing CreateTablet for tablet ea53b9ee66e343b18d301fe650b12554 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:09:41.517781 30487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea53b9ee66e343b18d301fe650b12554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.521520 30557 tablet_service.cc:1511] Processing CreateTablet for tablet ea53b9ee66e343b18d301fe650b12554 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:09:41.523133 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea53b9ee66e343b18d301fe650b12554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.526023 30632 tablet_service.cc:1511] Processing CreateTablet for tablet 55b4252704f04cae82d10d8b1bb4c88d (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:09:41.526759 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 55b4252704f04cae82d10d8b1bb4c88d (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:09:41.527341 30632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b4252704f04cae82d10d8b1bb4c88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.528139 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b4252704f04cae82d10d8b1bb4c88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.523984 30695 consensus_queue.cc:260] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.540235 30692 tablet_bootstrap.cc:654] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.546388 30695 ts_tablet_manager.cc:1434] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.078s	user 0.002s	sys 0.007s
I20260709 13:09:41.551894 30554 tablet_service.cc:1511] Processing CreateTablet for tablet bccb92fd917f4a97af88be6e7e8fed1f (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:09:41.556399 30554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bccb92fd917f4a97af88be6e7e8fed1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.557659 30690 tablet_bootstrap.cc:654] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.559226 30695 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:41.552982 30631 tablet_service.cc:1511] Processing CreateTablet for tablet bccb92fd917f4a97af88be6e7e8fed1f (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:09:41.562475 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bccb92fd917f4a97af88be6e7e8fed1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.554720 30408 tablet_service.cc:1511] Processing CreateTablet for tablet bccb92fd917f4a97af88be6e7e8fed1f (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:09:41.564250 30408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bccb92fd917f4a97af88be6e7e8fed1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.547004 30486 tablet_service.cc:1511] Processing CreateTablet for tablet 55b4252704f04cae82d10d8b1bb4c88d (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:09:41.572958 30723 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.575608 30709 raft_consensus.cc:493] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.575858 30723 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.576146 30709 raft_consensus.cc:515] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.577150 30692 tablet_bootstrap.cc:492] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:41.577642 30692 ts_tablet_manager.cc:1403] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.066s	user 0.008s	sys 0.010s
I20260709 13:09:41.578132 30723 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:41.583007 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:41.590057 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:41.590796 30570 raft_consensus.cc:2468] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:41.591245 30319 catalog_manager.cc:5697] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:41.592809 30610 leader_election.cc:343] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.593936 30388 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:41.594659 30486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b4252704f04cae82d10d8b1bb4c88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.594841 30723 raft_consensus.cc:2804] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:41.595216 30723 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.595536 30723 raft_consensus.cc:3060] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.599186 30487 tablet_service.cc:1511] Processing CreateTablet for tablet eba54fb250484a4e8102345d2a02e5cb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:09:41.600581 30487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eba54fb250484a4e8102345d2a02e5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.582434 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:41.604646 30557 tablet_service.cc:1511] Processing CreateTablet for tablet eba54fb250484a4e8102345d2a02e5cb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:09:41.605960 30557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eba54fb250484a4e8102345d2a02e5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.580920 30709 leader_election.cc:290] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:41.610741 30386 leader_election.cc:343] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.629027 30745 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.629767 30406 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3b4d4c8fc44213807c39418459447d (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:09:41.631145 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3b4d4c8fc44213807c39418459447d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.632268 30692 raft_consensus.cc:359] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.634383 30481 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3b4d4c8fc44213807c39418459447d (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:09:41.635051 30690 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:41.645373 30690 ts_tablet_manager.cc:1403] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.172s	user 0.009s	sys 0.007s
I20260709 13:09:41.639204 30691 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:41.646068 30691 ts_tablet_manager.cc:1403] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.285s	user 0.020s	sys 0.008s
I20260709 13:09:41.643631 30555 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3b4d4c8fc44213807c39418459447d (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:09:41.647181 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3b4d4c8fc44213807c39418459447d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.637326 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3b4d4c8fc44213807c39418459447d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.636539 30692 raft_consensus.cc:385] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.648582 30692 raft_consensus.cc:740] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.648867 30690 raft_consensus.cc:359] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.649588 30692 consensus_queue.cc:260] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.658553 30725 consensus_queue.cc:1048] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:41.652145 30723 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.653134 30626 tablet_service.cc:1511] Processing CreateTablet for tablet b4a97b4e180f4b618f578bfcc4d841a1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:09:41.663846 30626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a97b4e180f4b618f578bfcc4d841a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.667389 30723 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:41.668457 30692 ts_tablet_manager.cc:1434] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 13:09:41.629833 30745 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.668699 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:41.654453 30628 tablet_service.cc:1511] Processing CreateTablet for tablet eba54fb250484a4e8102345d2a02e5cb (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:09:41.651520 30691 raft_consensus.cc:359] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.671553 30695 tablet_bootstrap.cc:654] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:41.673048 30386 leader_election.cc:343] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.674933 30628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eba54fb250484a4e8102345d2a02e5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.675489 30691 raft_consensus.cc:385] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.675872 30691 raft_consensus.cc:740] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.676812 30691 consensus_queue.cc:260] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.678098 30692 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:41.679155 30696 raft_consensus.cc:493] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.679775 30696 raft_consensus.cc:515] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.681165 30745 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:41.682447 30696 leader_election.cc:290] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:41.683773 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:41.685359 30537 leader_election.cc:343] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.660609 30690 raft_consensus.cc:385] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.686522 30690 raft_consensus.cc:740] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.689958 30555 tablet_service.cc:1511] Processing CreateTablet for tablet 2908e68999c34a76b7343444d882b4e9 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:09:41.691466 30555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2908e68999c34a76b7343444d882b4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.691473 30631 tablet_service.cc:1511] Processing CreateTablet for tablet 2908e68999c34a76b7343444d882b4e9 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:09:41.693630 30631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2908e68999c34a76b7343444d882b4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.694572 30690 consensus_queue.cc:260] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.695926 30481 tablet_service.cc:1511] Processing CreateTablet for tablet b4a97b4e180f4b618f578bfcc4d841a1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:09:41.697264 30481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a97b4e180f4b618f578bfcc4d841a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.702411 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:41.703111 30571 raft_consensus.cc:3060] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.707482 30690 ts_tablet_manager.cc:1434] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.001s
I20260709 13:09:41.718227 30690 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:41.720767 30571 raft_consensus.cc:2468] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:41.722647 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:41.723457 30497 raft_consensus.cc:2468] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:41.725446 30692 tablet_bootstrap.cc:654] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.725641 30388 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:41.723367 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:41.727677 30745 raft_consensus.cc:2804] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:41.728236 30745 raft_consensus.cc:697] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:41.729288 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:41.729144 30745 consensus_queue.cc:237] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.730373 30695 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
W20260709 13:09:41.730535 30535 leader_election.cc:343] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.730962 30695 ts_tablet_manager.cc:1403] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.172s	user 0.010s	sys 0.002s
I20260709 13:09:41.731050 30535 leader_election.cc:304] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:41.731496 30691 ts_tablet_manager.cc:1434] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.085s	user 0.001s	sys 0.005s
I20260709 13:09:41.732336 30696 raft_consensus.cc:2749] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.732623 30691 tablet_bootstrap.cc:492] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:41.737116 30695 raft_consensus.cc:359] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.737990 30695 raft_consensus.cc:385] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.737790 30406 tablet_service.cc:1511] Processing CreateTablet for tablet b4a97b4e180f4b618f578bfcc4d841a1 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:09:41.738407 30695 raft_consensus.cc:740] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.739234 30406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a97b4e180f4b618f578bfcc4d841a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.743009 30696 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.744028 30696 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.745360 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:41.746632 30696 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:41.747097 30611 leader_election.cc:343] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.747742 30611 leader_election.cc:304] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:41.748755 30709 raft_consensus.cc:2749] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.750339 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:41.751056 30422 raft_consensus.cc:2468] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:41.754271 30409 tablet_service.cc:1511] Processing CreateTablet for tablet 2908e68999c34a76b7343444d882b4e9 (DEFAULT_TABLE table=test-workload [id=f16282f347ce419da027923592ae9bb2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:09:41.754603 30536 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:41.755512 30409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2908e68999c34a76b7343444d882b4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:41.756667 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:41.757170 30696 raft_consensus.cc:2804] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:41.757567 30696 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.757903 30696 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.760643 30691 tablet_bootstrap.cc:654] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.739351 30695 consensus_queue.cc:260] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:41.762285 30535 leader_election.cc:343] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.774623 30695 ts_tablet_manager.cc:1434] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:09:41.774350 30320 catalog_manager.cc:5697] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:41.775764 30695 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:41.767319 30388 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
W20260709 13:09:41.786994 30388 leader_election.cc:343] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.788506 30745 raft_consensus.cc:2804] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:41.787971 30696 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.789196 30745 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.777416 30690 tablet_bootstrap.cc:654] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.805822 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
W20260709 13:09:41.807899 30535 leader_election.cc:343] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.797866 30745 raft_consensus.cc:3060] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:41.809083 30462 consensus_peers.cc:597] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:41.812704 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:41.814087 30422 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.818895 30695 tablet_bootstrap.cc:654] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:41.819753 30745 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.823108 30745 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:41.825893 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:41.826546 30497 raft_consensus.cc:3060] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.829277 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
W20260709 13:09:41.831171 30388 leader_election.cc:343] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.834998 30709 raft_consensus.cc:493] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.835549 30709 raft_consensus.cc:515] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.813776 30696 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:41.836272 30497 raft_consensus.cc:2468] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:41.838194 30709 leader_election.cc:290] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:41.839501 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:41.848726 30692 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:41.848829 30690 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:41.849848 30690 ts_tablet_manager.cc:1403] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.132s	user 0.027s	sys 0.025s
I20260709 13:09:41.839824 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:41.840801 30422 raft_consensus.cc:2468] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:41.885980 30388 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:41.892330 30536 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
W20260709 13:09:41.898465 30610 leader_election.cc:343] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.899394 30728 raft_consensus.cc:493] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:41.900249 30611 leader_election.cc:343] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.901288 30692 ts_tablet_manager.cc:1403] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.223s	user 0.013s	sys 0.005s
I20260709 13:09:41.900894 30728 raft_consensus.cc:515] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.904234 30691 tablet_bootstrap.cc:492] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:41.903978 30611 leader_election.cc:304] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:41.905592 30696 raft_consensus.cc:2804] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:41.905920 30709 raft_consensus.cc:2749] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.906749 30696 raft_consensus.cc:697] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:41.909111 30696 consensus_queue.cc:237] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.912319 30690 raft_consensus.cc:359] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.913317 30690 raft_consensus.cc:385] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.913628 30690 raft_consensus.cc:740] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.898476 30745 raft_consensus.cc:2804] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:41.911763 30692 raft_consensus.cc:359] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.919952 30692 raft_consensus.cc:385] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.918321 30756 consensus_queue.cc:1048] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:41.928157 30690 consensus_queue.cc:260] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.917356 30745 raft_consensus.cc:697] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:41.930725 30745 consensus_queue.cc:237] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:41.922360 30728 leader_election.cc:290] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:41.922791 30691 ts_tablet_manager.cc:1403] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.190s	user 0.011s	sys 0.011s
I20260709 13:09:41.921716 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:41.932615 30690 ts_tablet_manager.cc:1434] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.082s	user 0.004s	sys 0.006s
I20260709 13:09:41.953099 30690 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:41.955705 30691 raft_consensus.cc:359] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.956635 30691 raft_consensus.cc:385] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:41.957520 30709 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.958506 30422 raft_consensus.cc:2468] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:41.958335 30709 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:41.947759 30692 raft_consensus.cc:740] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.961485 30692 consensus_queue.cc:260] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.959008 30691 raft_consensus.cc:740] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:41.963147 30695 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:41.963361 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:41.964116 30695 ts_tablet_manager.cc:1403] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.189s	user 0.011s	sys 0.012s
I20260709 13:09:41.964757 30709 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:41.965504 30610 leader_election.cc:343] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.963306 30691 consensus_queue.cc:260] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:41.967374 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:41.967473 30462 leader_election.cc:304] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
W20260709 13:09:41.973909 30611 leader_election.cc:343] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:41.974538 30611 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:41.975991 30709 raft_consensus.cc:2749] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:41.975800 30318 catalog_manager.cc:5697] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:41.979488 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:41.980175 30756 raft_consensus.cc:2804] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:41.980820 30756 raft_consensus.cc:493] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.982239 30756 raft_consensus.cc:3060] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:41.987802 30723 raft_consensus.cc:493] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.988655 30723 raft_consensus.cc:515] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:41.992976 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:41.993793 30570 raft_consensus.cc:2468] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:41.995505 30692 ts_tablet_manager.cc:1434] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.002s
I20260709 13:09:41.995757 30388 leader_election.cc:304] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:41.998771 30723 leader_election.cc:290] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:41.999521 30723 raft_consensus.cc:2804] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:41.999538 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.000789 30644 raft_consensus.cc:2468] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:41.999984 30723 raft_consensus.cc:493] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.003129 30723 raft_consensus.cc:3060] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.004843 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.005359 30690 tablet_bootstrap.cc:654] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:42.006846 30461 leader_election.cc:343] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:42.007231 30463 consensus_peers.cc:597] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:41.980803 30691 ts_tablet_manager.cc:1434] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.007s
I20260709 13:09:42.012359 30695 raft_consensus.cc:359] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.022545 30692 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.026068 30691 tablet_bootstrap.cc:492] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:42.019783 30695 raft_consensus.cc:385] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.031287 30695 raft_consensus.cc:740] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.032366 30695 consensus_queue.cc:260] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.034318 30756 raft_consensus.cc:515] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.038940 30320 catalog_manager.cc:5697] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.047899 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:42.048614 30421 raft_consensus.cc:3060] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.049132 30723 raft_consensus.cc:515] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.051165 30691 tablet_bootstrap.cc:654] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.051760 30723 leader_election.cc:290] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.042846 30692 tablet_bootstrap.cc:654] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.053083 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.053730 30644 raft_consensus.cc:3060] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.062227 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:42.062954 30570 raft_consensus.cc:3060] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.063153 30691 tablet_bootstrap.cc:492] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.063699 30691 ts_tablet_manager.cc:1403] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.002s
I20260709 13:09:42.043417 30690 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:42.044744 30695 ts_tablet_manager.cc:1434] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.080s	user 0.006s	sys 0.006s
I20260709 13:09:42.064368 30690 ts_tablet_manager.cc:1403] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.008s
I20260709 13:09:42.065719 30695 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:42.067520 30691 raft_consensus.cc:359] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.067788 30690 raft_consensus.cc:359] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.068315 30691 raft_consensus.cc:385] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.068799 30691 raft_consensus.cc:740] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.070022 30691 consensus_queue.cc:260] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.068691 30690 raft_consensus.cc:385] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.075596 30690 raft_consensus.cc:740] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.080518 30691 ts_tablet_manager.cc:1434] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 13:09:42.076648 30690 consensus_queue.cc:260] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.082614 30756 leader_election.cc:290] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.083408 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.085456 30691 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
W20260709 13:09:42.086390 30461 leader_election.cc:343] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.086526 30690 ts_tablet_manager.cc:1434] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 13:09:42.088441 30690 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:42.089110 30421 raft_consensus.cc:2468] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:42.090503 30462 leader_election.cc:304] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:42.091769 30756 raft_consensus.cc:2804] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.093062 30691 tablet_bootstrap.cc:654] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.095395 30690 tablet_bootstrap.cc:654] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.080519 30644 raft_consensus.cc:2468] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:42.083925 30695 tablet_bootstrap.cc:654] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.092458 30756 raft_consensus.cc:697] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:42.097625 30386 leader_election.cc:304] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:42.098053 30756 consensus_queue.cc:237] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.098899 30723 raft_consensus.cc:2804] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.099390 30723 raft_consensus.cc:697] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:42.084703 30570 raft_consensus.cc:2468] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:42.100421 30723 consensus_queue.cc:237] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.110363 30695 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:42.110965 30695 ts_tablet_manager.cc:1403] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.002s
I20260709 13:09:42.112620 30692 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:42.112268 30320 catalog_manager.cc:5697] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:42.113499 30692 ts_tablet_manager.cc:1403] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.011s
I20260709 13:09:42.114283 30695 raft_consensus.cc:359] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.114925 30696 raft_consensus.cc:493] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.115410 30695 raft_consensus.cc:385] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.115862 30695 raft_consensus.cc:740] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.115746 30696 raft_consensus.cc:515] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.117213 30695 consensus_queue.cc:260] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.121268 30695 ts_tablet_manager.cc:1434] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.010s
I20260709 13:09:42.123243 30695 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:42.124539 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845d59ad223a4f6c9c388017a170527b" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.123345 30696 leader_election.cc:290] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
W20260709 13:09:42.126713 30537 leader_election.cc:343] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.131110 30692 raft_consensus.cc:359] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.131953 30692 raft_consensus.cc:385] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.132387 30692 raft_consensus.cc:740] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.128393 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845d59ad223a4f6c9c388017a170527b" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:42.133459 30386 consensus_peers.cc:597] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 -> Peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Couldn't send request to peer b897660686b3465390a5bad32efea13c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:42.135303 30535 leader_election.cc:343] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.135797 30535 leader_election.cc:304] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:42.136607 30696 raft_consensus.cc:2749] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.136880 30691 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.137459 30691 ts_tablet_manager.cc:1403] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.005s
I20260709 13:09:42.140671 30690 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:42.132085 30695 tablet_bootstrap.cc:654] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.141355 30690 ts_tablet_manager.cc:1403] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.053s	user 0.029s	sys 0.004s
I20260709 13:09:42.140679 30691 raft_consensus.cc:359] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.142719 30691 raft_consensus.cc:385] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.141369 30692 consensus_queue.cc:260] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.145201 30691 raft_consensus.cc:740] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.146278 30691 consensus_queue.cc:260] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.147665 30692 ts_tablet_manager.cc:1434] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:09:42.144536 30690 raft_consensus.cc:359] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.149063 30691 ts_tablet_manager.cc:1434] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:09:42.149230 30690 raft_consensus.cc:385] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.149580 30690 raft_consensus.cc:740] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.150239 30695 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:42.150331 30690 consensus_queue.cc:260] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.152048 30691 tablet_bootstrap.cc:492] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:42.152613 30690 ts_tablet_manager.cc:1434] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:09:42.153766 30690 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:42.149065 30692 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.160045 30691 tablet_bootstrap.cc:654] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.159338 30320 catalog_manager.cc:5697] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:42.161942 30692 tablet_bootstrap.cc:654] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.150812 30695 ts_tablet_manager.cc:1403] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.028s	user 0.005s	sys 0.008s
I20260709 13:09:42.169646 30723 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.170152 30723 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.169791 30695 raft_consensus.cc:359] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.171062 30695 raft_consensus.cc:385] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.171509 30695 raft_consensus.cc:740] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.173173 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.173354 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.174001 30644 raft_consensus.cc:2468] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
W20260709 13:09:42.174962 30388 leader_election.cc:343] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.175621 30386 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44
I20260709 13:09:42.176504 30765 raft_consensus.cc:2804] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.176867 30765 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.177237 30765 raft_consensus.cc:3060] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.180353 30690 tablet_bootstrap.cc:654] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.181042 30723 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.172659 30695 consensus_queue.cc:260] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.184731 30691 tablet_bootstrap.cc:492] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.185259 30695 ts_tablet_manager.cc:1434] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.002s
I20260709 13:09:42.185261 30691 ts_tablet_manager.cc:1403] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.014s
I20260709 13:09:42.186622 30756 raft_consensus.cc:493] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.186960 30695 tablet_bootstrap.cc:492] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:42.187139 30756 raft_consensus.cc:515] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.188859 30691 raft_consensus.cc:359] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.189543 30691 raft_consensus.cc:385] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.189803 30691 raft_consensus.cc:740] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.190279 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309754d072834ef381a0c8a6121b05cb" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.190475 30691 consensus_queue.cc:260] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.190959 30570 raft_consensus.cc:2468] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.192517 30691 ts_tablet_manager.cc:1434] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:09:42.192559 30463 leader_election.cc:304] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:42.193742 30757 raft_consensus.cc:2804] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.193867 30691 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:42.194139 30757 raft_consensus.cc:493] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.194501 30757 raft_consensus.cc:3060] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.198567 30695 tablet_bootstrap.cc:654] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.201252 30691 tablet_bootstrap.cc:654] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.202271 30756 leader_election.cc:290] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.203498 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309754d072834ef381a0c8a6121b05cb" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.204945 30690 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
W20260709 13:09:42.204970 30461 leader_election.cc:343] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.205802 30690 ts_tablet_manager.cc:1403] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.009s
I20260709 13:09:42.211843 30690 raft_consensus.cc:359] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.212785 30690 raft_consensus.cc:385] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.213152 30690 raft_consensus.cc:740] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.214284 30765 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.217548 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.218267 30644 raft_consensus.cc:3060] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.214118 30690 consensus_queue.cc:260] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.222185 30765 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.226286 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:42.236038 30757 raft_consensus.cc:515] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.241195 30644 raft_consensus.cc:2468] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
W20260709 13:09:42.241914 30388 leader_election.cc:343] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.245630 30386 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44
I20260709 13:09:42.248714 30765 raft_consensus.cc:2804] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.249250 30765 raft_consensus.cc:697] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:42.249691 30690 ts_tablet_manager.cc:1434] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.005s
I20260709 13:09:42.250245 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309754d072834ef381a0c8a6121b05cb" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:42.250085 30765 consensus_queue.cc:237] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.250828 30570 raft_consensus.cc:3060] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.251232 30690 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:42.255182 30692 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:42.255672 30692 ts_tablet_manager.cc:1403] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.107s	user 0.031s	sys 0.032s
I20260709 13:09:42.258291 30692 raft_consensus.cc:359] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.258821 30692 raft_consensus.cc:385] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.259047 30692 raft_consensus.cc:740] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.259267 30690 tablet_bootstrap.cc:654] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.259675 30570 raft_consensus.cc:2468] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:42.259694 30692 consensus_queue.cc:260] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.261296 30757 leader_election.cc:290] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.261310 30463 leader_election.cc:304] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:42.261970 30692 ts_tablet_manager.cc:1434] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:09:42.262563 30757 raft_consensus.cc:2804] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.263327 30692 tablet_bootstrap.cc:492] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.263393 30757 raft_consensus.cc:697] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:42.263631 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309754d072834ef381a0c8a6121b05cb" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.264087 30691 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.264556 30691 ts_tablet_manager.cc:1403] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.015s
I20260709 13:09:42.264631 30757 consensus_queue.cc:237] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
W20260709 13:09:42.265483 30461 leader_election.cc:343] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.267590 30691 raft_consensus.cc:359] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.268291 30691 raft_consensus.cc:385] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.268563 30691 raft_consensus.cc:740] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.269364 30691 consensus_queue.cc:260] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.270810 30692 tablet_bootstrap.cc:654] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.271754 30691 ts_tablet_manager.cc:1434] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:42.272816 30691 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:42.278888 30691 tablet_bootstrap.cc:654] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.292327 30695 tablet_bootstrap.cc:492] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:42.293041 30695 ts_tablet_manager.cc:1403] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.106s	user 0.020s	sys 0.028s
I20260709 13:09:42.293867 30318 catalog_manager.cc:5697] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.300969 30421 raft_consensus.cc:3060] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.309933 30695 raft_consensus.cc:359] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.311144 30695 raft_consensus.cc:385] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.311589 30695 raft_consensus.cc:740] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.313390 30695 consensus_queue.cc:260] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.320410 30723 consensus_queue.cc:1048] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.329694 30692 tablet_bootstrap.cc:492] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:42.330068 30690 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:42.330381 30692 ts_tablet_manager.cc:1403] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.067s	user 0.033s	sys 0.031s
I20260709 13:09:42.330556 30757 raft_consensus.cc:493] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.330783 30690 ts_tablet_manager.cc:1403] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.027s
I20260709 13:09:42.333348 30692 raft_consensus.cc:359] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.334111 30692 raft_consensus.cc:385] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.334561 30692 raft_consensus.cc:740] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.335135 30757 raft_consensus.cc:515] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.335480 30691 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.335516 30692 consensus_queue.cc:260] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.336233 30691 ts_tablet_manager.cc:1403] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.000s
I20260709 13:09:42.339185 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e003633f9bf24b09b3789ddc3ac48cc1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.339327 30691 raft_consensus.cc:359] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.340040 30691 raft_consensus.cc:385] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.340410 30691 raft_consensus.cc:740] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.340998 30571 raft_consensus.cc:2468] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.341501 30692 ts_tablet_manager.cc:1434] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:09:42.341347 30691 consensus_queue.cc:260] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.342839 30692 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.343657 30691 ts_tablet_manager.cc:1434] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:09:42.344867 30463 leader_election.cc:304] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:42.345815 30756 raft_consensus.cc:2804] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.346218 30756 raft_consensus.cc:493] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.346606 30756 raft_consensus.cc:3060] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.349516 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e003633f9bf24b09b3789ddc3ac48cc1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.349934 30692 tablet_bootstrap.cc:654] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:42.351321 30461 leader_election.cc:343] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.350893 30319 catalog_manager.cc:5697] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.353185 30757 leader_election.cc:290] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.354019 30709 raft_consensus.cc:493] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.354485 30709 raft_consensus.cc:515] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.359311 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.360026 30497 raft_consensus.cc:2468] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:42.360203 30756 raft_consensus.cc:515] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.361411 30611 leader_election.cc:304] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:42.363055 30709 leader_election.cc:290] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:42.363584 30709 raft_consensus.cc:2804] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.364001 30709 raft_consensus.cc:493] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.364333 30709 raft_consensus.cc:3060] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.364849 30691 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:42.365017 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e003633f9bf24b09b3789ddc3ac48cc1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:42.366070 30570 raft_consensus.cc:3060] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.375583 30691 tablet_bootstrap.cc:654] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.376439 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.377472 30571 raft_consensus.cc:2468] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:42.378511 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e003633f9bf24b09b3789ddc3ac48cc1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.379175 30756 leader_election.cc:290] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
W20260709 13:09:42.380299 30461 leader_election.cc:343] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.381233 30692 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:42.381172 30690 raft_consensus.cc:359] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.381815 30692 ts_tablet_manager.cc:1403] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.016s
I20260709 13:09:42.382004 30690 raft_consensus.cc:385] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.382169 30570 raft_consensus.cc:2468] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:42.382477 30690 raft_consensus.cc:740] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.382799 30723 raft_consensus.cc:493] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.383281 30690 consensus_queue.cc:260] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.383785 30463 leader_election.cc:304] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:42.385056 30692 raft_consensus.cc:359] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.385584 30690 ts_tablet_manager.cc:1434] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:09:42.383350 30723 raft_consensus.cc:515] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.385831 30692 raft_consensus.cc:385] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.386165 30692 raft_consensus.cc:740] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.386814 30690 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:42.386749 30756 raft_consensus.cc:2804] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.387558 30756 raft_consensus.cc:697] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:42.388466 30756 consensus_queue.cc:237] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.390394 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.391130 30644 raft_consensus.cc:2468] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:42.386922 30692 consensus_queue.cc:260] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.392433 30386 leader_election.cc:304] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:42.393338 30765 raft_consensus.cc:2804] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.393747 30692 ts_tablet_manager.cc:1434] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:09:42.393762 30765 raft_consensus.cc:493] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.394505 30765 raft_consensus.cc:3060] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.396080 30723 leader_election.cc:290] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.391281 30691 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.396888 30691 ts_tablet_manager.cc:1403] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.001s
I20260709 13:09:42.397209 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.397958 30692 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.397939 30497 raft_consensus.cc:2468] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:42.400074 30691 raft_consensus.cc:359] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.400928 30691 raft_consensus.cc:385] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.401243 30691 raft_consensus.cc:740] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.406453 30690 tablet_bootstrap.cc:654] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.409372 30692 tablet_bootstrap.cc:654] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.412155 30691 consensus_queue.cc:260] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.409282 30787 raft_consensus.cc:493] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.415776 30787 raft_consensus.cc:515] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.421248 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.421098 30765 raft_consensus.cc:515] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.424464 30787 leader_election.cc:290] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.425549 30787 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.426364 30787 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.429304 30787 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.434136 30644 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.437142 30765 leader_election.cc:290] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.438086 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.438787 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:42.439618 30497 raft_consensus.cc:3060] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.444144 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.446025 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.446864 30421 raft_consensus.cc:2393] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 19b1445cece541d3b2abe177283fef44 in this term.
I20260709 13:09:42.446533 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.448253 30422 raft_consensus.cc:2393] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:42.448638 30645 raft_consensus.cc:3060] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.450510 30692 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:42.451001 30692 ts_tablet_manager.cc:1403] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.011s
I20260709 13:09:42.453926 30692 raft_consensus.cc:359] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.454603 30692 raft_consensus.cc:385] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.454869 30692 raft_consensus.cc:740] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.455647 30610 leader_election.cc:304] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:42.455698 30692 consensus_queue.cc:260] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.457254 30793 raft_consensus.cc:3060] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.459769 30692 ts_tablet_manager.cc:1434] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 13:09:42.459808 30571 raft_consensus.cc:3060] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.461786 30610 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:42.464655 30692 tablet_bootstrap.cc:492] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.471045 30709 raft_consensus.cc:515] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.471868 30793 raft_consensus.cc:2749] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.471717 30696 consensus_queue.cc:1048] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.475772 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:42.476502 30496 raft_consensus.cc:3060] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.481541 30692 tablet_bootstrap.cc:654] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.489841 30691 ts_tablet_manager.cc:1434] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 13:09:42.493829 30695 ts_tablet_manager.cc:1434] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.200s	user 0.012s	sys 0.001s
I20260709 13:09:42.495263 30695 tablet_bootstrap.cc:492] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:42.495119 30756 consensus_queue.cc:1048] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.495622 30497 raft_consensus.cc:2468] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:42.498258 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:42.499130 30709 leader_election.cc:290] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:42.501144 30690 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:42.508991 30690 ts_tablet_manager.cc:1403] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.006s
I20260709 13:09:42.508935 30793 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.509644 30793 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.510879 30765 consensus_queue.cc:1048] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:42.511801 30793 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.513159 30496 raft_consensus.cc:2468] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:42.514007 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.505098 30691 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:42.507284 30569 raft_consensus.cc:3060] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.516229 30765 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.516743 30765 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.520253 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.520459 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.521385 30566 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
W20260709 13:09:42.523232 30388 consensus_peers.cc:597] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:42.522718 30318 catalog_manager.cc:5697] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.524068 30388 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:42.525656 30789 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.526093 30789 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.526424 30789 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.526669 30765 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.507848 30695 tablet_bootstrap.cc:654] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.514325 30611 leader_election.cc:304] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:42.529312 30690 raft_consensus.cc:359] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.539955 30690 raft_consensus.cc:385] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.540568 30690 raft_consensus.cc:740] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.532174 30787 raft_consensus.cc:2749] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.543053 30388 leader_election.cc:304] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:42.524602 30569 raft_consensus.cc:2468] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:42.551473 30611 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:42.552945 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.553785 30696 raft_consensus.cc:493] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.554265 30696 raft_consensus.cc:515] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:42.556447 30696 leader_election.cc:290] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.545600 30709 raft_consensus.cc:2804] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.548475 30723 raft_consensus.cc:2804] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.558529 30723 raft_consensus.cc:697] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:42.559002 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.559387 30723 consensus_queue.cc:237] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.561342 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:42.561702 30386 leader_election.cc:343] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:42.563759 30537 leader_election.cc:343] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:42.565048 30536 leader_election.cc:343] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.565462 30536 leader_election.cc:304] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:42.591584 30690 consensus_queue.cc:260] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.585443 30793 raft_consensus.cc:2749] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.588219 30709 raft_consensus.cc:697] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:42.592811 30696 raft_consensus.cc:2749] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.606294 30745 consensus_queue.cc:1048] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.608047 30756 consensus_queue.cc:1048] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:42.641409 30692 tablet_bootstrap.cc:492] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:42.613853 30789 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.643399 30692 ts_tablet_manager.cc:1403] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.179s	user 0.044s	sys 0.040s
I20260709 13:09:42.649360 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:42.650279 30691 tablet_bootstrap.cc:654] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.650015 30692 raft_consensus.cc:359] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.650947 30692 raft_consensus.cc:385] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.650768 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:42.650089 30569 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.655205 30692 raft_consensus.cc:740] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.656862 30690 ts_tablet_manager.cc:1434] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.147s	user 0.009s	sys 0.000s
I20260709 13:09:42.648715 30789 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
W20260709 13:09:42.600226 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 48860.1us
I20260709 13:09:42.666215 30690 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:42.667244 30386 leader_election.cc:343] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.672742 30690 tablet_bootstrap.cc:654] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:42.662375 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 97375.4us
I20260709 13:09:42.681206 30745 consensus_queue.cc:1048] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.681782 30696 consensus_queue.cc:1048] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.663587 30709 consensus_queue.cc:237] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.691797 30642 raft_consensus.cc:3060] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.650696 30645 raft_consensus.cc:2468] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:42.697667 30569 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:42.700235 30692 consensus_queue.cc:260] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.703075 30692 ts_tablet_manager.cc:1434] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.001s
I20260709 13:09:42.718034 30388 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:42.719530 30789 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:42.706641 30756 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.730917 30789 raft_consensus.cc:697] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:42.762764 30814 raft_consensus.cc:493] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.763571 30814 raft_consensus.cc:515] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.766592 30814 leader_election.cc:290] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.748914 30757 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.768631 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.769270 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.744503 30756 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.776113 30691 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:42.776854 30691 ts_tablet_manager.cc:1403] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.274s	user 0.032s	sys 0.015s
I20260709 13:09:42.780757 30692 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:42.793716 30690 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:42.794267 30690 ts_tablet_manager.cc:1403] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.128s	user 0.015s	sys 0.008s
I20260709 13:09:42.773840 30757 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.799371 30757 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.800768 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.801724 30641 raft_consensus.cc:2393] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19b1445cece541d3b2abe177283fef44 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:42.802227 30756 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.791133 30422 raft_consensus.cc:2468] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.803804 30802 raft_consensus.cc:493] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.804322 30802 raft_consensus.cc:515] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.805364 30570 raft_consensus.cc:2468] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.802649 30320 catalog_manager.cc:5697] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:42.804759 30695 tablet_bootstrap.cc:492] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:42.808585 30695 ts_tablet_manager.cc:1403] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.314s	user 0.023s	sys 0.007s
I20260709 13:09:42.808463 30690 raft_consensus.cc:359] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.809644 30690 raft_consensus.cc:385] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.809991 30690 raft_consensus.cc:740] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.832774 30690 consensus_queue.cc:260] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.816705 30765 consensus_queue.cc:1048] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:09:42.841706 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.842669 30462 leader_election.cc:304] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:42.814968 30789 consensus_queue.cc:237] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.840039 30691 raft_consensus.cc:359] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.846710 30691 raft_consensus.cc:385] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.847214 30814 raft_consensus.cc:2804] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.848418 30691 raft_consensus.cc:740] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.848465 30814 raft_consensus.cc:493] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.852710 30690 ts_tablet_manager.cc:1434] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:09:42.852275 30691 consensus_queue.cc:260] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.842695 30644 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.851851 30814 raft_consensus.cc:3060] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.866961 30695 raft_consensus.cc:359] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.867668 30695 raft_consensus.cc:385] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:42.868057 30695 raft_consensus.cc:740] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:42.851694 30745 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.869355 30745 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:42.872005 30805 log.cc:927] Time spent T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Append to log took a long time: real 0.094s	user 0.002s	sys 0.001s
I20260709 13:09:42.872746 30692 tablet_bootstrap.cc:654] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.861953 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.862403 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.855034 30756 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.864063 30496 raft_consensus.cc:3060] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.864679 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.873106 30745 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.873576 30690 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:42.874356 30723 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.889124 30420 raft_consensus.cc:2468] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:42.889528 30723 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:42.890951 30536 leader_election.cc:304] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:42.891855 30723 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:42.874712 30495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.892622 30495 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:42.892680 30816 raft_consensus.cc:2804] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.893306 30816 raft_consensus.cc:493] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.893702 30816 raft_consensus.cc:3060] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.894228 30388 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:42.877233 30639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.897990 30639 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:42.898444 30765 consensus_queue.cc:1048] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:42.899397 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:42.901636 30638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.861511 30802 leader_election.cc:290] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:42.852831 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 56893.4us
I20260709 13:09:42.903431 30462 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:42.904392 30691 ts_tablet_manager.cc:1434] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.127s	user 0.008s	sys 0.000s
W20260709 13:09:42.906497 30386 leader_election.cc:343] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.888917 30419 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.887871 30756 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.910660 30690 tablet_bootstrap.cc:654] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:42.913414 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:42.914122 30756 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.921257 30756 raft_consensus.cc:2749] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.925810 30723 raft_consensus.cc:2804] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.940068 30825 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.940896 30825 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:42.926093 30388 leader_election.cc:343] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:42.947412 30461 consensus_peers.cc:597] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 -> Peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Couldn't send request to peer b897660686b3465390a5bad32efea13c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:42.954368 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:42.955320 30419 raft_consensus.cc:2468] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:42.949234 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.962769 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:42.963941 30814 raft_consensus.cc:515] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.965265 30496 raft_consensus.cc:2393] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate 19b1445cece541d3b2abe177283fef44 in this term.
W20260709 13:09:42.965503 30386 leader_election.cc:343] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:42.966071 30825 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:42.973526 30829 consensus_queue.cc:1048] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:42.944365 30462 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:42.977991 30695 consensus_queue.cc:260] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:42.951061 30388 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:42.981572 30723 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:42.981913 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 59690us
I20260709 13:09:42.982180 30723 raft_consensus.cc:3060] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:42.982445 30695 ts_tablet_manager.cc:1434] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.173s	user 0.010s	sys 0.001s
I20260709 13:09:42.983697 30695 tablet_bootstrap.cc:492] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:42.985025 30814 leader_election.cc:290] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:42.985152 30825 raft_consensus.cc:2749] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:42.988966 30816 raft_consensus.cc:515] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:42.989377 30814 raft_consensus.cc:2804] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:42.991158 30814 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.052798 30690 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:43.053576 30690 ts_tablet_manager.cc:1403] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.180s	user 0.019s	sys 0.016s
I20260709 13:09:43.054931 30814 raft_consensus.cc:3060] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.056313 30691 tablet_bootstrap.cc:492] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:43.064124 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:43.064558 30816 leader_election.cc:290] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.064990 30570 raft_consensus.cc:2393] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 19b1445cece541d3b2abe177283fef44 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:43.065285 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:43.066100 30419 raft_consensus.cc:3060] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:43.061216 30462 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 122279us
I20260709 13:09:43.066442 30495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
W20260709 13:09:43.069267 30461 consensus_peers.cc:597] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 -> Peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Couldn't send request to peer b897660686b3465390a5bad32efea13c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:43.072108 30830 consensus_queue.cc:1048] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:43.073837 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:43.074517 30494 raft_consensus.cc:2468] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:43.076210 30802 raft_consensus.cc:493] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.076522 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391e1197cab41039a7daf01b4cfb72b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:43.077096 30692 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:43.076718 30802 raft_consensus.cc:515] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.077581 30692 ts_tablet_manager.cc:1403] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.327s	user 0.021s	sys 0.023s
I20260709 13:09:43.078655 30462 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:43.079986 30831 raft_consensus.cc:2804] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:43.080053 30462 leader_election.cc:304] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:43.080508 30831 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.080799 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:43.081159 30831 raft_consensus.cc:3060] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.081696 30817 raft_consensus.cc:2749] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:09:43.082412 30535 leader_election.cc:343] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.078918 30802 leader_election.cc:290] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
W20260709 13:09:43.083319 30461 leader_election.cc:343] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.083662 30695 tablet_bootstrap.cc:654] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.084786 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:43.086458 30537 leader_election.cc:343] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.087018 30537 leader_election.cc:304] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:43.080461 30692 raft_consensus.cc:359] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.093444 30692 raft_consensus.cc:385] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.088899 30816 raft_consensus.cc:2749] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:43.095000 30692 raft_consensus.cc:740] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.096557 30692 consensus_queue.cc:260] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.101353 30692 ts_tablet_manager.cc:1434] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:09:43.092737 30419 raft_consensus.cc:2468] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:43.106217 30536 leader_election.cc:304] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44
I20260709 13:09:43.106439 30723 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.108104 30692 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:43.111130 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:43.113358 30642 raft_consensus.cc:3060] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.117892 30816 raft_consensus.cc:2804] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:43.129401 30692 tablet_bootstrap.cc:654] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.130036 30816 raft_consensus.cc:697] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:43.135900 30816 consensus_queue.cc:237] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.139369 30642 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:43.140730 30386 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:43.141902 30825 raft_consensus.cc:2804] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:43.142410 30825 raft_consensus.cc:697] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:43.143289 30825 consensus_queue.cc:237] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.127568 30690 raft_consensus.cc:359] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.144523 30690 raft_consensus.cc:385] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.144973 30690 raft_consensus.cc:740] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.136919 30691 tablet_bootstrap.cc:654] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.143239 30814 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.146201 30690 consensus_queue.cc:260] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.131474 30723 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:43.152664 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:43.153931 30690 ts_tablet_manager.cc:1434] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.100s	user 0.006s	sys 0.003s
I20260709 13:09:43.155326 30690 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:43.159420 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:43.160612 30814 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:43.140355 30831 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.163102 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:43.165566 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
W20260709 13:09:43.167402 30461 leader_election.cc:343] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:43.183183 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 111383us
I20260709 13:09:43.185482 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:43.186583 30497 raft_consensus.cc:2393] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 19b1445cece541d3b2abe177283fef44 in this term.
I20260709 13:09:43.190587 30831 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:43.191675 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:43.192356 30419 raft_consensus.cc:3060] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.194108 30723 consensus_queue.cc:1048] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:43.199307 30695 tablet_bootstrap.cc:492] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:43.199950 30695 ts_tablet_manager.cc:1403] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.217s	user 0.007s	sys 0.006s
I20260709 13:09:43.202356 30690 tablet_bootstrap.cc:654] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.176452 30642 raft_consensus.cc:2393] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 19b1445cece541d3b2abe177283fef44 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:43.214923 30419 raft_consensus.cc:2468] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:43.216585 30461 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:43.218294 30462 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:43.219951 30819 raft_consensus.cc:2749] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:43.220114 30831 raft_consensus.cc:2804] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:43.220871 30831 raft_consensus.cc:697] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:43.221688 30388 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:43.221972 30831 consensus_queue.cc:237] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.222785 30723 raft_consensus.cc:2764] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:09:43.231868 30695 raft_consensus.cc:359] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.232450 30695 raft_consensus.cc:385] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.232693 30695 raft_consensus.cc:740] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.236508 30691 tablet_bootstrap.cc:492] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:43.237154 30691 ts_tablet_manager.cc:1403] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.181s	user 0.019s	sys 0.012s
I20260709 13:09:43.239056 30825 consensus_queue.cc:1048] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:43.240288 30691 raft_consensus.cc:359] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.241137 30691 raft_consensus.cc:385] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.241441 30691 raft_consensus.cc:740] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.242139 30691 consensus_queue.cc:260] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.244014 30691 ts_tablet_manager.cc:1434] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:09:43.245672 30695 consensus_queue.cc:260] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.247081 30692 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:43.247689 30692 ts_tablet_manager.cc:1403] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.140s	user 0.023s	sys 0.002s
I20260709 13:09:43.249348 30691 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:43.245056 30709 consensus_queue.cc:1048] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:09:43.249646 30816 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.250232 30816 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.249755 30320 catalog_manager.cc:5697] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:43.251575 30320 catalog_manager.cc:5697] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:43.252405 30816 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.252071 30692 raft_consensus.cc:359] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.253515 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:43.253918 30692 raft_consensus.cc:385] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.254262 30692 raft_consensus.cc:740] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.256454 30692 consensus_queue.cc:260] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.258281 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:43.263091 30692 ts_tablet_manager.cc:1434] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:43.263273 30695 ts_tablet_manager.cc:1434] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.000s
I20260709 13:09:43.264369 30695 tablet_bootstrap.cc:492] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:43.264514 30692 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:43.273984 30536 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:43.276628 30691 tablet_bootstrap.cc:654] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.280318 30816 raft_consensus.cc:2749] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:43.283476 30642 raft_consensus.cc:3060] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.284976 30793 consensus_queue.cc:1048] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:09:43.299628 30690 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:43.300187 30690 ts_tablet_manager.cc:1403] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.145s	user 0.011s	sys 0.003s
I20260709 13:09:43.302997 30690 raft_consensus.cc:359] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.303714 30690 raft_consensus.cc:385] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.304003 30690 raft_consensus.cc:740] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.304693 30690 consensus_queue.cc:260] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.306519 30690 ts_tablet_manager.cc:1434] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:09:43.307458 30690 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:43.313354 30692 tablet_bootstrap.cc:654] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.315523 30318 catalog_manager.cc:5697] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:43.325784 30691 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:43.326467 30691 ts_tablet_manager.cc:1403] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.004s
I20260709 13:09:43.330641 30695 tablet_bootstrap.cc:654] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.335952 30691 raft_consensus.cc:359] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.336844 30691 raft_consensus.cc:385] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.337215 30691 raft_consensus.cc:740] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.338070 30691 consensus_queue.cc:260] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.339329 30690 tablet_bootstrap.cc:654] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.340581 30691 ts_tablet_manager.cc:1434] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:09:43.341872 30691 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:43.348053 30692 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
W20260709 13:09:43.347838 30386 consensus_peers.cc:597] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 -> Peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Couldn't send request to peer b897660686b3465390a5bad32efea13c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:43.348654 30692 ts_tablet_manager.cc:1403] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.084s	user 0.008s	sys 0.004s
I20260709 13:09:43.350172 30691 tablet_bootstrap.cc:654] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.351554 30692 raft_consensus.cc:359] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.352299 30692 raft_consensus.cc:385] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.352559 30692 raft_consensus.cc:740] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.359637 30317 catalog_manager.cc:5697] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:43.365319 30816 raft_consensus.cc:493] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.366168 30570 raft_consensus.cc:3060] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.365952 30816 raft_consensus.cc:515] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.368381 30816 leader_election.cc:290] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.369719 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95d00bccf5ce41f4b601bba7d235ffbb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:43.370350 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95d00bccf5ce41f4b601bba7d235ffbb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:43.370622 30496 raft_consensus.cc:2468] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:43.372084 30537 leader_election.cc:304] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
W20260709 13:09:43.372984 30536 leader_election.cc:343] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.374732 30816 raft_consensus.cc:2804] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:43.375234 30816 raft_consensus.cc:493] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.375635 30816 raft_consensus.cc:3060] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.378057 30692 consensus_queue.cc:260] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.391158 30692 ts_tablet_manager.cc:1434] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.001s
I20260709 13:09:43.391722 30695 tablet_bootstrap.cc:492] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:43.392205 30695 ts_tablet_manager.cc:1403] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.128s	user 0.012s	sys 0.001s
I20260709 13:09:43.399255 30695 raft_consensus.cc:359] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.400031 30695 raft_consensus.cc:385] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.400321 30695 raft_consensus.cc:740] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.401983 30691 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:43.402498 30691 ts_tablet_manager.cc:1403] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.004s
I20260709 13:09:43.405290 30691 raft_consensus.cc:359] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.405977 30691 raft_consensus.cc:385] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.406253 30691 raft_consensus.cc:740] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.406939 30691 consensus_queue.cc:260] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.408226 30692 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:43.410020 30319 catalog_manager.cc:5697] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:43.412856 30816 raft_consensus.cc:515] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.413218 30695 consensus_queue.cc:260] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.415148 30692 tablet_bootstrap.cc:654] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.415999 30816 leader_election.cc:290] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.416710 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95d00bccf5ce41f4b601bba7d235ffbb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:43.417379 30497 raft_consensus.cc:3060] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.417749 30691 ts_tablet_manager.cc:1434] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:09:43.418888 30691 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:43.419076 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95d00bccf5ce41f4b601bba7d235ffbb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
W20260709 13:09:43.420785 30536 leader_election.cc:343] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.443801 30816 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.444541 30816 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.451380 30695 ts_tablet_manager.cc:1434] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:09:43.451488 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:43.452497 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:43.451406 30816 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:43.452400 30690 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:43.452759 30695 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:43.453433 30690 ts_tablet_manager.cc:1403] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.146s	user 0.011s	sys 0.026s
W20260709 13:09:43.453856 30535 leader_election.cc:343] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.458307 30691 tablet_bootstrap.cc:654] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.453464 30419 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:43.459868 30695 tablet_bootstrap.cc:654] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.461061 30536 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:43.461956 30816 raft_consensus.cc:2804] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:43.462352 30816 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.462728 30816 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.465018 30825 consensus_queue.cc:1048] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:43.465505 30497 raft_consensus.cc:2468] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:43.466920 30537 leader_election.cc:304] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:43.468120 30696 raft_consensus.cc:2804] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:43.469579 30692 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:43.470193 30692 ts_tablet_manager.cc:1403] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.006s
I20260709 13:09:43.470343 30816 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.473542 30692 raft_consensus.cc:359] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.474179 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
W20260709 13:09:43.476289 30535 leader_election.cc:343] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.460934 30690 raft_consensus.cc:359] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.477320 30690 raft_consensus.cc:385] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.477731 30690 raft_consensus.cc:740] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.478572 30690 consensus_queue.cc:260] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.483068 30816 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:43.484061 30690 ts_tablet_manager.cc:1434] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 13:09:43.484202 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:43.484781 30419 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.485229 30690 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:43.474495 30692 raft_consensus.cc:385] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.491042 30692 raft_consensus.cc:740] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.495414 30419 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:43.496742 30536 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:43.491032 30825 raft_consensus.cc:493] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.497998 30816 raft_consensus.cc:2804] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:43.511297 30692 consensus_queue.cc:260] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.513541 30642 raft_consensus.cc:3060] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.497852 30825 raft_consensus.cc:515] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.519719 30825 leader_election.cc:290] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.520449 30816 raft_consensus.cc:697] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:43.521597 30816 consensus_queue.cc:237] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.523447 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24a265170a904a89a2801145c1a61172" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:43.525218 30690 tablet_bootstrap.cc:654] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.526441 30696 raft_consensus.cc:697] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:43.524762 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24a265170a904a89a2801145c1a61172" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:43.533630 30388 leader_election.cc:343] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.535372 30696 consensus_queue.cc:237] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.531651 30695 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
W20260709 13:09:43.537257 30386 leader_election.cc:343] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.537988 30386 leader_election.cc:304] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:43.539799 30723 raft_consensus.cc:2749] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:43.541306 30695 ts_tablet_manager.cc:1403] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.010s
I20260709 13:09:43.551097 30692 ts_tablet_manager.cc:1434] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.002s
I20260709 13:09:43.554621 30690 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:43.555227 30690 ts_tablet_manager.cc:1403] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.007s
I20260709 13:09:43.555785 30826 raft_consensus.cc:493] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.556545 30826 raft_consensus.cc:515] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.557803 30690 raft_consensus.cc:359] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.558346 30690 raft_consensus.cc:385] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.558563 30690 raft_consensus.cc:740] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.559777 30826 leader_election.cc:290] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:43.559194 30690 consensus_queue.cc:260] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.560971 30857 consensus_queue.cc:1048] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:09:43.561452 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845d59ad223a4f6c9c388017a170527b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:43.562099 30723 raft_consensus.cc:493] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.562273 30644 raft_consensus.cc:2468] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:43.562649 30723 raft_consensus.cc:515] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.564066 30461 leader_election.cc:304] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:43.565073 30826 raft_consensus.cc:2804] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:43.565512 30826 raft_consensus.cc:493] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.565910 30826 raft_consensus.cc:3060] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.552204 30695 raft_consensus.cc:359] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.567492 30695 raft_consensus.cc:385] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.567875 30690 ts_tablet_manager.cc:1434] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:09:43.567888 30695 raft_consensus.cc:740] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.569590 30765 raft_consensus.cc:493] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.569969 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:43.567415 30723 leader_election.cc:290] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:43.575286 30690 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:43.573875 30695 consensus_queue.cc:260] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.573347 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845d59ad223a4f6c9c388017a170527b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:43.585799 30695 ts_tablet_manager.cc:1434] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 13:09:43.575455 30765 raft_consensus.cc:515] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.588006 30571 raft_consensus.cc:2468] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:43.591009 30692 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:43.594326 30695 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
W20260709 13:09:43.594146 30386 leader_election.cc:343] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.594892 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95d00bccf5ce41f4b601bba7d235ffbb" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:43.595392 30765 leader_election.cc:290] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:43.595773 30497 raft_consensus.cc:2393] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:43.597697 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:43.597803 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95d00bccf5ce41f4b601bba7d235ffbb" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:43.598383 30691 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:43.599766 30691 ts_tablet_manager.cc:1403] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.014s
I20260709 13:09:43.602154 30695 tablet_bootstrap.cc:654] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.603884 30691 raft_consensus.cc:359] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.599056 30570 raft_consensus.cc:2468] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:43.604725 30691 raft_consensus.cc:385] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.600317 30388 leader_election.cc:304] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:43.610797 30388 leader_election.cc:304] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:43.612883 30825 raft_consensus.cc:3060] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.615474 30789 raft_consensus.cc:2804] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:43.615912 30789 raft_consensus.cc:493] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.616257 30789 raft_consensus.cc:3060] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.617460 30765 raft_consensus.cc:493] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.605494 30691 raft_consensus.cc:740] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.618101 30765 raft_consensus.cc:515] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.622530 30691 consensus_queue.cc:260] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.630018 30692 tablet_bootstrap.cc:654] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.648944 30691 ts_tablet_manager.cc:1434] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 13:09:43.636668 30826 raft_consensus.cc:515] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.656893 30691 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:43.672523 30709 raft_consensus.cc:493] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19b1445cece541d3b2abe177283fef44)
I20260709 13:09:43.681551 30826 leader_election.cc:290] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:43.670673 30723 consensus_queue.cc:1048] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:09:43.675031 30825 raft_consensus.cc:2749] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:43.686224 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845d59ad223a4f6c9c388017a170527b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:43.686911 30644 raft_consensus.cc:3060] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.691337 30789 raft_consensus.cc:515] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.683336 30709 raft_consensus.cc:515] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.693964 30765 leader_election.cc:290] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:43.696105 30857 consensus_queue.cc:1048] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:43.695887 30318 catalog_manager.cc:5697] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:43.698669 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:43.699314 30496 raft_consensus.cc:2468] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:43.701877 30695 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:43.702868 30695 ts_tablet_manager.cc:1403] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.109s	user 0.034s	sys 0.015s
I20260709 13:09:43.700659 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845d59ad223a4f6c9c388017a170527b" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:43.717229 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:43.718380 30570 raft_consensus.cc:2468] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:43.725126 30695 raft_consensus.cc:359] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.727571 30695 raft_consensus.cc:385] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.729303 30695 raft_consensus.cc:740] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.721477 30569 raft_consensus.cc:3060] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.732432 30690 tablet_bootstrap.cc:654] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.744451 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:43.745076 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e003633f9bf24b09b3789ddc3ac48cc1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:43.743737 30709 leader_election.cc:290] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:43.745518 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e003633f9bf24b09b3789ddc3ac48cc1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:43.747267 30692 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:43.747226 30611 leader_election.cc:304] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:43.747885 30692 ts_tablet_manager.cc:1403] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.166s	user 0.016s	sys 0.017s
I20260709 13:09:43.748466 30709 raft_consensus.cc:2749] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:43.750886 30692 raft_consensus.cc:359] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.751605 30692 raft_consensus.cc:385] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.751893 30692 raft_consensus.cc:740] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.752665 30692 consensus_queue.cc:260] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.727769 30789 leader_election.cc:290] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:43.755649 30692 ts_tablet_manager.cc:1434] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:43.760147 30695 consensus_queue.cc:260] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.763808 30692 tablet_bootstrap.cc:492] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:43.769479 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:43.770169 30570 raft_consensus.cc:3060] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:43.772622 30386 leader_election.cc:343] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:43.779470 30695 ts_tablet_manager.cc:1434] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 13:09:43.780838 30695 tablet_bootstrap.cc:492] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:43.782680 30644 raft_consensus.cc:2468] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:43.792507 30695 tablet_bootstrap.cc:654] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.768440 30388 leader_election.cc:304] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:43.802078 30692 tablet_bootstrap.cc:654] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.804950 30461 leader_election.cc:304] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:43.806766 30839 raft_consensus.cc:2804] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:43.807322 30839 raft_consensus.cc:697] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:43.808341 30826 consensus_queue.cc:1048] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:43.808162 30839 consensus_queue.cc:237] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.812867 30825 raft_consensus.cc:2804] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:43.816537 30691 tablet_bootstrap.cc:654] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.816800 30825 raft_consensus.cc:493] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:43.823896 30871 log.cc:927] Time spent T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:09:43.822717 30825 raft_consensus.cc:3060] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:43.829752 30569 raft_consensus.cc:2468] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:43.860421 30570 raft_consensus.cc:2468] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:43.870354 30692 tablet_bootstrap.cc:492] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:43.870710 30695 tablet_bootstrap.cc:492] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:43.870949 30692 ts_tablet_manager.cc:1403] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.107s	user 0.011s	sys 0.028s
I20260709 13:09:43.871248 30695 ts_tablet_manager.cc:1403] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.091s	user 0.027s	sys 0.003s
I20260709 13:09:43.867897 30825 raft_consensus.cc:515] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.873869 30825 leader_election.cc:290] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:43.874102 30692 raft_consensus.cc:359] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.874782 30692 raft_consensus.cc:385] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.875064 30691 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:43.875574 30691 ts_tablet_manager.cc:1403] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.221s	user 0.016s	sys 0.000s
I20260709 13:09:43.876566 30690 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:43.877117 30690 ts_tablet_manager.cc:1403] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.302s	user 0.015s	sys 0.002s
I20260709 13:09:43.878348 30691 raft_consensus.cc:359] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.879057 30691 raft_consensus.cc:385] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.879401 30691 raft_consensus.cc:740] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.880084 30690 raft_consensus.cc:359] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.880792 30690 raft_consensus.cc:385] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.880370 30691 consensus_queue.cc:260] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.881104 30690 raft_consensus.cc:740] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.881891 30690 consensus_queue.cc:260] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.882350 30695 raft_consensus.cc:359] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.883070 30695 raft_consensus.cc:385] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.883338 30695 raft_consensus.cc:740] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.884050 30695 consensus_queue.cc:260] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.884678 30690 ts_tablet_manager.cc:1434] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:09:43.886035 30690 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:43.886235 30695 ts_tablet_manager.cc:1434] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:09:43.887243 30695 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:43.890775 30691 ts_tablet_manager.cc:1434] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:43.892004 30691 tablet_bootstrap.cc:492] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:43.894387 30695 tablet_bootstrap.cc:654] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.875073 30692 raft_consensus.cc:740] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.896746 30692 consensus_queue.cc:260] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:43.900863 30695 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:43.901402 30692 ts_tablet_manager.cc:1434] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:09:43.901415 30695 ts_tablet_manager.cc:1403] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:09:43.902667 30692 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:43.904507 30695 raft_consensus.cc:359] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.905269 30695 raft_consensus.cc:385] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.905536 30695 raft_consensus.cc:740] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.906230 30695 consensus_queue.cc:260] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.937479 30695 ts_tablet_manager.cc:1434] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:09:43.938766 30695 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:43.946735 30695 tablet_bootstrap.cc:654] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.959137 30695 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:43.959782 30695 ts_tablet_manager.cc:1403] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:09:43.966292 30709 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.966922 30709 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.970371 30709 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.976366 30695 raft_consensus.cc:359] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.977224 30695 raft_consensus.cc:385] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.977491 30695 raft_consensus.cc:740] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.978261 30690 tablet_bootstrap.cc:654] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.978678 30692 tablet_bootstrap.cc:654] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.978411 30695 consensus_queue.cc:260] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:43.981492 30695 ts_tablet_manager.cc:1434] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:09:43.982471 30709 raft_consensus.cc:493] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.982733 30695 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:43.983158 30709 raft_consensus.cc:515] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:43.985579 30691 tablet_bootstrap.cc:654] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.989848 30695 tablet_bootstrap.cc:654] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:43.992604 30692 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:43.993178 30692 ts_tablet_manager.cc:1403] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.000s
I20260709 13:09:43.996577 30692 raft_consensus.cc:359] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:43.997589 30692 raft_consensus.cc:385] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:43.997908 30692 raft_consensus.cc:740] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:43.999009 30709 leader_election.cc:290] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:43.998991 30692 consensus_queue.cc:260] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.002285 30692 ts_tablet_manager.cc:1434] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
W20260709 13:09:43.812394 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 51681.5us
I20260709 13:09:44.007319 30690 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:44.007862 30690 ts_tablet_manager.cc:1403] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.122s	user 0.009s	sys 0.004s
I20260709 13:09:44.010226 30826 raft_consensus.cc:493] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.010764 30826 raft_consensus.cc:515] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.012336 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:44.012955 30826 leader_election.cc:290] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:44.013075 30571 raft_consensus.cc:3060] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.024252 30857 consensus_queue.cc:1048] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:44.026467 30643 raft_consensus.cc:3060] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.030892 30571 raft_consensus.cc:2468] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:44.032047 30495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24a265170a904a89a2801145c1a61172" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.032510 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c1fe97f07496c92182b3065536c15" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.033062 30495 raft_consensus.cc:2468] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:44.033244 30419 raft_consensus.cc:2468] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:44.016551 30644 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.035921 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:44.037853 30690 raft_consensus.cc:359] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.038601 30690 raft_consensus.cc:385] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.038859 30690 raft_consensus.cc:740] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.039534 30690 consensus_queue.cc:260] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.036913 30493 raft_consensus.cc:3060] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.041553 30611 leader_election.cc:304] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:44.042479 30709 raft_consensus.cc:2804] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:44.042948 30709 raft_consensus.cc:493] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.043282 30709 raft_consensus.cc:3060] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.018384 30695 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:44.048123 30695 ts_tablet_manager.cc:1403] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.002s
I20260709 13:09:44.048803 30462 leader_election.cc:304] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:44.049973 30690 ts_tablet_manager.cc:1434] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:09:44.018651 30692 tablet_bootstrap.cc:492] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:44.018980 30691 tablet_bootstrap.cc:492] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:44.018198 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.051764 30695 raft_consensus.cc:359] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.055418 30695 raft_consensus.cc:385] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.055771 30695 raft_consensus.cc:740] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.055851 30691 ts_tablet_manager.cc:1403] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.164s	user 0.015s	sys 0.010s
I20260709 13:09:44.056811 30695 consensus_queue.cc:260] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.060961 30695 ts_tablet_manager.cc:1434] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.005s
I20260709 13:09:44.065794 30695 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:44.065920 30691 raft_consensus.cc:359] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.067041 30691 raft_consensus.cc:385] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.067698 30691 raft_consensus.cc:740] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.070194 30691 consensus_queue.cc:260] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.072304 30831 raft_consensus.cc:2804] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:44.072851 30831 raft_consensus.cc:493] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.073257 30831 raft_consensus.cc:3060] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.073563 30691 ts_tablet_manager.cc:1434] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:09:44.074846 30691 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:44.076680 30690 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:44.077514 30857 consensus_queue.cc:1048] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:09:44.094240 30869 consensus_queue.cc:1048] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:44.100435 30388 leader_election.cc:304] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:44.101493 30869 raft_consensus.cc:2804] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:44.102387 30869 raft_consensus.cc:697] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:44.103386 30869 consensus_queue.cc:237] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.105605 30493 raft_consensus.cc:2468] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:44.113250 30388 leader_election.cc:304] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:44.114087 30868 raft_consensus.cc:2804] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:44.115072 30868 raft_consensus.cc:697] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:44.115928 30868 consensus_queue.cc:237] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.187007 30816 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.187561 30816 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.189890 30816 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:44.190790 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:44.192337 30535 leader_election.cc:343] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.220484 30709 raft_consensus.cc:515] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.077720 30723 raft_consensus.cc:493] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.223483 30723 raft_consensus.cc:515] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.224295 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24a265170a904a89a2801145c1a61172" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:44.224972 30493 raft_consensus.cc:3060] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.231721 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c1fe97f07496c92182b3065536c15" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:44.229251 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.234838 30571 raft_consensus.cc:2468] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:44.227741 30723 leader_election.cc:290] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:44.237852 30793 raft_consensus.cc:493] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.238013 30709 leader_election.cc:290] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:44.238456 30793 raft_consensus.cc:515] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.244061 30793 leader_election.cc:290] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:44.246480 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.247136 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:44.248674 30611 leader_election.cc:343] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.249616 30571 raft_consensus.cc:2468] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:44.251374 30611 leader_election.cc:304] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44
I20260709 13:09:44.252723 30793 raft_consensus.cc:2804] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:44.253345 30793 raft_consensus.cc:493] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.253708 30793 raft_consensus.cc:3060] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.255475 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:44.258769 30386 leader_election.cc:343] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.261960 30831 raft_consensus.cc:515] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.266462 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c1fe97f07496c92182b3065536c15" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:44.267143 30569 raft_consensus.cc:3060] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.269135 30709 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.269853 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b1c1fe97f07496c92182b3065536c15" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:44.269943 30709 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.271766 30692 tablet_bootstrap.cc:654] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:44.263156 30388 leader_election.cc:343] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.270638 30421 raft_consensus.cc:3060] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.275149 30709 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:44.274477 30388 leader_election.cc:304] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:44.276031 30723 raft_consensus.cc:2749] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.276449 30891 raft_consensus.cc:493] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.277467 30891 raft_consensus.cc:515] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.279973 30569 raft_consensus.cc:2468] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:44.280803 30831 leader_election.cc:290] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:44.280920 30709 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.281539 30709 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.282444 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:44.283977 30709 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:44.284940 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:44.285566 30891 leader_election.cc:290] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:44.287145 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "107dd7c115d0402a9fd4e3312f621930" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:44.287324 30421 raft_consensus.cc:2468] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
W20260709 13:09:44.288563 30535 leader_election.cc:343] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.292142 30793 raft_consensus.cc:515] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.296073 30462 leader_election.cc:304] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:44.292923 30493 raft_consensus.cc:2468] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:44.297621 30611 leader_election.cc:304] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:44.297833 30691 tablet_bootstrap.cc:654] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.298314 30709 raft_consensus.cc:2804] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:44.300056 30692 tablet_bootstrap.cc:492] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:44.300745 30692 ts_tablet_manager.cc:1403] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.297s	user 0.007s	sys 0.009s
I20260709 13:09:44.300784 30831 raft_consensus.cc:2804] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:09:44.301672 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 53283.3us
I20260709 13:09:44.303448 30831 raft_consensus.cc:697] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:44.305053 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "107dd7c115d0402a9fd4e3312f621930" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.305752 30494 raft_consensus.cc:2468] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:44.306999 30831 consensus_queue.cc:237] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.309532 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.308257 30537 leader_election.cc:304] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:44.311074 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24a265170a904a89a2801145c1a61172" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.311255 30891 raft_consensus.cc:2804] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:44.311753 30419 raft_consensus.cc:2468] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:44.312911 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24a265170a904a89a2801145c1a61172" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:44.313655 30420 raft_consensus.cc:3060] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.314118 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
W20260709 13:09:44.316062 30611 leader_election.cc:343] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.317471 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.319639 30417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.320552 30691 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:44.321210 30691 ts_tablet_manager.cc:1403] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.247s	user 0.016s	sys 0.002s
I20260709 13:09:44.319754 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:44.322301 30566 raft_consensus.cc:3060] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.326229 30642 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.334570 30691 raft_consensus.cc:359] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.335510 30691 raft_consensus.cc:385] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.335912 30691 raft_consensus.cc:740] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.338388 30691 consensus_queue.cc:260] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.340420 30691 ts_tablet_manager.cc:1434] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 13:09:44.341856 30691 tablet_bootstrap.cc:492] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:44.347927 30692 raft_consensus.cc:359] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.362689 30692 raft_consensus.cc:385] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.363125 30692 raft_consensus.cc:740] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.364076 30692 consensus_queue.cc:260] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.366670 30692 ts_tablet_manager.cc:1434] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.004s
I20260709 13:09:44.367327 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.368393 30692 tablet_bootstrap.cc:492] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:44.357208 30691 tablet_bootstrap.cc:654] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.348807 30793 leader_election.cc:290] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:44.360961 30566 raft_consensus.cc:2468] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:44.320880 30417 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:44.311759 30891 raft_consensus.cc:493] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.393738 30611 leader_election.cc:304] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44
I20260709 13:09:44.392233 30610 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:44.398458 30420 raft_consensus.cc:2468] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:44.394188 30891 raft_consensus.cc:3060] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.412247 30690 tablet_bootstrap.cc:654] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.052311 30826 consensus_queue.cc:1048] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:09:44.417706 30827 consensus_queue.cc:1048] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:44.419039 30695 tablet_bootstrap.cc:654] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.422503 30709 raft_consensus.cc:697] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
W20260709 13:09:44.423871 30536 leader_election.cc:343] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.472718 30896 raft_consensus.cc:493] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.473311 30893 raft_consensus.cc:2804] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:44.473387 30896 raft_consensus.cc:515] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.474452 30691 tablet_bootstrap.cc:492] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:44.475020 30691 ts_tablet_manager.cc:1403] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.133s	user 0.015s	sys 0.009s
I20260709 13:09:44.463125 30536 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:44.480127 30816 raft_consensus.cc:2749] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:44.480273 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 57841.4us
I20260709 13:09:44.485148 30709 consensus_queue.cc:237] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.487790 30692 tablet_bootstrap.cc:654] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:44.468081 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 74485us
I20260709 13:09:44.488499 30891 raft_consensus.cc:515] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
W20260709 13:09:44.475831 30610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 90250.1us
I20260709 13:09:44.508656 30891 leader_election.cc:290] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:44.505621 30896 leader_election.cc:290] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:44.503067 30802 consensus_queue.cc:1048] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:09:44.535411 30318 catalog_manager.cc:5697] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:44.537772 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.527729 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "107dd7c115d0402a9fd4e3312f621930" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:44.539639 30494 raft_consensus.cc:3060] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.545284 30691 raft_consensus.cc:359] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.546025 30691 raft_consensus.cc:385] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.546306 30691 raft_consensus.cc:740] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.547358 30691 consensus_queue.cc:260] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.544627 30827 consensus_queue.cc:1048] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:09:44.552218 30691 ts_tablet_manager.cc:1434] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 13:09:44.553447 30691 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:44.555258 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:44.556196 30569 raft_consensus.cc:2393] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:44.557541 30816 consensus_queue.cc:1048] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:09:44.559993 30691 tablet_bootstrap.cc:654] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.606694 30695 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:44.607268 30695 ts_tablet_manager.cc:1403] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.542s	user 0.024s	sys 0.008s
I20260709 13:09:44.609068 30690 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:44.609741 30690 ts_tablet_manager.cc:1403] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.554s	user 0.017s	sys 0.011s
I20260709 13:09:44.625650 30610 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:44.627013 30610 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 955d2e588381448981f85fadf225aa05
W20260709 13:09:44.605147 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 61174.4us
I20260709 13:09:44.629956 30611 leader_election.cc:304] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:44.613020 30690 raft_consensus.cc:359] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.632539 30690 raft_consensus.cc:385] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.633054 30690 raft_consensus.cc:740] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.613991 30893 raft_consensus.cc:697] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:44.643616 30891 raft_consensus.cc:493] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.629978 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "107dd7c115d0402a9fd4e3312f621930" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:44.657486 30494 raft_consensus.cc:2468] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:44.658005 30692 tablet_bootstrap.cc:492] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:44.637765 30793 raft_consensus.cc:3060] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.678638 30891 raft_consensus.cc:515] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.666949 30896 raft_consensus.cc:2804] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:44.670039 30902 raft_consensus.cc:2749] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.654234 30695 raft_consensus.cc:359] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.675414 30690 consensus_queue.cc:260] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.691377 30867 consensus_queue.cc:1048] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:09:44.692490 30537 leader_election.cc:304] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:44.694715 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.695199 30891 leader_election.cc:290] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:44.695899 30891 raft_consensus.cc:2804] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:44.696353 30896 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955d2e588381448981f85fadf225aa05)
I20260709 13:09:44.677158 30893 consensus_queue.cc:237] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.686931 30692 ts_tablet_manager.cc:1403] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.319s	user 0.020s	sys 0.012s
I20260709 13:09:44.690886 30695 raft_consensus.cc:385] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.705334 30695 raft_consensus.cc:740] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
W20260709 13:09:44.696774 30537 leader_election.cc:343] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.706990 30695 consensus_queue.cc:260] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.696926 30896 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:44.702903 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:44.698740 30891 raft_consensus.cc:697] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:44.709507 30908 raft_consensus.cc:2749] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.715209 30695 ts_tablet_manager.cc:1434] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.108s	user 0.010s	sys 0.000s
I20260709 13:09:44.719511 30690 ts_tablet_manager.cc:1434] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
I20260709 13:09:44.726125 30691 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:44.726758 30691 ts_tablet_manager.cc:1403] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.174s	user 0.021s	sys 0.008s
I20260709 13:09:44.726543 30692 raft_consensus.cc:359] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.727233 30692 raft_consensus.cc:385] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.727568 30692 raft_consensus.cc:740] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.728245 30692 consensus_queue.cc:260] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.729770 30691 raft_consensus.cc:359] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.730509 30691 raft_consensus.cc:385] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.735419 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:44.741053 30826 raft_consensus.cc:493] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.742153 30692 ts_tablet_manager.cc:1434] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:09:44.741858 30826 raft_consensus.cc:515] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.743357 30692 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
W20260709 13:09:44.735705 30535 leader_election.cc:343] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.730809 30691 raft_consensus.cc:740] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.749847 30695 tablet_bootstrap.cc:492] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:44.755807 30690 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:44.756934 30891 consensus_queue.cc:237] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.741165 30916 raft_consensus.cc:493] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.760736 30896 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.764329 30691 consensus_queue.cc:260] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.765532 30826 leader_election.cc:290] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:44.769101 30691 ts_tablet_manager.cc:1434] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.003s
I20260709 13:09:44.765976 30827 raft_consensus.cc:493] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.776240 30827 raft_consensus.cc:515] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.776477 30869 consensus_queue.cc:1048] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:44.776825 30691 tablet_bootstrap.cc:492] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:44.770716 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:44.779215 30535 leader_election.cc:343] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.779691 30535 leader_election.cc:304] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:44.779706 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad2905df5e84f04bcc21f6c9713c744" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:44.780167 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.780726 30571 raft_consensus.cc:2393] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19b1445cece541d3b2abe177283fef44 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:44.765727 30916 raft_consensus.cc:515] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.782078 30857 raft_consensus.cc:2749] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.783264 30690 tablet_bootstrap.cc:654] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.784351 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:44.785027 30419 raft_consensus.cc:2468] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:44.785439 30839 consensus_queue.cc:1048] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:44.786054 30610 leader_election.cc:304] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:44.783341 30916 leader_election.cc:290] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:44.786989 30692 tablet_bootstrap.cc:654] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.786912 30908 raft_consensus.cc:2804] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:09:44.783625 30462 leader_election.cc:343] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.787528 30908 raft_consensus.cc:493] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.787941 30908 raft_consensus.cc:3060] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.790127 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:44.791042 30494 raft_consensus.cc:2468] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:44.804867 30789 consensus_queue.cc:1048] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:44.807803 30827 leader_election.cc:290] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:44.807977 30793 raft_consensus.cc:2749] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.810456 30896 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:44.811088 30695 tablet_bootstrap.cc:654] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.812716 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:44.801239 30643 raft_consensus.cc:1240] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Rejecting Update request from peer 955d2e588381448981f85fadf225aa05 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:44.802103 30463 leader_election.cc:304] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:44.829807 30827 raft_consensus.cc:3060] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.822134 30908 raft_consensus.cc:515] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.827685 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:44.827445 30789 consensus_queue.cc:1048] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:44.828013 30886 consensus_queue.cc:1048] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:44.835307 30691 tablet_bootstrap.cc:654] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.836573 30816 consensus_queue.cc:1059] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:09:44.840399 30816 raft_consensus.cc:3055] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:44.840786 30816 raft_consensus.cc:740] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:44.841602 30816 consensus_queue.cc:260] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.842948 30816 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:44.843910 30610 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:44.845222 30896 raft_consensus.cc:2749] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:44.850234 30690 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:44.850975 30690 ts_tablet_manager.cc:1403] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.009s
I20260709 13:09:44.852100 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:44.865319 30908 leader_election.cc:290] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:44.866212 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:44.869637 30419 raft_consensus.cc:3060] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.870493 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:44.871114 30494 raft_consensus.cc:3060] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:44.871196 30692 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:44.871852 30692 ts_tablet_manager.cc:1403] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.129s	user 0.014s	sys 0.022s
I20260709 13:09:44.877240 30695 tablet_bootstrap.cc:492] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:44.877691 30691 tablet_bootstrap.cc:492] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:44.877892 30695 ts_tablet_manager.cc:1403] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.128s	user 0.024s	sys 0.020s
I20260709 13:09:44.878382 30691 ts_tablet_manager.cc:1403] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.004s
I20260709 13:09:44.881631 30692 raft_consensus.cc:359] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.882391 30692 raft_consensus.cc:385] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.882493 30419 raft_consensus.cc:2468] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:44.882716 30692 raft_consensus.cc:740] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.883488 30692 consensus_queue.cc:260] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:44.883893 30610 leader_election.cc:304] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:44.892720 30691 raft_consensus.cc:359] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.893548 30691 raft_consensus.cc:385] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.893865 30691 raft_consensus.cc:740] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.894605 30691 consensus_queue.cc:260] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.896116 30691 ts_tablet_manager.cc:1434] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:09:44.889328 30908 raft_consensus.cc:2804] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:44.898311 30908 raft_consensus.cc:697] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:44.898926 30827 raft_consensus.cc:2749] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.899204 30908 consensus_queue.cc:237] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:44.901644 30695 raft_consensus.cc:359] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.903050 30695 raft_consensus.cc:385] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.903295 30695 raft_consensus.cc:740] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.904019 30695 consensus_queue.cc:260] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.906469 30692 ts_tablet_manager.cc:1434] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:09:44.908543 30695 ts_tablet_manager.cc:1434] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:09:44.909688 30695 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:44.910393 30691 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
W20260709 13:09:44.905246 30463 leader_election.cc:343] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.918222 30463 leader_election.cc:304] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:44.919179 30839 raft_consensus.cc:2749] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:44.921149 30692 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:44.919708 30494 raft_consensus.cc:2468] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:44.935979 30857 raft_consensus.cc:493] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.936769 30857 raft_consensus.cc:515] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:44.970659 30692 tablet_bootstrap.cc:654] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.971390 30690 raft_consensus.cc:359] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.972110 30690 raft_consensus.cc:385] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:44.972393 30690 raft_consensus.cc:740] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:44.975649 30695 tablet_bootstrap.cc:654] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:44.977054 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:44.978058 30857 leader_election.cc:290] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
W20260709 13:09:44.978935 30535 leader_election.cc:343] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:44.981794 30690 consensus_queue.cc:260] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:44.987020 30690 ts_tablet_manager.cc:1434] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.136s	user 0.011s	sys 0.003s
I20260709 13:09:44.988117 30908 consensus_queue.cc:1048] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:44.989543 30690 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:44.991719 30691 tablet_bootstrap.cc:654] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.012531 30867 raft_consensus.cc:493] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.016983 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.021310 30908 consensus_queue.cc:1048] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:45.018049 30493 raft_consensus.cc:2468] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:45.017360 30867 raft_consensus.cc:515] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.027595 30537 leader_election.cc:304] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:45.029237 30867 leader_election.cc:290] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:45.029886 30318 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36828 (request call id 32) took 1005 ms. Trace:
I20260709 13:09:45.030968 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86637a5e467340ae9ac4f57d22c68bf6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.031195 30696 raft_consensus.cc:2804] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.031576 30696 raft_consensus.cc:493] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.032090 30696 raft_consensus.cc:3060] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.038437 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86637a5e467340ae9ac4f57d22c68bf6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.038630 30316 catalog_manager.cc:5697] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:45.041136 30908 raft_consensus.cc:493] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.041697 30908 raft_consensus.cc:515] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.043919 30908 leader_election.cc:290] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:09:45.033095 30386 leader_election.cc:343] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:45.044792 30388 leader_election.cc:343] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.045157 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "107dd7c115d0402a9fd4e3312f621930" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:45.042848 30316 catalog_manager.cc:5697] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.030318 30318 rpcz_store.cc:276] 0709 13:09:44.024925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.025168 (+   243us) service_pool.cc:224] Handling call
0709 13:09:45.029821 (+1004653us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:44.674512 (+     0us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b4400022b4c. stack: 00007f3b6d6b0343 00007f3b6d6b0240 00007f3b6d03f1d4 00007f3b76d7d304 00007f3b76d7d299 00007f3b6d56adb1 00007f3b6d608c85 00007f3b6d6096a2 00007f3b7037b1db 00007f3b70376aaf 00007f3b6d6c50ac 00007f3b7651fab9 00007f3b6d6e169b 00007f3b6d6df93f 00007f3b6d6df455 00007f3b7420416c
0709 13:09:44.866883 (+192371us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:44.907976 (+ 41093us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:44.908008 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:09:44.908020 (+    12us) tablet.cc:662] Decoding operations
0709 13:09:44.908411 (+   391us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:44.908479 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:44.908509 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:44.908701 (+   192us) tablet.cc:701] Row locks acquired
0709 13:09:44.908718 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:09:44.908862 (+   144us) write_op.cc:270] Start()
0709 13:09:44.909087 (+   225us) write_op.cc:276] Timestamp: P: 1783602584908815 usec, L: 0
0709 13:09:44.909109 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:09:44.909798 (+   689us) log.cc:844] Serialized 540 byte log entry
0709 13:09:44.979538 (+ 69740us) op_driver.cc:464] REPLICATION: finished
0709 13:09:45.021607 (+ 42069us) write_op.cc:301] APPLY: starting
0709 13:09:45.021709 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:45.021831 (+   122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:45.021849 (+    18us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:45.022097 (+   248us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:45.022243 (+   146us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:45.022286 (+    43us) write_op.cc:312] APPLY: finished
0709 13:09:45.022494 (+   208us) log.cc:844] Serialized 22 byte log entry
0709 13:09:45.022647 (+   153us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:45.022727 (+    80us) write_op.cc:454] Released schema lock
0709 13:09:45.022767 (+    40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":132203,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":41795,"apply.run_cpu_time_us":1581,"apply.run_wall_time_us":4836,"num_ops":1,"prepare.queue_time_us":232011,"prepare.run_cpu_time_us":14902,"prepare.run_wall_time_us":110571,"raft.queue_time_us":2806,"raft.run_cpu_time_us":17904,"raft.run_wall_time_us":40413,"replication_time_us":69259,"spinlock_wait_cycles":16664960,"thread_start_us":231819,"threads_started":4,"wal-append.queue_time_us":63884}]]}
I20260709 13:09:45.044821 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "107dd7c115d0402a9fd4e3312f621930" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.051864 30494 raft_consensus.cc:2393] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:45.055107 30690 tablet_bootstrap.cc:654] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.054800 30611 leader_election.cc:304] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:45.056560 30908 raft_consensus.cc:3060] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.047075 30388 leader_election.cc:304] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:45.060923 30869 raft_consensus.cc:2749] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.063476 30696 raft_consensus.cc:515] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.070168 30908 raft_consensus.cc:2749] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.079919 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:45.080798 30494 raft_consensus.cc:3060] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.083422 30696 leader_election.cc:290] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:45.085386 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:45.086081 30692 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:45.086604 30692 ts_tablet_manager.cc:1403] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.167s	user 0.031s	sys 0.029s
I20260709 13:09:45.086962 30695 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:45.087484 30695 ts_tablet_manager.cc:1403] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.178s	user 0.026s	sys 0.012s
I20260709 13:09:45.089365 30692 raft_consensus.cc:359] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.090113 30691 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.090457 30692 raft_consensus.cc:385] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.090588 30691 ts_tablet_manager.cc:1403] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.194s	user 0.011s	sys 0.023s
I20260709 13:09:45.090806 30692 raft_consensus.cc:740] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.091830 30692 consensus_queue.cc:260] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.091701 30695 raft_consensus.cc:359] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.092705 30695 raft_consensus.cc:385] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.092988 30695 raft_consensus.cc:740] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.093605 30691 raft_consensus.cc:359] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.094511 30691 raft_consensus.cc:385] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.094759 30691 raft_consensus.cc:740] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.093796 30695 consensus_queue.cc:260] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.095228 30692 ts_tablet_manager.cc:1434] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:45.095422 30691 consensus_queue.cc:260] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.096499 30692 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
W20260709 13:09:45.090497 30535 leader_election.cc:343] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.098075 30691 ts_tablet_manager.cc:1434] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:45.098520 30908 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.099131 30691 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.099088 30908 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.102397 30908 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.097401 30695 ts_tablet_manager.cc:1434] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:09:45.103452 30692 tablet_bootstrap.cc:654] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.104076 30695 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:45.106990 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:45.104053 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:45.109861 30518 debug-util.cc:398] Leaking SignalData structure 0x7b08002b99c0 after lost signal to thread 20749
W20260709 13:09:45.111296 30518 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6a80 after lost signal to thread 30354
W20260709 13:09:45.113615 30518 debug-util.cc:398] Leaking SignalData structure 0x7b080020d540 after lost signal to thread 30446
W20260709 13:09:45.117753 30518 debug-util.cc:398] Leaking SignalData structure 0x7b080029e9e0 after lost signal to thread 30521
W20260709 13:09:45.119441 30518 debug-util.cc:398] Leaking SignalData structure 0x7b08000e30e0 after lost signal to thread 30595
I20260709 13:09:45.119961 30816 raft_consensus.cc:493] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.120848 30691 tablet_bootstrap.cc:654] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.120755 30816 raft_consensus.cc:515] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:45.121034 30518 debug-util.cc:398] Leaking SignalData structure 0x7b080015f2e0 after lost signal to thread 30669
I20260709 13:09:45.125355 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6ae4a76cb84d41a9dcc0c1cce9cba2" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.126852 30695 tablet_bootstrap.cc:654] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.127576 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6ae4a76cb84d41a9dcc0c1cce9cba2" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.128289 30816 leader_election.cc:290] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:45.127904 30536 leader_election.cc:343] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:45.130141 30535 leader_election.cc:343] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.130661 30535 leader_election.cc:304] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:45.132112 30692 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:45.132802 30692 ts_tablet_manager.cc:1403] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.009s
I20260709 13:09:45.133306 30691 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.133849 30691 ts_tablet_manager.cc:1403] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.003s
I20260709 13:09:45.134687 30816 raft_consensus.cc:2749] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.136902 30691 raft_consensus.cc:359] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.137730 30691 raft_consensus.cc:385] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.137568 30692 raft_consensus.cc:359] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.138051 30691 raft_consensus.cc:740] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.138479 30692 raft_consensus.cc:385] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.138823 30692 raft_consensus.cc:740] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.139204 30691 consensus_queue.cc:260] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.140156 30692 consensus_queue.cc:260] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.142414 30691 ts_tablet_manager.cc:1434] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:09:45.142589 30692 ts_tablet_manager.cc:1434] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:09:45.143532 30691 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.143535 30692 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:45.149430 30570 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:45.108827 30416 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:45.151227 30691 tablet_bootstrap.cc:654] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.151484 30610 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:45.152436 30908 raft_consensus.cc:2804] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.152838 30908 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.153191 30908 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.167289 30816 raft_consensus.cc:493] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.168108 30816 raft_consensus.cc:515] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.172204 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:45.173748 30536 leader_election.cc:343] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.175264 30908 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.175969 30857 consensus_queue.cc:1048] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:45.175326 30816 leader_election.cc:290] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:45.178475 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:45.178699 30908 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.179128 30566 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.182830 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.185945 30491 raft_consensus.cc:2468] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:45.184454 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:45.190888 30416 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.191988 30537 leader_election.cc:304] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:45.188809 30494 raft_consensus.cc:2468] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:45.194283 30816 raft_consensus.cc:2804] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.194895 30816 raft_consensus.cc:493] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.196442 30816 raft_consensus.cc:3060] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.195683 30537 leader_election.cc:304] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:45.197036 30566 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:45.201736 30857 raft_consensus.cc:2804] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.203349 30857 raft_consensus.cc:697] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:45.207285 30857 consensus_queue.cc:237] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.233801 30908 consensus_queue.cc:1048] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.236047 30611 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:45.237843 30908 raft_consensus.cc:2804] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.238430 30908 raft_consensus.cc:697] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:45.239359 30908 consensus_queue.cc:237] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.245646 30692 tablet_bootstrap.cc:654] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.270800 30690 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:45.271867 30690 ts_tablet_manager.cc:1403] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.284s	user 0.008s	sys 0.005s
I20260709 13:09:45.265117 30319 catalog_manager.cc:5697] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.284835 30816 raft_consensus.cc:515] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.287218 30691 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.287766 30691 ts_tablet_manager.cc:1403] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.144s	user 0.053s	sys 0.057s
I20260709 13:09:45.289834 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:45.289870 30831 raft_consensus.cc:493] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.290729 30691 raft_consensus.cc:359] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.291447 30691 raft_consensus.cc:385] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.291712 30691 raft_consensus.cc:740] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.290742 30831 raft_consensus.cc:515] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.293151 30691 consensus_queue.cc:260] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.301093 30416 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:45.305936 30690 raft_consensus.cc:359] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.307878 30690 raft_consensus.cc:385] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.308197 30690 raft_consensus.cc:740] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.309192 30831 leader_election.cc:290] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.308786 30691 ts_tablet_manager.cc:1434] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:09:45.311625 30691 tablet_bootstrap.cc:492] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.309362 30690 consensus_queue.cc:260] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.314409 30695 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:45.315052 30695 ts_tablet_manager.cc:1403] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.211s	user 0.031s	sys 0.024s
I20260709 13:09:45.317210 30690 ts_tablet_manager.cc:1434] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.003s
I20260709 13:09:45.318540 30690 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:45.320075 30695 raft_consensus.cc:359] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.320886 30695 raft_consensus.cc:385] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.321278 30695 raft_consensus.cc:740] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.322021 30695 consensus_queue.cc:260] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.323135 30902 consensus_queue.cc:1048] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.330001 30891 consensus_queue.cc:1048] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.332065 30691 tablet_bootstrap.cc:654] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.336440 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86637a5e467340ae9ac4f57d22c68bf6" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.335799 30320 catalog_manager.cc:5697] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.337355 30642 raft_consensus.cc:2468] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:45.356047 30461 leader_election.cc:304] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:45.357782 30827 raft_consensus.cc:2804] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.358338 30827 raft_consensus.cc:493] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.358680 30827 raft_consensus.cc:3060] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.361276 30691 tablet_bootstrap.cc:492] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.361863 30691 ts_tablet_manager.cc:1403] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.005s
I20260709 13:09:45.362661 30692 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:45.363274 30692 ts_tablet_manager.cc:1403] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.220s	user 0.009s	sys 0.005s
I20260709 13:09:45.364915 30691 raft_consensus.cc:359] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.351460 30320 catalog_manager.cc:5697] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:45.365720 30691 raft_consensus.cc:385] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.366781 30691 raft_consensus.cc:740] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.339190 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86637a5e467340ae9ac4f57d22c68bf6" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.367741 30691 consensus_queue.cc:260] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.340927 30695 ts_tablet_manager.cc:1434] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:09:45.344118 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:45.370785 30497 raft_consensus.cc:3060] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.371301 30695 tablet_bootstrap.cc:492] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:45.344801 30816 leader_election.cc:290] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:45.346419 30690 tablet_bootstrap.cc:654] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.383540 30497 raft_consensus.cc:2468] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:45.369879 30419 raft_consensus.cc:2468] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
W20260709 13:09:45.342622 30536 leader_election.cc:343] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.385154 30537 leader_election.cc:304] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
W20260709 13:09:45.387333 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 90579.5us
I20260709 13:09:45.371601 30692 raft_consensus.cc:359] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.391378 30692 raft_consensus.cc:385] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.391099 30827 raft_consensus.cc:515] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.378681 30695 tablet_bootstrap.cc:654] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.380275 30691 ts_tablet_manager.cc:1434] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:09:45.394471 30691 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.392200 30692 raft_consensus.cc:740] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.378677 30857 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.399144 30857 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.401674 30891 raft_consensus.cc:2804] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.401994 30857 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.402315 30891 raft_consensus.cc:697] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:45.403111 30692 consensus_queue.cc:260] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.403323 30891 consensus_queue.cc:237] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.404578 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:45.408258 30536 leader_election.cc:343] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.409741 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.414548 30690 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:45.415832 30690 ts_tablet_manager.cc:1403] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.011s
I20260709 13:09:45.416627 30318 catalog_manager.cc:5697] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:45.422166 30535 leader_election.cc:343] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.422363 30696 raft_consensus.cc:493] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.422827 30535 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:45.423110 30696 raft_consensus.cc:515] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.423841 30695 tablet_bootstrap.cc:492] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:45.425706 30695 ts_tablet_manager.cc:1403] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.001s
I20260709 13:09:45.428494 30696 leader_election.cc:290] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.431097 30696 raft_consensus.cc:2749] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.431517 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf9aa835b48038aa80c194df56bd6" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:45.434295 30535 leader_election.cc:343] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.436147 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf9aa835b48038aa80c194df56bd6" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.438148 30692 ts_tablet_manager.cc:1434] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.003s
I20260709 13:09:45.439465 30692 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:45.442528 30691 tablet_bootstrap.cc:654] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.446099 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86637a5e467340ae9ac4f57d22c68bf6" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:45.446834 30642 raft_consensus.cc:3060] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.446729 30695 raft_consensus.cc:359] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.447481 30695 raft_consensus.cc:385] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.447805 30695 raft_consensus.cc:740] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.451357 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86637a5e467340ae9ac4f57d22c68bf6" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:45.452158 30416 raft_consensus.cc:3060] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.446364 30827 leader_election.cc:290] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.420617 30690 raft_consensus.cc:359] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.454116 30690 raft_consensus.cc:385] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.454454 30690 raft_consensus.cc:740] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.455111 30690 consensus_queue.cc:260] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.448913 30695 consensus_queue.cc:260] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.458098 30908 consensus_queue.cc:1048] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.461428 30695 ts_tablet_manager.cc:1434] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:09:45.463068 30695 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:45.468454 30690 ts_tablet_manager.cc:1434] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.000s
W20260709 13:09:45.468941 30536 leader_election.cc:343] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.469739 30690 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:45.474339 30536 leader_election.cc:304] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:45.475853 30891 raft_consensus.cc:2749] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.470376 30642 raft_consensus.cc:2468] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:45.478035 30461 leader_election.cc:304] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:45.478619 30695 tablet_bootstrap.cc:654] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.479000 30827 raft_consensus.cc:2804] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.479759 30827 raft_consensus.cc:697] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:45.480634 30827 consensus_queue.cc:237] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.471138 30692 tablet_bootstrap.cc:654] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.484854 30320 catalog_manager.cc:5697] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } }
I20260709 13:09:45.487398 30691 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.490983 30902 consensus_queue.cc:1048] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:45.491459 30691 ts_tablet_manager.cc:1403] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.097s	user 0.014s	sys 0.000s
I20260709 13:09:45.493762 30416 raft_consensus.cc:2468] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:45.495772 30691 raft_consensus.cc:359] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.496587 30691 raft_consensus.cc:385] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.496970 30691 raft_consensus.cc:740] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.515488 30691 consensus_queue.cc:260] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.516171 30692 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:45.516782 30692 ts_tablet_manager.cc:1403] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.000s
I20260709 13:09:45.518399 30691 ts_tablet_manager.cc:1434] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:09:45.520081 30691 tablet_bootstrap.cc:492] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.522130 30692 raft_consensus.cc:359] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.523048 30692 raft_consensus.cc:385] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.528896 30692 raft_consensus.cc:740] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.529778 30692 consensus_queue.cc:260] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.532433 30692 ts_tablet_manager.cc:1434] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:09:45.536499 30826 raft_consensus.cc:493] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.537288 30826 raft_consensus.cc:515] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.544356 30691 tablet_bootstrap.cc:654] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.544081 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a582514dd6e459fa48675d41a9ff078" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.545447 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a582514dd6e459fa48675d41a9ff078" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:45.547572 30461 leader_election.cc:343] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.548607 30826 leader_election.cc:290] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:45.545855 30692 tablet_bootstrap.cc:492] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
W20260709 13:09:45.550300 30463 leader_election.cc:343] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.553596 30463 leader_election.cc:304] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:45.555073 30827 raft_consensus.cc:2749] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.562352 30690 tablet_bootstrap.cc:654] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.542639 30316 catalog_manager.cc:5697] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.564864 30316 catalog_manager.cc:5697] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.569685 30695 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:45.570008 30692 tablet_bootstrap.cc:654] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.570365 30695 ts_tablet_manager.cc:1403] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.108s	user 0.022s	sys 0.003s
I20260709 13:09:45.573706 30695 raft_consensus.cc:359] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.574378 30695 raft_consensus.cc:385] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.574736 30695 raft_consensus.cc:740] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.576082 30695 consensus_queue.cc:260] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.578214 30695 ts_tablet_manager.cc:1434] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:45.580623 30692 tablet_bootstrap.cc:492] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:45.581339 30692 ts_tablet_manager.cc:1403] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.003s
I20260709 13:09:45.588898 30692 raft_consensus.cc:359] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.589883 30692 raft_consensus.cc:385] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.590205 30692 raft_consensus.cc:740] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.587010 30869 raft_consensus.cc:493] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.594048 30692 consensus_queue.cc:260] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.597040 30695 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:45.598089 30691 tablet_bootstrap.cc:492] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.598587 30691 ts_tablet_manager.cc:1403] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.004s
I20260709 13:09:45.595523 30869 raft_consensus.cc:515] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.601348 30690 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:45.601672 30869 leader_election.cc:290] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:45.601914 30690 ts_tablet_manager.cc:1403] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.132s	user 0.018s	sys 0.008s
I20260709 13:09:45.604442 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.605250 30691 raft_consensus.cc:359] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.606176 30691 raft_consensus.cc:385] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.606491 30691 raft_consensus.cc:740] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.606688 30491 raft_consensus.cc:2468] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:45.607226 30691 consensus_queue.cc:260] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.607522 30320 catalog_manager.cc:5697] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:45.610172 30691 ts_tablet_manager.cc:1434] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:09:45.611481 30691 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.605793 30692 ts_tablet_manager.cc:1434] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.005s
I20260709 13:09:45.620421 30826 raft_consensus.cc:493] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.614118 30690 raft_consensus.cc:359] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.621315 30690 raft_consensus.cc:385] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.621783 30690 raft_consensus.cc:740] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.621227 30318 catalog_manager.cc:5697] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.622457 30690 consensus_queue.cc:260] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.615417 30692 tablet_bootstrap.cc:492] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:45.618947 30388 leader_election.cc:304] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:45.610265 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:45.629122 30566 raft_consensus.cc:2468] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:45.629194 30869 raft_consensus.cc:2804] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.629966 30869 raft_consensus.cc:493] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.630653 30869 raft_consensus.cc:3060] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.631788 30692 tablet_bootstrap.cc:654] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.633431 30690 ts_tablet_manager.cc:1434] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:09:45.634531 30690 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:45.621222 30826 raft_consensus.cc:515] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.643414 30690 tablet_bootstrap.cc:654] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.643623 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.646122 30691 tablet_bootstrap.cc:654] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:45.646488 30461 leader_election.cc:343] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.646988 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.647698 30418 raft_consensus.cc:2468] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:45.650239 30462 leader_election.cc:304] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:45.651145 30827 raft_consensus.cc:2804] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.651522 30827 raft_consensus.cc:493] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.652047 30827 raft_consensus.cc:3060] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.663563 30902 raft_consensus.cc:493] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.664209 30902 raft_consensus.cc:515] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.666687 30902 leader_election.cc:290] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:45.671032 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:45.674517 30318 catalog_manager.cc:5697] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:45.616209 30695 tablet_bootstrap.cc:654] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.681820 30826 leader_election.cc:290] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.686507 30826 raft_consensus.cc:493] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.687068 30826 raft_consensus.cc:515] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.692811 30826 leader_election.cc:290] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:45.701941 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.703040 30491 raft_consensus.cc:2393] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:45.709503 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:45.716560 30463 leader_election.cc:343] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.718868 30692 tablet_bootstrap.cc:492] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:45.717749 30691 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:45.719574 30692 ts_tablet_manager.cc:1403] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.104s	user 0.011s	sys 0.013s
I20260709 13:09:45.719707 30691 ts_tablet_manager.cc:1403] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.109s	user 0.018s	sys 0.007s
I20260709 13:09:45.721065 30611 leader_election.cc:304] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:45.722146 30902 raft_consensus.cc:3060] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.729499 30908 raft_consensus.cc:493] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.729681 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.730096 30908 raft_consensus.cc:515] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:45.731479 30462 leader_election.cc:343] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.732009 30462 leader_election.cc:304] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:45.732470 30908 leader_election.cc:290] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.732879 30826 raft_consensus.cc:2749] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.734436 30916 consensus_queue.cc:1048] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.698997 30827 raft_consensus.cc:515] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.748802 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6ae4a76cb84d41a9dcc0c1cce9cba2" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.728628 30869 raft_consensus.cc:515] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.758522 30869 leader_election.cc:290] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:45.760087 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
W20260709 13:09:45.761778 30461 leader_election.cc:343] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:45.762924 30610 leader_election.cc:343] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.764770 30827 leader_election.cc:290] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.767282 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:45.768379 30416 raft_consensus.cc:3060] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.776918 30695 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:45.777782 30695 ts_tablet_manager.cc:1403] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.007s
I20260709 13:09:45.779553 30690 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:45.780452 30690 ts_tablet_manager.cc:1403] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.146s	user 0.023s	sys 0.041s
I20260709 13:09:45.781709 30695 raft_consensus.cc:359] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.782624 30695 raft_consensus.cc:385] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.782927 30695 raft_consensus.cc:740] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.783685 30695 consensus_queue.cc:260] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.785825 30695 ts_tablet_manager.cc:1434] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:09:45.784943 30690 raft_consensus.cc:359] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.786962 30690 raft_consensus.cc:385] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.787395 30690 raft_consensus.cc:740] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.801162 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:45.801966 30497 raft_consensus.cc:3060] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.812193 30690 consensus_queue.cc:260] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:45.810962 30691 raft_consensus.cc:359] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.815769 30691 raft_consensus.cc:385] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.816411 30691 raft_consensus.cc:740] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.818928 30691 consensus_queue.cc:260] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.819118 30692 raft_consensus.cc:359] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.819974 30692 raft_consensus.cc:385] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.820299 30692 raft_consensus.cc:740] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.822721 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:45.823468 30571 raft_consensus.cc:3060] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.824275 30816 consensus_queue.cc:1048] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:09:45.811940 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 76768.9us
W20260709 13:09:45.810950 30932 log.cc:927] Time spent T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:45.831074 30690 ts_tablet_manager.cc:1434] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:09:45.832793 30691 ts_tablet_manager.cc:1434] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.001s
I20260709 13:09:45.833323 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6ae4a76cb84d41a9dcc0c1cce9cba2" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:45.810779 30789 raft_consensus.cc:493] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.837754 30692 consensus_queue.cc:260] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.840369 30690 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:45.840626 30789 raft_consensus.cc:515] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.841912 30695 tablet_bootstrap.cc:492] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:45.853278 30692 ts_tablet_manager.cc:1434] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.133s	user 0.009s	sys 0.000s
I20260709 13:09:45.854841 30692 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:45.858716 30902 raft_consensus.cc:2749] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:45.860883 30695 tablet_bootstrap.cc:654] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.870579 30571 raft_consensus.cc:2468] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:45.873219 30902 consensus_queue.cc:1048] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.845569 30570 raft_consensus.cc:2468] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:45.885962 30691 tablet_bootstrap.cc:492] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:45.893867 30816 consensus_queue.cc:1048] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.898813 30692 tablet_bootstrap.cc:654] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.896944 30419 raft_consensus.cc:3060] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.902832 30611 leader_election.cc:304] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:45.905943 30970 raft_consensus.cc:2804] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.906512 30970 raft_consensus.cc:493] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.906867 30970 raft_consensus.cc:3060] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.911617 30416 raft_consensus.cc:2468] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:45.913476 30789 leader_election.cc:290] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:45.915349 30826 raft_consensus.cc:493] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.925140 30826 raft_consensus.cc:515] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:45.925508 30462 leader_election.cc:304] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:45.914390 30388 leader_election.cc:304] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:45.941531 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69a30e3a4a8473a9f1f8652ea9463d8" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:45.942682 30789 raft_consensus.cc:2804] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:09:45.946202 30461 leader_election.cc:343] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:45.946574 30789 raft_consensus.cc:697] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:45.947292 30695 tablet_bootstrap.cc:492] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:45.947805 30695 ts_tablet_manager.cc:1403] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.002s
I20260709 13:09:45.945073 30826 leader_election.cc:290] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:45.954394 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69a30e3a4a8473a9f1f8652ea9463d8" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:45.955713 30826 consensus_queue.cc:1048] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:45.957543 30695 raft_consensus.cc:359] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.958411 30695 raft_consensus.cc:385] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.958690 30695 raft_consensus.cc:740] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.961710 30696 consensus_queue.cc:1048] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:45.971295 30826 consensus_queue.cc:1048] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:45.947890 30789 consensus_queue.cc:237] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:45.981657 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:45.982527 30494 raft_consensus.cc:2393] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:45.932902 30973 raft_consensus.cc:2804] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.984598 30973 raft_consensus.cc:697] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:45.984676 30497 raft_consensus.cc:2468] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:45.986074 30695 consensus_queue.cc:260] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:45.995638 30695 ts_tablet_manager.cc:1434] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.048s	user 0.010s	sys 0.001s
I20260709 13:09:45.999358 30690 tablet_bootstrap.cc:654] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.001808 30908 raft_consensus.cc:493] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.007449 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac46eb166db448cb0c6f504819da525" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:46.016755 30462 leader_election.cc:343] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.016973 30816 raft_consensus.cc:493] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.039868 30826 raft_consensus.cc:493] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.029379 30908 raft_consensus.cc:515] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.040756 30973 consensus_queue.cc:237] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:46.046128 30826 raft_consensus.cc:515] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.079794 30826 leader_election.cc:290] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.066951 30908 leader_election.cc:290] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:46.045307 30816 raft_consensus.cc:515] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.068463 30388 leader_election.cc:304] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:46.083808 30816 leader_election.cc:290] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
W20260709 13:09:46.052654 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:46.082581 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:46.027647 30970 raft_consensus.cc:515] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.086108 30566 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:46.045221 30462 leader_election.cc:304] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
W20260709 13:09:46.087965 30462 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 74788.7us
I20260709 13:09:46.084219 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:46.049441 30695 tablet_bootstrap.cc:492] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:46.088100 30970 leader_election.cc:290] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:46.097513 30641 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:46.098482 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6ae4a76cb84d41a9dcc0c1cce9cba2" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:46.099361 30418 raft_consensus.cc:3060] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.102556 30463 leader_election.cc:304] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:46.104254 30827 raft_consensus.cc:2804] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.104799 30827 raft_consensus.cc:493] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.105257 30827 raft_consensus.cc:3060] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.173379 30691 tablet_bootstrap.cc:654] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.091045 30826 raft_consensus.cc:2749] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:46.199782 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a582514dd6e459fa48675d41a9ff078" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:46.206789 30908 raft_consensus.cc:493] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.223606 30985 raft_consensus.cc:493] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.253893 30908 raft_consensus.cc:515] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:46.284236 30867 raft_consensus.cc:2749] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:46.267089 30985 raft_consensus.cc:515] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.294052 30869 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955d2e588381448981f85fadf225aa05)
I20260709 13:09:46.279465 30857 consensus_queue.cc:1048] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
W20260709 13:09:46.293254 30462 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 132282us
I20260709 13:09:46.299060 30816 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.313791 30869 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.316048 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6ae4a76cb84d41a9dcc0c1cce9cba2" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:46.316699 30567 raft_consensus.cc:3060] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.317112 30827 raft_consensus.cc:515] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.319443 30827 leader_election.cc:290] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.312167 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a582514dd6e459fa48675d41a9ff078" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:46.320547 30497 raft_consensus.cc:2468] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:46.322851 30695 tablet_bootstrap.cc:654] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.305928 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:46.338306 30692 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:46.338759 30985 leader_election.cc:290] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.339108 30692 ts_tablet_manager.cc:1403] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.485s	user 0.048s	sys 0.076s
I20260709 13:09:46.340183 30571 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:46.325999 30816 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.343706 30816 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:46.344878 30690 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:46.345580 30690 ts_tablet_manager.cc:1403] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.513s	user 0.020s	sys 0.019s
I20260709 13:09:46.332018 30869 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.365813 30691 tablet_bootstrap.cc:492] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:46.366551 30691 ts_tablet_manager.cc:1403] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.521s	user 0.022s	sys 0.028s
I20260709 13:09:46.368954 30690 raft_consensus.cc:359] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.369954 30690 raft_consensus.cc:385] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.370287 30690 raft_consensus.cc:740] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.371733 30690 consensus_queue.cc:260] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.372788 30318 catalog_manager.cc:5697] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.384195 30690 ts_tablet_manager.cc:1434] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:09:46.378592 30537 leader_election.cc:304] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:46.370469 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:46.364760 30418 raft_consensus.cc:2468] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:46.391052 30692 raft_consensus.cc:359] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.395833 30692 raft_consensus.cc:385] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.396260 30692 raft_consensus.cc:740] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.397146 30692 consensus_queue.cc:260] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.393643 30891 raft_consensus.cc:2804] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.410770 30690 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:46.408772 30497 raft_consensus.cc:2393] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 19b1445cece541d3b2abe177283fef44 in this term.
I20260709 13:09:46.413493 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:46.414163 30641 raft_consensus.cc:3060] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.410610 30891 raft_consensus.cc:493] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.417603 30891 raft_consensus.cc:3060] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.387559 30611 leader_election.cc:304] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:46.395254 30908 leader_election.cc:290] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:46.418852 30692 ts_tablet_manager.cc:1434] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.079s	user 0.012s	sys 0.000s
W20260709 13:09:46.427421 30986 log.cc:927] Time spent T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Append to log took a long time: real 0.090s	user 0.003s	sys 0.001s
I20260709 13:09:46.418910 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:46.431015 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 53991.1us
I20260709 13:09:46.461179 30320 catalog_manager.cc:5697] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:46.463763 30567 raft_consensus.cc:2468] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:46.465943 30891 raft_consensus.cc:515] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.469509 30692 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:46.471588 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:46.496613 30691 raft_consensus.cc:359] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.497412 30691 raft_consensus.cc:385] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.497711 30691 raft_consensus.cc:740] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.500758 30691 consensus_queue.cc:260] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.510133 30643 raft_consensus.cc:2393] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 2: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:46.515043 30891 leader_election.cc:290] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.498648 30610 leader_election.cc:304] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
W20260709 13:09:46.517424 30535 leader_election.cc:343] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.518313 30902 raft_consensus.cc:2804] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.521133 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:46.523241 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a582514dd6e459fa48675d41a9ff078" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:46.528986 30690 tablet_bootstrap.cc:654] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.531036 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:46.531594 30902 raft_consensus.cc:697] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:46.532426 30491 raft_consensus.cc:3060] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.532968 30902 consensus_queue.cc:237] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.542474 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:46.542904 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:46.543130 30999 consensus_queue.cc:1048] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:46.543329 30419 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:46.546362 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:46.547399 30418 raft_consensus.cc:2393] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 19b1445cece541d3b2abe177283fef44 in this term.
I20260709 13:09:46.547708 30695 tablet_bootstrap.cc:492] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:46.543584 30567 raft_consensus.cc:3060] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.548352 30695 ts_tablet_manager.cc:1403] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.499s	user 0.013s	sys 0.027s
I20260709 13:09:46.551100 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:46.557230 30643 raft_consensus.cc:3060] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.560021 30695 raft_consensus.cc:359] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.560776 30695 raft_consensus.cc:385] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.561058 30695 raft_consensus.cc:740] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.561738 30695 consensus_queue.cc:260] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.563817 30695 ts_tablet_manager.cc:1434] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:09:46.552392 30645 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:46.564821 30695 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:46.574105 30691 ts_tablet_manager.cc:1434] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.207s	user 0.009s	sys 0.000s
I20260709 13:09:46.578647 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:46.579133 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a582514dd6e459fa48675d41a9ff078" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:46.579463 30570 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:09:46.579792 30644 raft_consensus.cc:3060] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.527482 30494 raft_consensus.cc:2468] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:46.585547 30691 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:46.594429 30536 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:46.596225 30985 raft_consensus.cc:2804] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.596602 30985 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.596910 30985 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.600848 30610 leader_election.cc:304] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:46.606070 30537 leader_election.cc:304] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:46.607369 30891 raft_consensus.cc:2804] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.607740 30891 raft_consensus.cc:493] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.608048 30891 raft_consensus.cc:3060] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.605201 30692 tablet_bootstrap.cc:654] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.619760 30816 raft_consensus.cc:493] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.620332 30816 raft_consensus.cc:515] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.622509 30816 leader_election.cc:290] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:46.623214 30641 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:46.623997 30908 raft_consensus.cc:2764] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:09:46.624611 30695 tablet_bootstrap.cc:654] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.626353 30388 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:46.627004 30826 consensus_queue.cc:1048] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:46.628511 30999 raft_consensus.cc:2804] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.628921 30999 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955d2e588381448981f85fadf225aa05)
I20260709 13:09:46.629331 30999 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:46.626933 30535 leader_election.cc:343] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.647228 30690 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:46.647814 30690 ts_tablet_manager.cc:1403] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.237s	user 0.013s	sys 0.009s
I20260709 13:09:46.647737 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:46.648631 30492 raft_consensus.cc:2468] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:46.657135 30916 raft_consensus.cc:2749] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:46.616503 30644 raft_consensus.cc:2468] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:46.659019 30996 consensus_queue.cc:1048] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:46.660960 30461 leader_election.cc:304] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:46.661957 30535 leader_election.cc:304] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:46.662613 30826 raft_consensus.cc:2804] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.662724 30696 raft_consensus.cc:2804] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.663182 30826 raft_consensus.cc:697] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:46.663237 30696 raft_consensus.cc:697] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:46.664155 30696 consensus_queue.cc:237] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.665931 30994 consensus_queue.cc:1048] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:46.620865 30491 raft_consensus.cc:2468] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:46.664155 30826 consensus_queue.cc:237] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.672056 30891 raft_consensus.cc:515] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.673578 30537 leader_election.cc:304] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:46.674571 30567 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:46.675704 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:46.672626 30985 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:46.684566 30536 leader_election.cc:343] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.685770 30891 leader_election.cc:290] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.690048 30691 tablet_bootstrap.cc:654] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.694253 30867 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.695502 30867 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.697297 30690 raft_consensus.cc:359] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.698022 30690 raft_consensus.cc:385] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.698266 30690 raft_consensus.cc:740] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.698443 30867 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.698894 30690 consensus_queue.cc:260] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.700148 30985 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:46.701625 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:46.701563 30690 ts_tablet_manager.cc:1434] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 13:09:46.701830 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:46.702725 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:46.692202 30816 raft_consensus.cc:2804] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.704726 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:46.704452 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:46.710995 30641 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.712018 30690 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:46.711661 30386 leader_election.cc:343] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.709262 30492 raft_consensus.cc:3060] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.708761 30816 raft_consensus.cc:493] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.720233 30816 raft_consensus.cc:3060] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.726186 30692 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:46.726627 30692 ts_tablet_manager.cc:1403] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.276s	user 0.014s	sys 0.009s
W20260709 13:09:46.748634 30535 leader_election.cc:343] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.750703 30999 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.751747 30419 raft_consensus.cc:3060] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.752347 30641 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:09:46.756778 30535 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:46.771368 30816 raft_consensus.cc:515] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.759155 30492 raft_consensus.cc:2468] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:46.761834 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:46.777361 30571 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:46.760742 30419 raft_consensus.cc:2468] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:09:46.854188 30537 leader_election.cc:304] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:46.855494 30691 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:46.856055 30691 ts_tablet_manager.cc:1403] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.271s	user 0.010s	sys 0.009s
I20260709 13:09:46.859333 30695 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:46.859926 30695 ts_tablet_manager.cc:1403] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.295s	user 0.006s	sys 0.012s
I20260709 13:09:46.862191 30999 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:46.875115 30690 tablet_bootstrap.cc:654] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.879400 30816 leader_election.cc:290] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:46.884281 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:46.885488 30641 raft_consensus.cc:2368] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 for earlier term 2. Current term is 3.
I20260709 13:09:46.890125 30691 raft_consensus.cc:359] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.891220 30691 raft_consensus.cc:385] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.891691 30691 raft_consensus.cc:740] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.892403 30691 consensus_queue.cc:260] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.894604 30691 ts_tablet_manager.cc:1434] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:09:46.895743 30691 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:46.892870 30690 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:46.898346 30690 ts_tablet_manager.cc:1403] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.187s	user 0.018s	sys 0.010s
I20260709 13:09:46.906196 30691 tablet_bootstrap.cc:654] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.908133 30690 raft_consensus.cc:359] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.908875 30690 raft_consensus.cc:385] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.909268 30690 raft_consensus.cc:740] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.910125 30690 consensus_queue.cc:260] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.918601 30691 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:46.919346 30691 ts_tablet_manager.cc:1403] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.009s
I20260709 13:09:46.921239 30690 ts_tablet_manager.cc:1434] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:09:46.922266 30690 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:46.922374 30691 raft_consensus.cc:359] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:46.923046 30691 raft_consensus.cc:385] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.923309 30691 raft_consensus.cc:740] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.923903 30691 consensus_queue.cc:260] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:46.925494 30691 ts_tablet_manager.cc:1434] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:09:46.926561 30691 tablet_bootstrap.cc:492] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:46.934552 30691 tablet_bootstrap.cc:654] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.940306 30692 raft_consensus.cc:359] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.941059 30386 leader_election.cc:400] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Vote denied by peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099) with higher term. Message: Invalid argument: T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 for earlier term 2. Current term is 3.
I20260709 13:09:46.941335 30692 raft_consensus.cc:385] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.948094 30692 raft_consensus.cc:740] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.948143 30816 raft_consensus.cc:2804] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.946090 30857 raft_consensus.cc:2804] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:46.948865 30816 raft_consensus.cc:697] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:46.949309 30857 raft_consensus.cc:697] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:46.949955 30816 consensus_queue.cc:237] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.950220 30857 consensus_queue.cc:237] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.953346 30692 consensus_queue.cc:260] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:46.956094 30692 ts_tablet_manager.cc:1434] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.229s	user 0.018s	sys 0.006s
W20260709 13:09:46.944267 30535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 187586us
I20260709 13:09:46.947587 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:46.947757 30695 raft_consensus.cc:359] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.957432 30695 raft_consensus.cc:385] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.957765 30695 raft_consensus.cc:740] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
W20260709 13:09:46.958439 30536 leader_election.cc:343] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:46.958474 30695 consensus_queue.cc:260] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:46.961513 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:46.965272 30989 consensus_queue.cc:1048] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:46.967289 30692 tablet_bootstrap.cc:492] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
W20260709 13:09:46.947435 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 93348.9us
I20260709 13:09:46.970218 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:46.971079 30493 raft_consensus.cc:3060] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.977891 30695 ts_tablet_manager.cc:1434] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.117s	user 0.011s	sys 0.008s
I20260709 13:09:46.979063 30695 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:46.980899 30691 tablet_bootstrap.cc:492] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:46.981448 30691 ts_tablet_manager.cc:1403] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.004s
I20260709 13:09:46.985373 30691 raft_consensus.cc:359] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.986099 30691 raft_consensus.cc:385] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.986374 30691 raft_consensus.cc:740] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.987022 30691 consensus_queue.cc:260] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:46.988945 30691 ts_tablet_manager.cc:1434] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:09:46.990082 30691 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:46.996362 30695 tablet_bootstrap.cc:654] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.997435 30388 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:46.998181 30691 tablet_bootstrap.cc:654] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.998405 30997 raft_consensus.cc:2804] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.998966 30997 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.976817 30690 tablet_bootstrap.cc:654] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.999320 30997 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.011523 30908 consensus_queue.cc:1048] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.016780 30386 leader_election.cc:403] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:09:47.017773 30867 raft_consensus.cc:2749] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099) with higher term. Message: Invalid argument: T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 for earlier term 2. Current term is 3.
W20260709 13:09:47.018561 30386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 124001us
I20260709 13:09:47.034184 30690 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:47.035220 30690 ts_tablet_manager.cc:1403] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.002s
I20260709 13:09:47.035957 30493 raft_consensus.cc:2468] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:47.039875 30908 raft_consensus.cc:493] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.040459 30908 raft_consensus.cc:515] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.041365 30695 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:47.042825 30908 leader_election.cc:290] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:47.044275 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf9aa835b48038aa80c194df56bd6" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:47.045058 30571 raft_consensus.cc:2468] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:47.044704 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf9aa835b48038aa80c194df56bd6" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:47.046221 30611 leader_election.cc:304] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:47.046274 30419 raft_consensus.cc:2468] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:47.050102 30908 raft_consensus.cc:2804] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:47.050794 30908 raft_consensus.cc:493] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.051512 30908 raft_consensus.cc:3060] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.940403 30317 catalog_manager.cc:5697] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.042083 30695 ts_tablet_manager.cc:1403] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.000s
W20260709 13:09:47.064845 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 67501.3us
I20260709 13:09:47.056471 30537 leader_election.cc:304] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:47.070061 30857 raft_consensus.cc:2804] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:47.070796 30857 raft_consensus.cc:697] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:47.071907 30857 consensus_queue.cc:237] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.077541 30997 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.080093 30997 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:47.081328 30691 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:47.081928 30691 ts_tablet_manager.cc:1403] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.092s	user 0.047s	sys 0.030s
I20260709 13:09:47.068791 30695 raft_consensus.cc:359] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.095765 30695 raft_consensus.cc:385] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.096278 30695 raft_consensus.cc:740] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.099184 30691 raft_consensus.cc:359] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.106801 30691 raft_consensus.cc:385] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.107156 30691 raft_consensus.cc:740] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.107390 30695 consensus_queue.cc:260] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.107827 30691 consensus_queue.cc:260] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.111116 30691 ts_tablet_manager.cc:1434] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 13:09:47.099195 30908 raft_consensus.cc:515] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.114432 30908 leader_election.cc:290] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:47.117851 30695 ts_tablet_manager.cc:1434] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:09:47.118121 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf9aa835b48038aa80c194df56bd6" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:47.118687 30691 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:47.119159 30695 tablet_bootstrap.cc:492] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:47.118858 30571 raft_consensus.cc:3060] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.123817 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf9aa835b48038aa80c194df56bd6" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:47.124547 30418 raft_consensus.cc:3060] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.125452 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:47.126139 30641 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.130580 30695 tablet_bootstrap.cc:654] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.134614 30857 consensus_queue.cc:1048] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.136127 30692 tablet_bootstrap.cc:654] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.151602 30857 consensus_queue.cc:1048] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:47.039373 30690 raft_consensus.cc:359] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.142572 30418 raft_consensus.cc:2468] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:47.153223 30690 raft_consensus.cc:385] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.153585 30690 raft_consensus.cc:740] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.154145 30610 leader_election.cc:304] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:47.154276 30690 consensus_queue.cc:260] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.155123 30896 raft_consensus.cc:2804] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:47.155629 30896 raft_consensus.cc:697] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:47.156543 30896 consensus_queue.cc:237] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.161623 30691 tablet_bootstrap.cc:654] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.180536 30690 ts_tablet_manager.cc:1434] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.145s	user 0.007s	sys 0.001s
I20260709 13:09:47.183248 30571 raft_consensus.cc:2468] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:47.213169 30641 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:47.214586 30972 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.217167 30972 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.224043 30972 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:47.232681 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:47.234483 30494 raft_consensus.cc:2468] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:47.257937 30388 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:47.261052 30972 raft_consensus.cc:2804] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:47.261502 30972 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.261845 30972 raft_consensus.cc:3060] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.265476 30318 catalog_manager.cc:5697] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.273185 30972 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.273733 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:47.276149 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:47.279812 30831 consensus_queue.cc:1048] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.234730 30386 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:47.294492 30999 raft_consensus.cc:2804] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:47.295074 30999 raft_consensus.cc:697] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:47.295971 30999 consensus_queue.cc:237] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.288323 30696 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.291886 30567 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.306555 30691 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:47.306756 30695 tablet_bootstrap.cc:492] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:47.307308 30691 ts_tablet_manager.cc:1403] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.189s	user 0.006s	sys 0.007s
I20260709 13:09:47.307466 30695 ts_tablet_manager.cc:1403] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.189s	user 0.032s	sys 0.043s
W20260709 13:09:47.293524 30386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 58939.6us
I20260709 13:09:47.299410 30696 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.335359 30696 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:47.342083 30571 raft_consensus.cc:2468] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:47.343426 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:47.344203 30492 raft_consensus.cc:2468] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:47.345148 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:47.344600 30316 catalog_manager.cc:5697] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.345561 30826 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.346017 30418 raft_consensus.cc:2393] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:47.346076 30826 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.346890 30537 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:47.348449 30826 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:47.349135 30985 raft_consensus.cc:2804] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:47.350653 30985 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.351029 30985 raft_consensus.cc:3060] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.335619 30691 raft_consensus.cc:359] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.358125 30691 raft_consensus.cc:385] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.358551 30691 raft_consensus.cc:740] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.359138 30690 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:47.361799 30695 raft_consensus.cc:359] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.363036 30695 raft_consensus.cc:385] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.363507 30695 raft_consensus.cc:740] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.364347 30691 consensus_queue.cc:260] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.368911 30695 consensus_queue.cc:260] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
W20260709 13:09:47.376518 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:47.361720 30567 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:47.381419 30691 ts_tablet_manager.cc:1434] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.002s
I20260709 13:09:47.383037 30695 ts_tablet_manager.cc:1434] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 13:09:47.384249 30695 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:47.398124 30826 raft_consensus.cc:493] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.398763 30826 raft_consensus.cc:515] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.405409 30691 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:47.390777 30994 consensus_queue.cc:1048] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:09:47.421025 30691 tablet_bootstrap.cc:654] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.426653 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:47.428176 30831 raft_consensus.cc:493] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.430085 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:47.433390 30973 raft_consensus.cc:493] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.433975 30973 raft_consensus.cc:515] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.436640 30972 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:09:47.439008 30462 leader_election.cc:343] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.437039 30973 leader_election.cc:290] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:47.428915 30831 raft_consensus.cc:515] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.449124 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "040a6253e94b4c19952ed01cb9d8c6a5" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:47.456188 30463 leader_election.cc:343] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.456979 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "366b9fff4017464a9b99ab4878dbb119" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:47.459754 30417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "040a6253e94b4c19952ed01cb9d8c6a5" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:47.462098 30462 leader_election.cc:343] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.462206 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:47.462914 30462 leader_election.cc:343] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.463351 30462 leader_election.cc:304] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:47.447603 30831 leader_election.cc:290] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:47.448042 30695 tablet_bootstrap.cc:654] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.467743 30690 tablet_bootstrap.cc:654] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.471535 30692 tablet_bootstrap.cc:492] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:47.439666 30972 raft_consensus.cc:493] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.472388 30692 ts_tablet_manager.cc:1403] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.515s	user 0.012s	sys 0.017s
I20260709 13:09:47.458527 30826 leader_election.cc:290] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:47.471525 30318 catalog_manager.cc:5697] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.476545 30972 raft_consensus.cc:515] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.472391 30973 raft_consensus.cc:2749] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:47.491657 30417 raft_consensus.cc:1275] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:09:47.469151 30461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 98838.6us
I20260709 13:09:47.494038 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:47.495396 30891 consensus_queue.cc:1048] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:47.495625 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:47.500757 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "366b9fff4017464a9b99ab4878dbb119" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:47.502269 30463 leader_election.cc:343] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.486953 30972 leader_election.cc:290] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:47.488860 30641 raft_consensus.cc:1275] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:47.519654 30317 catalog_manager.cc:5697] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:47.522387 30317 catalog_manager.cc:5697] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.525278 30317 catalog_manager.cc:5697] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.527472 30816 consensus_queue.cc:1048] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.514863 30696 consensus_queue.cc:1048] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.538580 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:47.539404 30494 raft_consensus.cc:3060] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:47.540489 30461 leader_election.cc:343] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.540817 30692 raft_consensus.cc:359] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.541601 30692 raft_consensus.cc:385] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.541864 30692 raft_consensus.cc:740] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.542662 30691 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:47.542520 30692 consensus_queue.cc:260] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.543249 30691 ts_tablet_manager.cc:1403] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.138s	user 0.019s	sys 0.010s
I20260709 13:09:47.545337 30692 ts_tablet_manager.cc:1434] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.002s
I20260709 13:09:47.545156 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:47.546563 30692 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:47.547710 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c678eac1173e4cdaa75a8279bc764f29" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:47.548635 30497 raft_consensus.cc:2393] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:47.563242 30985 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.565408 30567 raft_consensus.cc:2393] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:47.569411 30461 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
W20260709 13:09:47.571156 30461 leader_election.cc:343] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.571585 30461 leader_election.cc:304] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:47.573896 30691 raft_consensus.cc:359] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.574703 30691 raft_consensus.cc:385] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.575039 30691 raft_consensus.cc:740] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.576320 30691 consensus_queue.cc:260] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.577639 30994 raft_consensus.cc:2749] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:47.580794 30973 raft_consensus.cc:2749] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:47.596894 30690 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:47.597626 30690 ts_tablet_manager.cc:1403] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.239s	user 0.017s	sys 0.021s
I20260709 13:09:47.600154 30691 ts_tablet_manager.cc:1434] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
W20260709 13:09:47.598883 30461 leader_election.cc:343] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.587553 30494 raft_consensus.cc:2468] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:47.612402 30985 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:47.613588 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:47.614498 30419 raft_consensus.cc:2393] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:47.612854 30644 raft_consensus.cc:3060] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.622732 30691 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:47.630076 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:47.635655 30494 raft_consensus.cc:2393] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:47.633705 30891 consensus_queue.cc:1048] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.637333 30692 tablet_bootstrap.cc:654] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.607872 30461 leader_election.cc:304] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:47.640197 30537 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:47.641891 30826 raft_consensus.cc:2749] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:47.642719 30891 raft_consensus.cc:2749] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:09:47.642967 30461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 50563.2us
I20260709 13:09:47.639135 30316 catalog_manager.cc:5697] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:47.663095 30690 raft_consensus.cc:359] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.665194 30690 raft_consensus.cc:385] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.666564 30690 raft_consensus.cc:740] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.664062 30695 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:47.667982 30695 ts_tablet_manager.cc:1403] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.284s	user 0.022s	sys 0.007s
I20260709 13:09:47.668604 30388 leader_election.cc:304] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:47.669961 30690 consensus_queue.cc:260] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.674670 30972 raft_consensus.cc:3060] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.684963 30989 consensus_queue.cc:1048] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.694746 30972 raft_consensus.cc:2749] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:47.707571 30388 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:47.709046 31000 raft_consensus.cc:2804] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:47.709513 31000 raft_consensus.cc:697] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:47.709776 30695 raft_consensus.cc:359] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.710511 30695 raft_consensus.cc:385] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.710291 31000 consensus_queue.cc:237] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:47.716373 30989 raft_consensus.cc:493] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.717078 30989 raft_consensus.cc:515] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.719439 30989 leader_election.cc:290] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:47.710785 30695 raft_consensus.cc:740] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.721928 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:47.727948 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:47.729105 30690 ts_tablet_manager.cc:1434] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.130s	user 0.008s	sys 0.001s
I20260709 13:09:47.722697 30695 consensus_queue.cc:260] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.734119 30690 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:47.746642 30695 ts_tablet_manager.cc:1434] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260709 13:09:47.758244 30695 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:47.758427 30611 leader_election.cc:304] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:47.767632 30896 raft_consensus.cc:2749] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:47.769057 30997 consensus_queue.cc:1048] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:09:47.780525 30696 consensus_queue.cc:1048] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:47.781746 30891 consensus_queue.cc:1048] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:47.782950 30896 consensus_queue.cc:1048] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:47.793484 30695 tablet_bootstrap.cc:654] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.798070 30690 tablet_bootstrap.cc:654] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.807122 30692 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:47.808974 30692 ts_tablet_manager.cc:1403] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.263s	user 0.021s	sys 0.018s
I20260709 13:09:47.830081 30994 raft_consensus.cc:493] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.845525 30891 raft_consensus.cc:493] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.846112 30994 raft_consensus.cc:515] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.873875 30695 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:47.874461 30695 ts_tablet_manager.cc:1403] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.116s	user 0.022s	sys 0.009s
I20260709 13:09:47.878118 30691 tablet_bootstrap.cc:654] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.880059 30891 raft_consensus.cc:515] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.886059 30891 leader_election.cc:290] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:47.887986 30495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:47.888717 30495 raft_consensus.cc:2468] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:47.892329 30537 leader_election.cc:304] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:47.893664 30816 raft_consensus.cc:2804] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:47.889675 30972 consensus_queue.cc:1048] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:09:47.894119 30816 raft_consensus.cc:493] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.895257 30816 raft_consensus.cc:3060] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.918069 30318 catalog_manager.cc:5697] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.928010 30692 raft_consensus.cc:359] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.928874 30692 raft_consensus.cc:385] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.929172 30692 raft_consensus.cc:740] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.929883 30692 consensus_queue.cc:260] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:47.932574 30692 ts_tablet_manager.cc:1434] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.122s	user 0.006s	sys 0.000s
I20260709 13:09:47.932668 30994 leader_election.cc:290] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:47.936858 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69a30e3a4a8473a9f1f8652ea9463d8" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:47.938400 30461 leader_election.cc:343] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.940753 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69a30e3a4a8473a9f1f8652ea9463d8" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:47.941470 30419 raft_consensus.cc:2468] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 0.
I20260709 13:09:47.948077 30462 leader_election.cc:304] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:47.949223 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:47.950162 30831 raft_consensus.cc:2804] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:47.950832 30831 raft_consensus.cc:493] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.951370 30831 raft_consensus.cc:3060] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.952334 30816 raft_consensus.cc:515] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.954859 30816 leader_election.cc:290] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:47.958321 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:47.959017 30494 raft_consensus.cc:3060] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.965090 30692 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
W20260709 13:09:47.968243 30535 leader_election.cc:343] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:47.975142 30494 raft_consensus.cc:2468] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:47.980806 30537 leader_election.cc:304] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:47.981918 30816 raft_consensus.cc:2804] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:47.982640 30816 raft_consensus.cc:697] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:47.983458 30816 consensus_queue.cc:237] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:47.988847 30691 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:47.989559 30691 ts_tablet_manager.cc:1403] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.367s	user 0.038s	sys 0.033s
I20260709 13:09:47.992362 30695 raft_consensus.cc:359] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.993173 30695 raft_consensus.cc:385] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.983551 30831 raft_consensus.cc:515] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.993489 30695 raft_consensus.cc:740] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.994140 30695 consensus_queue.cc:260] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.996168 30831 leader_election.cc:290] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:47.992688 30691 raft_consensus.cc:359] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.999498 30691 raft_consensus.cc:385] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.999752 30692 tablet_bootstrap.cc:654] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.999907 30691 raft_consensus.cc:740] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.000838 30691 consensus_queue.cc:260] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:47.998242 30417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69a30e3a4a8473a9f1f8652ea9463d8" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:48.004199 30417 raft_consensus.cc:3060] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.997234 30973 raft_consensus.cc:493] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.005502 30973 raft_consensus.cc:515] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.006621 30691 ts_tablet_manager.cc:1434] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:09:48.011936 30691 tablet_bootstrap.cc:492] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:47.996986 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69a30e3a4a8473a9f1f8652ea9463d8" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:48.013022 30644 raft_consensus.cc:3060] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.018736 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:48.010725 30973 leader_election.cc:290] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:48.019985 30695 ts_tablet_manager.cc:1434] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.000s
I20260709 13:09:48.031777 30690 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:48.037786 30690 ts_tablet_manager.cc:1403] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.307s	user 0.019s	sys 0.009s
I20260709 13:09:48.029838 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:48.039584 30695 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:48.026995 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:48.026227 30463 leader_election.cc:343] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:48.043407 30462 leader_election.cc:343] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.045710 30462 leader_election.cc:304] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
W20260709 13:09:48.046597 30535 leader_election.cc:343] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.048055 30695 tablet_bootstrap.cc:654] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.044574 30690 raft_consensus.cc:359] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.048947 30690 raft_consensus.cc:385] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.049270 30690 raft_consensus.cc:740] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.050509 30831 raft_consensus.cc:2749] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.065275 30690 consensus_queue.cc:260] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.072722 30690 ts_tablet_manager.cc:1434] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.001s
I20260709 13:09:48.075042 30690 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:48.079670 30695 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:48.080484 30695 ts_tablet_manager.cc:1403] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.008s
I20260709 13:09:48.081991 30691 tablet_bootstrap.cc:654] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.094036 30690 tablet_bootstrap.cc:654] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.103760 30972 consensus_queue.cc:1048] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:48.112413 30417 raft_consensus.cc:2468] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:48.122797 30462 leader_election.cc:304] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:48.124039 30831 raft_consensus.cc:2804] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:48.124603 30831 raft_consensus.cc:697] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:48.125766 30831 consensus_queue.cc:237] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.118166 30692 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:48.130369 30692 ts_tablet_manager.cc:1403] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.166s	user 0.017s	sys 0.001s
I20260709 13:09:48.133980 30695 raft_consensus.cc:359] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.135012 30896 raft_consensus.cc:493] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.136353 30896 raft_consensus.cc:515] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.140504 31000 consensus_queue.cc:1048] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:48.135496 30695 raft_consensus.cc:385] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.142244 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e52d69d8f9044099d83595faf8a7c88" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:48.142496 30695 raft_consensus.cc:740] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.142966 30567 raft_consensus.cc:2468] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:48.143215 30695 consensus_queue.cc:260] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.145104 30417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e52d69d8f9044099d83595faf8a7c88" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:48.146051 30994 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.152535 30994 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.158560 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:48.159237 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:48.160473 30462 leader_election.cc:343] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.161254 30994 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:48.146986 30896 leader_election.cc:290] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:48.148635 30611 leader_election.cc:304] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:48.162930 30896 raft_consensus.cc:2804] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.163398 30896 raft_consensus.cc:493] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.163828 30896 raft_consensus.cc:3060] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:48.165616 30461 leader_election.cc:343] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.166903 30461 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:48.173383 30831 raft_consensus.cc:2749] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.175928 30695 ts_tablet_manager.cc:1434] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.004s
W20260709 13:09:48.176193 30610 leader_election.cc:343] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.177820 30695 tablet_bootstrap.cc:492] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:48.177870 30644 raft_consensus.cc:2468] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:48.183586 30318 catalog_manager.cc:5697] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:48.190259 30896 raft_consensus.cc:515] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.183929 30691 tablet_bootstrap.cc:492] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:48.207145 30691 ts_tablet_manager.cc:1403] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.196s	user 0.022s	sys 0.021s
I20260709 13:09:48.197957 30695 tablet_bootstrap.cc:654] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.200920 30317 catalog_manager.cc:5697] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:48.140275 30692 raft_consensus.cc:359] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.209990 30692 raft_consensus.cc:385] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.210350 30692 raft_consensus.cc:740] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.210431 30691 raft_consensus.cc:359] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.211112 30692 consensus_queue.cc:260] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.211644 30691 raft_consensus.cc:385] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.212846 30691 raft_consensus.cc:740] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.214365 30691 consensus_queue.cc:260] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.216624 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e52d69d8f9044099d83595faf8a7c88" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:48.226603 30567 raft_consensus.cc:3060] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.217609 30691 ts_tablet_manager.cc:1434] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:09:48.234786 30567 raft_consensus.cc:2468] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:48.219193 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e52d69d8f9044099d83595faf8a7c88" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:48.215443 30896 leader_election.cc:290] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:48.238289 30610 leader_election.cc:343] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.238401 30691 tablet_bootstrap.cc:492] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:48.240835 30611 leader_election.cc:304] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:48.220726 30690 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:48.244302 30690 ts_tablet_manager.cc:1403] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.169s	user 0.013s	sys 0.003s
I20260709 13:09:48.244439 30896 raft_consensus.cc:2804] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:48.244961 30896 raft_consensus.cc:697] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:48.246392 30896 consensus_queue.cc:237] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.246191 30691 tablet_bootstrap.cc:654] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.265666 30691 tablet_bootstrap.cc:492] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:48.266156 30691 ts_tablet_manager.cc:1403] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.000s
I20260709 13:09:48.266516 30692 ts_tablet_manager.cc:1434] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.135s	user 0.004s	sys 0.004s
I20260709 13:09:48.272835 30692 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:48.271508 30691 raft_consensus.cc:359] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.293709 30691 raft_consensus.cc:385] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.294265 30691 raft_consensus.cc:740] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.296041 30691 consensus_queue.cc:260] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.284247 30695 tablet_bootstrap.cc:492] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:48.288414 30690 raft_consensus.cc:359] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.299338 30690 raft_consensus.cc:385] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.299726 30690 raft_consensus.cc:740] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.304139 30695 ts_tablet_manager.cc:1403] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.122s	user 0.022s	sys 0.011s
I20260709 13:09:48.305719 30691 ts_tablet_manager.cc:1434] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.003s
I20260709 13:09:48.306867 30691 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:48.309988 30690 consensus_queue.cc:260] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.328682 30695 raft_consensus.cc:359] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.329663 30695 raft_consensus.cc:385] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.329982 30695 raft_consensus.cc:740] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.330758 30695 consensus_queue.cc:260] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.340006 30692 tablet_bootstrap.cc:654] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.346386 30695 ts_tablet_manager.cc:1434] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.042s	user 0.003s	sys 0.004s
I20260709 13:09:48.347862 30690 ts_tablet_manager.cc:1434] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.103s	user 0.008s	sys 0.000s
I20260709 13:09:48.348531 30695 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:48.348941 30690 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:48.360735 30695 tablet_bootstrap.cc:654] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.382658 30691 tablet_bootstrap.cc:654] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.390719 30690 tablet_bootstrap.cc:654] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.408878 30695 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:48.410830 30695 ts_tablet_manager.cc:1403] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.007s
I20260709 13:09:48.416730 30826 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.417680 30826 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.420641 30826 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:48.434706 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:48.434285 30695 raft_consensus.cc:359] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.457752 30644 raft_consensus.cc:3060] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.442427 30695 raft_consensus.cc:385] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.460846 30695 raft_consensus.cc:740] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
W20260709 13:09:48.466631 30461 leader_election.cc:343] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.471011 30318 catalog_manager.cc:5697] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:48.478055 30896 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.478855 30896 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.482589 30972 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.483243 30896 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:48.483194 30972 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.466372 30695 consensus_queue.cc:260] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.496922 30695 ts_tablet_manager.cc:1434] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.086s	user 0.013s	sys 0.000s
I20260709 13:09:48.498262 30695 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:48.535702 30695 tablet_bootstrap.cc:654] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.521858 31054 raft_consensus.cc:493] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.538669 31054 raft_consensus.cc:515] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.543444 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:48.545503 30496 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:48.555689 30690 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:48.556470 30690 ts_tablet_manager.cc:1403] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.208s	user 0.021s	sys 0.043s
I20260709 13:09:48.563274 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:48.563321 30696 consensus_queue.cc:1048] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:48.566020 30611 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:48.569967 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:48.571498 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:48.534233 30972 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:48.576376 31054 leader_election.cc:290] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:48.583768 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c56097e8594d298daed39079975dd1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:48.584172 30816 raft_consensus.cc:493] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.591934 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c56097e8594d298daed39079975dd1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:48.598706 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:48.598927 31041 consensus_queue.cc:1048] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:48.600332 30690 raft_consensus.cc:359] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.602288 30997 raft_consensus.cc:493] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:48.618307 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:48.633298 30645 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:48.656643 30690 raft_consensus.cc:385] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.673206 30690 raft_consensus.cc:740] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.674317 30690 consensus_queue.cc:260] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.674970 30691 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:48.675904 30691 ts_tablet_manager.cc:1403] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.369s	user 0.028s	sys 0.029s
I20260709 13:09:48.680540 30690 ts_tablet_manager.cc:1434] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.124s	user 0.007s	sys 0.003s
I20260709 13:09:48.638639 30816 raft_consensus.cc:515] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.656550 30494 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:48.698007 30690 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:48.656530 30418 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
W20260709 13:09:48.647231 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 81301.9us
I20260709 13:09:48.715602 30816 leader_election.cc:290] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:48.685343 30896 raft_consensus.cc:2804] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.719367 30691 raft_consensus.cc:359] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.721164 30691 raft_consensus.cc:385] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.721480 30691 raft_consensus.cc:740] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.722543 30388 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
W20260709 13:09:48.694746 30386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 99667.8us
I20260709 13:09:48.725551 30691 consensus_queue.cc:260] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:48.686543 30997 raft_consensus.cc:515] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:48.740083 30691 ts_tablet_manager.cc:1434] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.002s
I20260709 13:09:48.737267 31080 raft_consensus.cc:2804] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.725528 30989 raft_consensus.cc:493] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.748705 31080 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.771083 30695 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:48.772343 30695 ts_tablet_manager.cc:1403] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.274s	user 0.010s	sys 0.024s
I20260709 13:09:48.776585 30691 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:48.757906 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "040a6253e94b4c19952ed01cb9d8c6a5" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:48.750545 30989 raft_consensus.cc:515] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:48.810977 30692 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:48.811589 30692 ts_tablet_manager.cc:1403] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.539s	user 0.051s	sys 0.042s
I20260709 13:09:48.774495 31080 raft_consensus.cc:3060] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:48.796751 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 70442.7us
I20260709 13:09:48.807582 30494 raft_consensus.cc:2468] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:48.833585 30417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "040a6253e94b4c19952ed01cb9d8c6a5" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
W20260709 13:09:48.816867 30463 leader_election.cc:343] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.877530 30569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e52d69d8f9044099d83595faf8a7c88" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:48.856910 30417 raft_consensus.cc:2468] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:48.879060 30569 raft_consensus.cc:2393] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:48.857121 30463 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:48.897895 30997 leader_election.cc:290] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:48.919654 30857 consensus_queue.cc:1048] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:09:48.912706 30826 raft_consensus.cc:2749] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:48.912055 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 57088us
I20260709 13:09:48.915918 30831 raft_consensus.cc:493] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.939095 30691 tablet_bootstrap.cc:654] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.939100 30690 tablet_bootstrap.cc:654] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.958232 30695 raft_consensus.cc:359] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:48.959576 30695 raft_consensus.cc:385] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.960227 30695 raft_consensus.cc:740] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.961997 30695 consensus_queue.cc:260] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:48.918373 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 106448us
I20260709 13:09:48.917953 30537 leader_election.cc:304] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:49.008498 30690 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:49.017344 30690 ts_tablet_manager.cc:1403] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.320s	user 0.013s	sys 0.024s
I20260709 13:09:49.024695 30695 ts_tablet_manager.cc:1434] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.246s	user 0.011s	sys 0.001s
I20260709 13:09:49.024674 30692 raft_consensus.cc:359] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:49.025763 30692 raft_consensus.cc:385] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.026078 30692 raft_consensus.cc:740] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.996845 31084 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.036064 30690 raft_consensus.cc:359] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.038218 30690 raft_consensus.cc:385] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.038532 30690 raft_consensus.cc:740] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
W20260709 13:09:48.999918 30463 leader_election.cc:343] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:49.000783 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 89337us
I20260709 13:09:48.962304 30831 raft_consensus.cc:515] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.047647 30690 consensus_queue.cc:260] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.039441 31084 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:49.044747 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 46654.3us
I20260709 13:09:49.039444 31083 raft_consensus.cc:2804] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:49.067471 31083 raft_consensus.cc:493] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.068228 31083 raft_consensus.cc:3060] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.078641 30831 leader_election.cc:290] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:49.091320 31084 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:49.114270 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e52d69d8f9044099d83595faf8a7c88" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:49.117744 31080 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.120340 30896 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.127187 30896 raft_consensus.cc:3060] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.131568 30692 consensus_queue.cc:260] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:49.137506 30989 leader_election.cc:290] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:49.157699 31080 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:49.158622 30386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 81623.9us
I20260709 13:09:49.163318 30691 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:49.164101 30691 ts_tablet_manager.cc:1403] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.399s	user 0.020s	sys 0.026s
I20260709 13:09:49.171146 30690 ts_tablet_manager.cc:1434] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.153s	user 0.011s	sys 0.001s
I20260709 13:09:49.174134 30816 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.174896 30816 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:49.178187 30692 ts_tablet_manager.cc:1434] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.366s	user 0.012s	sys 0.002s
W20260709 13:09:49.172529 30461 leader_election.cc:343] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.186918 30461 leader_election.cc:304] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:49.186594 30896 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.189158 30816 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:49.190649 31054 raft_consensus.cc:2749] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.191294 30896 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:49.198421 30691 raft_consensus.cc:359] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.200575 30691 raft_consensus.cc:385] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.200913 30691 raft_consensus.cc:740] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
W20260709 13:09:49.209087 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 165118us
I20260709 13:09:49.206560 30691 consensus_queue.cc:260] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:49.201244 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 146197us
I20260709 13:09:49.218531 30691 ts_tablet_manager.cc:1434] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.054s	user 0.010s	sys 0.003s
I20260709 13:09:49.223035 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:49.229756 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:49.217319 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:49.222860 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:49.225575 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 96210.8us
I20260709 13:09:49.230170 30497 raft_consensus.cc:3060] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.243256 30695 tablet_bootstrap.cc:492] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:49.225540 30896 consensus_queue.cc:1048] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:09:49.277700 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:49.278535 30566 raft_consensus.cc:2468] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:49.280422 30497 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:49.291220 30642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:49.292157 30642 raft_consensus.cc:2393] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:49.294998 30565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:49.273983 31104 consensus_queue.cc:1048] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:09:49.276038 31083 raft_consensus.cc:515] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.302942 31083 leader_election.cc:290] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:49.304190 30462 leader_election.cc:343] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.304179 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "040a6253e94b4c19952ed01cb9d8c6a5" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:49.310974 30497 raft_consensus.cc:3060] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.311522 30418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:49.312716 30416 raft_consensus.cc:3060] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.313546 30422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:49.317483 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:49.339781 30896 raft_consensus.cc:493] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.341761 30691 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:49.344647 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:49.345713 30690 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:49.346544 30692 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:49.359163 30695 tablet_bootstrap.cc:654] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:49.358219 30611 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:49.369658 30611 leader_election.cc:304] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:49.374576 30611 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:49.364910 30896 raft_consensus.cc:515] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.393610 31102 raft_consensus.cc:2749] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:49.396064 30896 leader_election.cc:290] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:49.385565 30386 leader_election.cc:304] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:49.398979 31101 raft_consensus.cc:2804] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:49.370419 30417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "040a6253e94b4c19952ed01cb9d8c6a5" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:49.403970 31101 raft_consensus.cc:697] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:49.404328 30417 raft_consensus.cc:3060] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.405252 31101 consensus_queue.cc:237] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.387916 30420 raft_consensus.cc:2393] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:49.419555 30692 tablet_bootstrap.cc:654] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:49.429003 30386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 55529.6us
W20260709 13:09:49.431257 30536 leader_election.cc:343] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:49.429420 30610 leader_election.cc:343] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.429138 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "366b9fff4017464a9b99ab4878dbb119" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:49.435240 30420 raft_consensus.cc:2468] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:49.436621 30690 tablet_bootstrap.cc:654] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:49.437312 30571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:49.442865 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "366b9fff4017464a9b99ab4878dbb119" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:49.443684 30491 raft_consensus.cc:2468] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:49.453409 30386 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:49.368108 30636 raft_consensus.cc:2468] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:49.456408 30972 raft_consensus.cc:2749] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.456904 31087 raft_consensus.cc:2804] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:49.465749 31087 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.468501 31087 raft_consensus.cc:3060] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.470427 30826 raft_consensus.cc:493] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.472059 30826 raft_consensus.cc:515] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:49.480067 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 52210.5us
I20260709 13:09:49.487761 30535 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:49.497864 30611 leader_election.cc:304] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:49.506335 30891 raft_consensus.cc:2804] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:09:49.496076 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 72132.3us
I20260709 13:09:49.531502 30891 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.559929 30691 tablet_bootstrap.cc:654] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:49.564529 30497 raft_consensus.cc:2468] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:49.572335 30826 leader_election.cc:290] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:49.593770 30463 leader_election.cc:343] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.595054 30463 leader_election.cc:304] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:49.592335 30537 leader_election.cc:304] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:49.552217 31098 raft_consensus.cc:2804] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:49.606088 31098 raft_consensus.cc:493] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.606600 31098 raft_consensus.cc:3060] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.607877 30690 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:49.608480 30690 ts_tablet_manager.cc:1403] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.263s	user 0.012s	sys 0.006s
I20260709 13:09:49.564924 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:49.606137 30417 raft_consensus.cc:2468] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:49.574924 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:49.568624 30891 raft_consensus.cc:3060] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.616413 30690 raft_consensus.cc:359] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.624701 30690 raft_consensus.cc:385] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.627215 30690 raft_consensus.cc:740] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.628506 30690 consensus_queue.cc:260] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.629165 31041 raft_consensus.cc:2749] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:49.639811 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 47762us
W20260709 13:09:49.639583 30461 leader_election.cc:343] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:49.641232 30462 leader_election.cc:343] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.641433 31082 raft_consensus.cc:2804] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:49.642009 31082 raft_consensus.cc:697] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:49.643049 31082 consensus_queue.cc:237] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.641819 30462 leader_election.cc:304] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:49.649672 30692 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:49.650799 30692 ts_tablet_manager.cc:1403] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.304s	user 0.016s	sys 0.007s
I20260709 13:09:49.695897 30690 ts_tablet_manager.cc:1434] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.087s	user 0.010s	sys 0.002s
I20260709 13:09:49.660164 30831 raft_consensus.cc:2749] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:49.674691 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 75287.9us
I20260709 13:09:49.765861 31000 raft_consensus.cc:2749] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.764498 30891 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:49.802374 30690 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:49.801158 30891 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:49.843119 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:49.844460 30695 tablet_bootstrap.cc:492] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:49.845731 30695 ts_tablet_manager.cc:1403] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.603s	user 0.033s	sys 0.032s
I20260709 13:09:49.856952 31087 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:49.857748 30896 consensus_queue.cc:1048] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:49.861779 30690 tablet_bootstrap.cc:654] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:49.862630 31087 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:49.865506 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:49.873646 31088 raft_consensus.cc:493] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.875528 31088 raft_consensus.cc:515] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.888669 30695 raft_consensus.cc:359] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:49.893138 30695 raft_consensus.cc:385] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.894042 30695 raft_consensus.cc:740] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.860981 30692 raft_consensus.cc:359] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.895033 30692 raft_consensus.cc:385] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.895388 30692 raft_consensus.cc:740] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.898342 30692 consensus_queue.cc:260] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.902312 30695 consensus_queue.cc:260] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:49.905387 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:49.906229 30493 raft_consensus.cc:2468] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:49.908468 30695 ts_tablet_manager.cc:1434] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.002s
I20260709 13:09:49.918051 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:49.918964 30636 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:49.920549 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:49.921586 30567 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:49.922459 30691 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:49.923137 30691 ts_tablet_manager.cc:1403] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.654s	user 0.028s	sys 0.013s
I20260709 13:09:49.924515 30388 leader_election.cc:304] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:49.942866 30692 ts_tablet_manager.cc:1434] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.291s	user 0.009s	sys 0.003s
I20260709 13:09:49.947567 30692 tablet_bootstrap.cc:492] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:49.952931 30691 raft_consensus.cc:359] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.957481 30691 raft_consensus.cc:385] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.957940 30691 raft_consensus.cc:740] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.936501 31093 raft_consensus.cc:2804] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:49.958870 30691 consensus_queue.cc:260] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.864717 31098 raft_consensus.cc:515] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:49.982208 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:49.987221 30695 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c: Bootstrap starting.
W20260709 13:09:49.956223 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 141786us
I20260709 13:09:49.970737 31054 raft_consensus.cc:493] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.996500 31054 raft_consensus.cc:515] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:49.997735 31098 leader_election.cc:290] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:49.974921 31093 raft_consensus.cc:493] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:49.949057 30536 leader_election.cc:343] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.949182 30989 consensus_queue.cc:1048] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260709 13:09:49.956580 31088 leader_election.cc:290] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:50.040623 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "366b9fff4017464a9b99ab4878dbb119" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:50.041694 30493 raft_consensus.cc:3060] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.020797 30536 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:50.011363 31093 raft_consensus.cc:3060] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.064221 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "366b9fff4017464a9b99ab4878dbb119" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:50.065086 30420 raft_consensus.cc:3060] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:50.067181 30610 leader_election.cc:343] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.068964 30610 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:50.074872 31054 leader_election.cc:290] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:09:50.079389 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 53979.6us
I20260709 13:09:50.082926 31104 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.083632 30691 ts_tablet_manager.cc:1434] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.160s	user 0.009s	sys 0.000s
I20260709 13:09:50.083745 31104 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.085472 30420 raft_consensus.cc:2468] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
W20260709 13:09:50.100116 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 151100us
I20260709 13:09:50.101263 30891 raft_consensus.cc:2749] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.109635 31087 raft_consensus.cc:2749] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.113653 31104 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:50.124694 30461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 62444.5us
I20260709 13:09:50.136621 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:50.139257 30692 tablet_bootstrap.cc:654] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:50.144598 30461 leader_election.cc:343] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.146000 30691 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
W20260709 13:09:50.154963 30535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 64674.8us
W20260709 13:09:50.165727 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 80379.4us
I20260709 13:09:50.176347 30690 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:50.190143 30690 ts_tablet_manager.cc:1403] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.388s	user 0.014s	sys 0.029s
I20260709 13:09:50.211899 30695 tablet_bootstrap.cc:654] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.246145 30320 catalog_manager.cc:5697] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:50.252852 30690 raft_consensus.cc:359] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:50.224342 30610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 157072us
I20260709 13:09:50.254146 30690 raft_consensus.cc:385] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.240427 30493 raft_consensus.cc:2468] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:50.257591 30690 raft_consensus.cc:740] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.276677 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:50.278069 30611 leader_election.cc:304] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:50.279371 30989 raft_consensus.cc:2804] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:50.279927 30989 raft_consensus.cc:697] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:50.281030 30989 consensus_queue.cc:237] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.273608 30690 consensus_queue.cc:260] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:50.315079 30462 leader_election.cc:343] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.256218 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:50.321874 30535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 52290.4us
I20260709 13:09:50.290042 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:50.317374 31087 raft_consensus.cc:493] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.331274 30567 raft_consensus.cc:2468] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:50.384523 30692 tablet_bootstrap.cc:492] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:50.391809 30692 ts_tablet_manager.cc:1403] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.444s	user 0.040s	sys 0.026s
I20260709 13:09:50.390688 30690 ts_tablet_manager.cc:1434] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.200s	user 0.011s	sys 0.002s
I20260709 13:09:50.361073 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:50.403554 30691 tablet_bootstrap.cc:654] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:50.408318 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 84539.5us
I20260709 13:09:50.363945 31087 raft_consensus.cc:515] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.413527 31087 leader_election.cc:290] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:09:50.422665 30462 leader_election.cc:343] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.423473 30462 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:50.437539 30690 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:50.430496 31123 raft_consensus.cc:2749] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.391988 31093 raft_consensus.cc:515] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:50.481774 31083 consensus_queue.cc:1048] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:50.482172 31093 leader_election.cc:290] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:50.484107 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c56097e8594d298daed39079975dd1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:50.484906 30695 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
W20260709 13:09:50.491708 30463 leader_election.cc:343] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.492827 30695 ts_tablet_manager.cc:1403] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.537s	user 0.037s	sys 0.028s
I20260709 13:09:50.502869 31132 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.503018 30463 leader_election.cc:304] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:50.503773 31132 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.507153 31104 raft_consensus.cc:2749] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.517180 30494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c56097e8594d298daed39079975dd1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:50.518431 30494 raft_consensus.cc:2468] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:50.496233 30695 raft_consensus.cc:359] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.521652 30695 raft_consensus.cc:385] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.521965 30695 raft_consensus.cc:740] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.528119 30695 consensus_queue.cc:260] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.525368 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:50.526876 30691 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:50.541393 30691 ts_tablet_manager.cc:1403] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.396s	user 0.018s	sys 0.019s
I20260709 13:09:50.541389 30692 raft_consensus.cc:359] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.542322 30692 raft_consensus.cc:385] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.542611 30692 raft_consensus.cc:740] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.543345 30692 consensus_queue.cc:260] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.546489 30692 ts_tablet_manager.cc:1434] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.154s	user 0.005s	sys 0.003s
I20260709 13:09:50.540823 30695 ts_tablet_manager.cc:1434] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:09:50.550714 30695 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:50.540722 30570 raft_consensus.cc:3060] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.570691 30690 tablet_bootstrap.cc:654] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.556691 30691 raft_consensus.cc:359] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.574493 30691 raft_consensus.cc:385] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.574869 30691 raft_consensus.cc:740] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.576108 30691 consensus_queue.cc:260] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.590430 31132 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:50.591646 30691 ts_tablet_manager.cc:1434] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:09:50.576833 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:50.584362 30690 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:50.585889 30692 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:50.594450 30492 raft_consensus.cc:3060] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.597329 30690 ts_tablet_manager.cc:1403] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.173s	user 0.017s	sys 0.000s
I20260709 13:09:50.600237 30611 leader_election.cc:304] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:50.602476 30695 tablet_bootstrap.cc:654] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.603446 30891 consensus_queue.cc:1048] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:50.606945 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:50.607841 30645 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:50.608609 30690 raft_consensus.cc:359] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.609473 30690 raft_consensus.cc:385] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.609776 30690 raft_consensus.cc:740] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.610472 30690 consensus_queue.cc:260] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.612326 30691 tablet_bootstrap.cc:492] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44: Bootstrap starting.
I20260709 13:09:50.614179 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:50.615110 30567 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:50.624037 30695 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:50.631104 30695 ts_tablet_manager.cc:1403] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.081s	user 0.007s	sys 0.005s
W20260709 13:09:50.632593 30611 leader_election.cc:343] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.635586 30692 tablet_bootstrap.cc:654] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.627350 31102 raft_consensus.cc:2804] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:50.637046 31102 raft_consensus.cc:493] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:50.637401 31102 raft_consensus.cc:3060] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.645486 30695 raft_consensus.cc:359] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.646310 30695 raft_consensus.cc:385] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.646571 30695 raft_consensus.cc:740] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.647025 30386 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:50.649314 31088 raft_consensus.cc:3060] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.650367 31084 consensus_queue.cc:1048] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:50.647250 30695 consensus_queue.cc:260] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.645627 30492 raft_consensus.cc:2468] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:50.624537 30570 raft_consensus.cc:2468] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:50.673327 30388 leader_election.cc:304] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:50.674832 31132 raft_consensus.cc:2804] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:50.675599 31132 raft_consensus.cc:697] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:50.675989 31088 raft_consensus.cc:2749] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.680953 30690 ts_tablet_manager.cc:1434] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 13:09:50.681535 30691 tablet_bootstrap.cc:654] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.682571 30690 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:50.706290 30695 ts_tablet_manager.cc:1434] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.001s
I20260709 13:09:50.707584 30695 tablet_bootstrap.cc:492] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:50.683864 31102 raft_consensus.cc:515] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.711781 31102 leader_election.cc:290] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:50.676796 31132 consensus_queue.cc:237] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:50.715381 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c56097e8594d298daed39079975dd1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:50.716030 30564 raft_consensus.cc:3060] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.721262 30690 tablet_bootstrap.cc:654] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.746546 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c56097e8594d298daed39079975dd1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:50.753782 30496 raft_consensus.cc:3060] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.775182 30564 raft_consensus.cc:2468] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:50.774326 30317 catalog_manager.cc:5697] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.808580 30611 leader_election.cc:304] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:50.809353 30496 raft_consensus.cc:2468] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:50.810945 30692 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:50.811455 30692 ts_tablet_manager.cc:1403] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.264s	user 0.022s	sys 0.012s
I20260709 13:09:50.814429 31102 raft_consensus.cc:2804] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:50.815199 31102 raft_consensus.cc:697] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:50.817827 30691 tablet_bootstrap.cc:492] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44: No bootstrap required, opened a new log
I20260709 13:09:50.818492 30691 ts_tablet_manager.cc:1403] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44: Time spent bootstrapping tablet: real 0.206s	user 0.017s	sys 0.003s
I20260709 13:09:50.833424 30691 raft_consensus.cc:359] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.834959 30691 raft_consensus.cc:385] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.835309 30691 raft_consensus.cc:740] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.836952 30691 consensus_queue.cc:260] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.840396 30691 ts_tablet_manager.cc:1434] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.003s
I20260709 13:09:50.816879 31102 consensus_queue.cc:237] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.853456 31091 consensus_queue.cc:1048] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:50.857041 31154 log.cc:927] Time spent T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:09:50.883586 30690 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:50.884155 30690 ts_tablet_manager.cc:1403] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.202s	user 0.031s	sys 0.038s
I20260709 13:09:50.880805 30692 raft_consensus.cc:359] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.893189 30692 raft_consensus.cc:385] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.893546 30692 raft_consensus.cc:740] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.895664 31125 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.896242 31125 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.894356 30692 consensus_queue.cc:260] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.898667 30692 ts_tablet_manager.cc:1434] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.000s
I20260709 13:09:50.900063 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:50.896415 30690 raft_consensus.cc:359] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.900921 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:50.902302 30463 leader_election.cc:343] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:50.904848 30461 leader_election.cc:343] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.904881 30908 consensus_queue.cc:1048] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:50.905313 30692 tablet_bootstrap.cc:492] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:50.906906 30461 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:50.898679 31125 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:50.907948 30994 raft_consensus.cc:2749] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.901904 30690 raft_consensus.cc:385] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:50.910145 30690 raft_consensus.cc:740] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:50.910569 30891 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.911147 30891 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.912724 30994 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.913213 30994 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:50.913757 31102 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.914649 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:50.914670 31102 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.915372 30994 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:50.916522 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:50.917444 31102 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:50.918720 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:50.919096 30641 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:50.920035 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:50.920650 30564 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
W20260709 13:09:50.922379 30462 leader_election.cc:343] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.922470 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:50.912540 30690 consensus_queue.cc:260] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.923271 30416 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:50.924389 30535 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:50.925158 30690 ts_tablet_manager.cc:1434] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.002s
I20260709 13:09:50.925199 30985 raft_consensus.cc:2804] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:50.925807 30985 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:50.926146 30985 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.926417 30690 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:50.926353 30610 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
W20260709 13:09:50.926903 30461 leader_election.cc:343] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.927330 30461 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:50.928555 30994 raft_consensus.cc:2749] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.930472 30908 raft_consensus.cc:2804] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:50.931264 30908 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:50.931680 30908 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.933465 30690 tablet_bootstrap.cc:654] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.934613 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:50.935320 30416 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:50.933465 30891 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:50.938431 30695 tablet_bootstrap.cc:654] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.945883 30316 catalog_manager.cc:5697] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:50.937740 30985 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:50.955349 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:50.955703 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:50.957245 30416 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:50.952379 30985 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:50.948853 30692 tablet_bootstrap.cc:654] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:50.964211 30320 catalog_manager.cc:5697] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:50.969303 30831 raft_consensus.cc:493] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.969985 30831 raft_consensus.cc:515] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:50.976069 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:50.978404 30463 leader_election.cc:343] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.982033 30831 leader_election.cc:290] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:50.985729 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:50.989573 30461 leader_election.cc:343] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:50.990154 30461 leader_election.cc:304] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:50.991780 30994 raft_consensus.cc:2749] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:50.992020 30690 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:51.002988 30690 ts_tablet_manager.cc:1403] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.077s	user 0.008s	sys 0.007s
I20260709 13:09:51.006709 30690 raft_consensus.cc:359] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.010048 31102 raft_consensus.cc:493] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.010903 30690 raft_consensus.cc:385] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.011111 31102 raft_consensus.cc:515] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.011307 30690 raft_consensus.cc:740] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.013481 30690 consensus_queue.cc:260] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.019670 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.020272 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:51.021953 30496 raft_consensus.cc:2468] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
W20260709 13:09:51.022935 30610 leader_election.cc:343] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.023965 30611 leader_election.cc:304] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:51.024789 31084 raft_consensus.cc:2804] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.025189 31084 raft_consensus.cc:493] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.025548 31084 raft_consensus.cc:3060] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.029325 31132 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.025349 31102 leader_election.cc:290] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.029867 31132 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.019831 30690 ts_tablet_manager.cc:1434] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:09:51.033479 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.034035 30690 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:51.034170 30493 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:51.038015 31132 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.040455 30388 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:51.041854 31079 raft_consensus.cc:2804] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.042225 31079 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.042564 31079 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.047719 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:51.049472 30386 leader_election.cc:343] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.053385 30690 tablet_bootstrap.cc:654] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:51.057974 30367 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:51.059192 31079 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.006831 30908 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.063016 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:51.063189 30644 raft_consensus.cc:2393] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
W20260709 13:09:51.061395 31159 log.cc:927] Time spent T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:09:51.039922 30320 catalog_manager.cc:5697] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:51.078513 30908 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:51.078878 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:51.079856 30564 raft_consensus.cc:2393] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:51.081810 31084 raft_consensus.cc:515] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:51.069892 30386 leader_election.cc:343] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.083920 31084 leader_election.cc:290] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.087472 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:51.088865 30493 raft_consensus.cc:3060] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.092433 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:51.092585 30695 tablet_bootstrap.cc:492] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:51.094738 30695 ts_tablet_manager.cc:1403] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.387s	user 0.018s	sys 0.000s
I20260709 13:09:51.095827 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:51.096220 31079 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:51.100442 30610 leader_election.cc:343] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.101336 30610 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:51.102284 31087 raft_consensus.cc:2749] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:51.125232 30416 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:51.127897 30536 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:51.129379 30696 raft_consensus.cc:2804] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:51.130013 30696 raft_consensus.cc:697] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:51.131448 30696 consensus_queue.cc:237] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.133852 30493 raft_consensus.cc:2468] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:51.137984 30611 leader_election.cc:304] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:51.149652 30690 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:51.150264 30690 ts_tablet_manager.cc:1403] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.003s
W20260709 13:09:51.153357 30461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 46572.3us
I20260709 13:09:51.155194 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:51.155849 30493 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.156630 31087 raft_consensus.cc:2804] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:51.156358 30695 raft_consensus.cc:359] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.157151 31087 raft_consensus.cc:697] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:51.157128 30695 raft_consensus.cc:385] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.158636 31087 consensus_queue.cc:237] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.160707 30692 tablet_bootstrap.cc:492] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:51.161232 30692 ts_tablet_manager.cc:1403] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.256s	user 0.015s	sys 0.017s
I20260709 13:09:51.158342 30695 raft_consensus.cc:740] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.162586 30695 consensus_queue.cc:260] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.164054 30692 raft_consensus.cc:359] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.164755 30692 raft_consensus.cc:385] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.165050 30317 catalog_manager.cc:5697] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.165490 30690 raft_consensus.cc:359] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.166616 30690 raft_consensus.cc:385] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.166901 30690 raft_consensus.cc:740] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.165046 30692 raft_consensus.cc:740] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.168540 30690 consensus_queue.cc:260] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.169026 30692 consensus_queue.cc:260] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.171677 30692 ts_tablet_manager.cc:1434] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:09:51.172310 30493 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:51.172331 30690 ts_tablet_manager.cc:1434] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:09:51.173224 30692 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:51.174618 30695 ts_tablet_manager.cc:1434] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.080s	user 0.005s	sys 0.009s
I20260709 13:09:51.177476 30695 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:51.182029 30692 tablet_bootstrap.cc:654] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.184752 30388 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:51.186656 31132 raft_consensus.cc:2804] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:51.187204 31132 raft_consensus.cc:697] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:51.188382 31132 consensus_queue.cc:237] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.194797 30695 tablet_bootstrap.cc:654] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.212828 30690 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:51.223229 30695 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:51.223963 30695 ts_tablet_manager.cc:1403] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.006s
I20260709 13:09:51.234588 30695 raft_consensus.cc:359] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.236060 30695 raft_consensus.cc:385] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.242700 30695 raft_consensus.cc:740] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.243489 30695 consensus_queue.cc:260] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.245396 30692 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:51.245728 30695 ts_tablet_manager.cc:1434] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 13:09:51.246096 30692 ts_tablet_manager.cc:1403] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.073s	user 0.026s	sys 0.018s
I20260709 13:09:51.250386 30690 tablet_bootstrap.cc:654] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.251749 30316 catalog_manager.cc:5697] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.256518 30692 raft_consensus.cc:359] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.257611 30692 raft_consensus.cc:385] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.257900 30692 raft_consensus.cc:740] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.258560 30692 consensus_queue.cc:260] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.261415 30692 ts_tablet_manager.cc:1434] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:09:51.263662 31093 consensus_queue.cc:1048] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:51.270288 30692 tablet_bootstrap.cc:492] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:51.283636 31093 raft_consensus.cc:493] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.284351 31093 raft_consensus.cc:515] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.285617 30318 catalog_manager.cc:5697] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.287803 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:51.287742 30690 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:51.289212 30690 ts_tablet_manager.cc:1403] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.005s
W20260709 13:09:51.290120 30386 leader_election.cc:343] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b897660686b3465390a5bad32efea13c (127.20.66.132:45099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.290448 30695 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:51.291401 31093 leader_election.cc:290] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:51.297750 30692 tablet_bootstrap.cc:654] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.302603 30695 tablet_bootstrap.cc:654] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.294976 30690 raft_consensus.cc:359] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.309371 30690 raft_consensus.cc:385] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.309684 30690 raft_consensus.cc:740] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.311540 30690 consensus_queue.cc:260] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.318684 30695 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:51.319350 30695 ts_tablet_manager.cc:1403] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.005s
I20260709 13:09:51.319854 31088 raft_consensus.cc:493] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.320695 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:51.320531 31088 raft_consensus.cc:515] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.322384 30695 raft_consensus.cc:359] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.322999 31088 leader_election.cc:290] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:51.325546 30690 ts_tablet_manager.cc:1434] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 13:09:51.333285 30690 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:51.323088 30695 raft_consensus.cc:385] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.334658 30695 raft_consensus.cc:740] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.335879 30695 consensus_queue.cc:260] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.341674 30908 consensus_queue.cc:1048] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.345476 30695 ts_tablet_manager.cc:1434] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:09:51.355098 30695 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:51.357729 30690 tablet_bootstrap.cc:654] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:51.362066 30388 leader_election.cc:343] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.362576 30388 leader_election.cc:304] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:51.363842 31088 raft_consensus.cc:2749] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:51.367724 30695 tablet_bootstrap.cc:654] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.384007 31093 consensus_queue.cc:1048] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:51.402388 30692 tablet_bootstrap.cc:492] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:51.403064 30692 ts_tablet_manager.cc:1403] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.133s	user 0.016s	sys 0.027s
I20260709 13:09:51.410193 30696 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.411885 30696 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.415230 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.415982 30496 raft_consensus.cc:2468] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:51.419260 30537 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:51.421113 30696 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:51.421774 30985 raft_consensus.cc:2804] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.423476 30690 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:51.424175 30690 ts_tablet_manager.cc:1403] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.091s	user 0.026s	sys 0.000s
I20260709 13:09:51.427786 30690 raft_consensus.cc:359] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.422276 30985 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.429430 30985 raft_consensus.cc:3060] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.428730 30690 raft_consensus.cc:385] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.431023 30690 raft_consensus.cc:740] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.329149 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.431946 30690 consensus_queue.cc:260] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.439198 31102 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.439920 31102 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.442518 31102 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:51.444638 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:51.445367 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:51.446477 30564 raft_consensus.cc:2468] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:51.445371 30645 raft_consensus.cc:2468] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:51.451344 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.452356 30493 raft_consensus.cc:2468] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:51.453665 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:51.459342 30611 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:51.459995 30690 ts_tablet_manager.cc:1434] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.004s
I20260709 13:09:51.461402 31087 raft_consensus.cc:2804] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.461824 31087 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.462157 31087 raft_consensus.cc:3060] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.464890 30690 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:51.465386 30695 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:51.465916 30695 ts_tablet_manager.cc:1403] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.111s	user 0.035s	sys 0.017s
I20260709 13:09:51.469636 30692 raft_consensus.cc:359] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.470556 30692 raft_consensus.cc:385] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.470817 30692 raft_consensus.cc:740] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.471468 30692 consensus_queue.cc:260] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.475771 30908 consensus_queue.cc:1048] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.476137 30692 ts_tablet_manager.cc:1434] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.002s
I20260709 13:09:51.448560 30492 raft_consensus.cc:2468] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:51.477059 30388 leader_election.cc:304] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:51.478235 30695 raft_consensus.cc:359] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.479038 30695 raft_consensus.cc:385] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.479259 30695 raft_consensus.cc:740] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.479792 30695 consensus_queue.cc:260] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.487171 30695 ts_tablet_manager.cc:1434] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:09:51.486186 31093 raft_consensus.cc:2804] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.491477 31093 raft_consensus.cc:493] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.492057 31093 raft_consensus.cc:3060] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.488556 30908 raft_consensus.cc:493] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.496003 30908 raft_consensus.cc:515] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.499979 30690 tablet_bootstrap.cc:654] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.502521 30692 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:51.503199 30695 tablet_bootstrap.cc:492] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:51.508160 31087 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.512815 30908 leader_election.cc:290] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:51.513622 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:51.514448 30496 raft_consensus.cc:3060] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.515089 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:51.515817 30416 raft_consensus.cc:2468] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:51.519196 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:51.520061 30610 leader_election.cc:304] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:51.520709 30695 tablet_bootstrap.cc:654] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.521202 31102 raft_consensus.cc:2804] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.521628 31102 raft_consensus.cc:493] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.521986 31102 raft_consensus.cc:3060] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.531044 31087 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:09:51.534832 30611 leader_election.cc:343] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.478010 30985 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.544385 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:51.545778 30690 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:51.546387 30690 ts_tablet_manager.cc:1403] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.007s
I20260709 13:09:51.542320 31132 raft_consensus.cc:493] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.549607 31132 raft_consensus.cc:515] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.550980 30690 raft_consensus.cc:359] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.552243 31132 leader_election.cc:290] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.552491 30690 raft_consensus.cc:385] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.552898 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:51.556249 30495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b4252704f04cae82d10d8b1bb4c88d" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.557175 30495 raft_consensus.cc:2393] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:51.560468 30388 leader_election.cc:304] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:51.561704 31133 raft_consensus.cc:3060] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.554045 30690 raft_consensus.cc:740] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.563697 30690 consensus_queue.cc:260] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.572479 30566 raft_consensus.cc:2393] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:51.574771 30985 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:51.576350 30692 tablet_bootstrap.cc:654] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.578811 30695 tablet_bootstrap.cc:492] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:51.579334 30695 ts_tablet_manager.cc:1403] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.000s
I20260709 13:09:51.582408 30690 ts_tablet_manager.cc:1434] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 13:09:51.583523 30644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:51.584429 30644 raft_consensus.cc:2393] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:51.595919 30695 raft_consensus.cc:359] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.596956 30695 raft_consensus.cc:385] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.597317 30695 raft_consensus.cc:740] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.597977 30695 consensus_queue.cc:260] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.599378 31102 raft_consensus.cc:515] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.606434 31093 raft_consensus.cc:515] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.610908 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:51.615312 30695 ts_tablet_manager.cc:1434] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:09:51.604233 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:51.611831 30566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:51.615670 30493 raft_consensus.cc:3060] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.617450 30566 raft_consensus.cc:3060] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.617781 30537 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:51.618957 31103 raft_consensus.cc:2749] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:51.620739 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:51.610265 30496 raft_consensus.cc:2468] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:51.621419 30416 raft_consensus.cc:3060] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.626317 31133 raft_consensus.cc:2749] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:51.629962 30644 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.632115 30493 raft_consensus.cc:2468] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:51.633841 30388 leader_election.cc:304] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:51.640928 30690 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:51.641752 30611 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:51.659291 30566 raft_consensus.cc:2468] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:51.660989 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:51.661573 31091 raft_consensus.cc:2804] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:51.662150 31091 raft_consensus.cc:697] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:51.663103 31091 consensus_queue.cc:237] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.678772 30695 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c: Bootstrap starting.
I20260709 13:09:51.677225 31084 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.611250 31093 leader_election.cc:290] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:51.640980 31132 raft_consensus.cc:2804] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:09:51.689890 30611 leader_election.cc:343] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.676788 31079 consensus_queue.cc:1048] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:09:51.681099 31084 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.613590 31102 leader_election.cc:290] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:51.729144 31084 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.711721 31132 raft_consensus.cc:697] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:51.742770 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.743503 30496 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:51.747042 30611 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:51.747596 31133 consensus_queue.cc:1048] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:09:51.752615 30416 raft_consensus.cc:2468] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:51.758025 31132 consensus_queue.cc:237] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.758797 30692 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:51.759240 30690 tablet_bootstrap.cc:654] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.761951 30692 ts_tablet_manager.cc:1403] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.262s	user 0.030s	sys 0.049s
I20260709 13:09:51.783454 30908 raft_consensus.cc:2804] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:51.788899 30985 consensus_queue.cc:1048] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:51.792218 30610 leader_election.cc:304] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:51.793694 30695 tablet_bootstrap.cc:654] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:51.795153 31087 raft_consensus.cc:2804] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:51.798606 31132 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.814381 31132 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.810143 31087 raft_consensus.cc:697] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:51.815925 31087 consensus_queue.cc:237] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:51.819445 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:51.825471 30388 leader_election.cc:343] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.826234 31133 raft_consensus.cc:493] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.826386 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:51.826828 31133 raft_consensus.cc:515] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.831296 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:51.832572 30908 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:51.833127 30908 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:51.834523 30388 leader_election.cc:343] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:51.834949 30692 raft_consensus.cc:359] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.835852 30692 raft_consensus.cc:385] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.836107 30692 raft_consensus.cc:740] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.836782 30692 consensus_queue.cc:260] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.838035 31132 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:51.838861 30692 ts_tablet_manager.cc:1434] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 13:09:51.839841 30692 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:51.828859 31133 leader_election.cc:290] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:51.841645 30696 consensus_queue.cc:1048] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.849391 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.851043 30493 raft_consensus.cc:2468] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:51.863267 30388 leader_election.cc:304] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:51.864948 30692 tablet_bootstrap.cc:654] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:51.876470 30610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 84777.5us
I20260709 13:09:51.865099 31093 raft_consensus.cc:2804] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.926791 31093 raft_consensus.cc:493] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.927141 31093 raft_consensus.cc:3060] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.928925 30908 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.931098 30908 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.871433 30318 catalog_manager.cc:5697] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.951262 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:51.935940 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:51.937240 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:51.942231 31091 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.911062 30690 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: No bootstrap required, opened a new log
I20260709 13:09:51.952895 31091 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.954429 30695 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c: No bootstrap required, opened a new log
I20260709 13:09:51.955286 30695 ts_tablet_manager.cc:1403] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c: Time spent bootstrapping tablet: real 0.299s	user 0.014s	sys 0.011s
I20260709 13:09:51.956125 31091 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.944188 31190 raft_consensus.cc:493] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.945369 31189 consensus_queue.cc:1048] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.957523 31190 raft_consensus.cc:515] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.951571 30985 raft_consensus.cc:493] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.953400 30690 ts_tablet_manager.cc:1403] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.313s	user 0.010s	sys 0.027s
I20260709 13:09:51.958622 30985 raft_consensus.cc:515] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:51.960034 31190 leader_election.cc:290] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:51.875986 30316 catalog_manager.cc:5697] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:51.961372 30985 leader_election.cc:290] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:51.943297 30641 raft_consensus.cc:1240] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Rejecting Update request from peer 5825cf9d14b4469c9960cca89b0010f1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:51.963550 30690 raft_consensus.cc:359] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.964247 30690 raft_consensus.cc:385] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:51.964519 30690 raft_consensus.cc:740] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:51.965484 30690 consensus_queue.cc:260] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.966321 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.967021 30492 raft_consensus.cc:2468] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:51.967669 31132 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.968151 31132 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.969355 30537 leader_election.cc:304] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:51.975298 31184 consensus_queue.cc:1059] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.975551 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:51.976241 31184 raft_consensus.cc:3055] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.976469 30416 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:51.977011 31132 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:51.977811 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.978145 30690 ts_tablet_manager.cc:1434] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:09:51.962555 31093 raft_consensus.cc:515] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:51.978644 30492 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:51.980348 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.981072 30496 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:51.976614 31184 raft_consensus.cc:740] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:51.982273 30696 raft_consensus.cc:2804] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.982682 30696 raft_consensus.cc:493] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.983024 30696 raft_consensus.cc:3060] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.984247 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:51.985109 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:51.985738 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:51.985980 30496 raft_consensus.cc:2468] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 0.
I20260709 13:09:51.986515 30641 raft_consensus.cc:2468] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:51.991513 30611 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:51.992417 31091 raft_consensus.cc:2749] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:51.991892 30388 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:51.992870 30985 raft_consensus.cc:493] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.993716 31088 raft_consensus.cc:2804] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:51.994236 31088 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.994712 31088 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:51.983024 31184 consensus_queue.cc:260] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:51.996402 31184 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.993599 30985 raft_consensus.cc:515] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:52.004180 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:52.004956 30641 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:52.008126 30985 leader_election.cc:290] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:52.010901 30610 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:52.018674 30611 leader_election.cc:304] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:52.013581 30419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:52.017349 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:52.021325 30492 raft_consensus.cc:2468] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 0.
I20260709 13:09:52.019496 31091 raft_consensus.cc:2804] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:52.027613 31091 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.028132 31091 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.019590 31189 raft_consensus.cc:2804] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:52.031494 31189 raft_consensus.cc:493] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.032213 31189 raft_consensus.cc:3060] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.016664 30696 raft_consensus.cc:515] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:52.033875 30564 raft_consensus.cc:2393] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:52.038829 31190 consensus_queue.cc:1048] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:52.040627 30537 leader_election.cc:304] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:52.039538 30696 leader_election.cc:290] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:52.042377 31192 raft_consensus.cc:2804] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:52.042781 31192 raft_consensus.cc:493] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.043130 31192 raft_consensus.cc:3060] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.043036 30695 raft_consensus.cc:359] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.043933 30695 raft_consensus.cc:385] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:52.044214 30695 raft_consensus.cc:740] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Initialized, Role: FOLLOWER
I20260709 13:09:52.044243 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:52.045559 30695 consensus_queue.cc:260] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.039660 31093 leader_election.cc:290] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:52.048468 30695 ts_tablet_manager.cc:1434] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.000s
I20260709 13:09:52.051009 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:52.055514 30645 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 0.
I20260709 13:09:52.056921 31192 raft_consensus.cc:515] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:52.057639 31088 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:52.059880 31088 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:52.062027 31192 leader_election.cc:290] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:52.069506 30692 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:52.070972 30692 ts_tablet_manager.cc:1403] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.231s	user 0.008s	sys 0.007s
I20260709 13:09:52.069463 31189 raft_consensus.cc:515] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:52.071923 30641 raft_consensus.cc:2393] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:52.074998 31189 leader_election.cc:290] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:52.020421 30419 raft_consensus.cc:2393] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:52.044922 30493 raft_consensus.cc:3060] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.142060 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:52.144248 30420 raft_consensus.cc:2393] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:52.148073 31091 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:52.150993 30386 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:52.148274 30692 raft_consensus.cc:359] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.156489 30692 raft_consensus.cc:385] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:52.153317 31091 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:52.154253 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:52.156898 30692 raft_consensus.cc:740] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:52.187587 31132 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:52.188334 31132 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.188757 31132 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.202046 30492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:52.203104 30492 raft_consensus.cc:3060] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:52.210575 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 65294.2us
I20260709 13:09:52.215291 30637 raft_consensus.cc:2393] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:52.215759 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:52.217262 30564 raft_consensus.cc:2393] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:52.220913 31132 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.226440 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:52.230271 30496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:52.229543 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3b4d4c8fc44213807c39418459447d" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:52.231443 30496 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.233011 30567 raft_consensus.cc:2393] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5825cf9d14b4469c9960cca89b0010f1 in current term 1: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:52.255314 30692 consensus_queue.cc:260] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.258318 30692 ts_tablet_manager.cc:1434] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.186s	user 0.009s	sys 0.000s
I20260709 13:09:52.260690 30692 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: Bootstrap starting.
I20260709 13:09:52.269249 30388 leader_election.cc:304] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:52.271613 31133 raft_consensus.cc:2749] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:52.272856 30420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:52.274336 31184 consensus_queue.cc:1048] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:09:52.275457 30420 raft_consensus.cc:2393] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:52.279230 30496 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:52.287880 31088 consensus_queue.cc:1048] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:52.297478 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:52.304234 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:52.310839 30637 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.315385 31132 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:52.321352 31087 consensus_queue.cc:1048] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:52.322307 30570 raft_consensus.cc:3060] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.326763 31190 consensus_queue.cc:1048] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:52.334991 30495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eba54fb250484a4e8102345d2a02e5cb" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:52.335441 30489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:52.336346 30489 raft_consensus.cc:2393] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 1: Already voted for candidate 5825cf9d14b4469c9960cca89b0010f1 in this term.
I20260709 13:09:52.344911 30316 catalog_manager.cc:5697] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 1 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:52.308209 30388 leader_election.cc:343] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:52.363471 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 55293.3us
W20260709 13:09:52.362162 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 83721.2us
I20260709 13:09:52.362373 31091 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.368963 31091 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.371364 31091 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:52.380148 30985 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.381032 30985 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.387216 30985 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:52.389684 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:52.391032 30416 raft_consensus.cc:2468] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:52.389823 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:52.393745 30610 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:52.394131 30421 raft_consensus.cc:2468] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:52.395617 31091 raft_consensus.cc:2804] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:52.396062 31091 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.396464 31091 raft_consensus.cc:3060] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.396306 30536 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:52.378504 30388 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:52.410694 31207 raft_consensus.cc:2804] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:52.411336 31207 raft_consensus.cc:493] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.411945 31207 raft_consensus.cc:3060] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:52.439395 30388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 63672.8us
I20260709 13:09:52.440048 31184 raft_consensus.cc:2804] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:52.450428 30492 raft_consensus.cc:2468] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:52.452104 30537 leader_election.cc:304] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
W20260709 13:09:52.460250 31208 log.cc:927] Time spent T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:09:52.460449 30696 raft_consensus.cc:2804] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:52.474617 30696 raft_consensus.cc:697] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:52.475703 30696 consensus_queue.cc:237] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:52.454738 31184 raft_consensus.cc:697] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:52.483460 31184 consensus_queue.cc:237] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:52.488528 30692 tablet_bootstrap.cc:654] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:52.488273 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:52.495992 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:52.498883 30637 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 1.
I20260709 13:09:52.504532 30908 consensus_queue.cc:1048] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:09:52.505841 30386 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 955d2e588381448981f85fadf225aa05
I20260709 13:09:52.511214 31133 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:52.518910 31133 raft_consensus.cc:697] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:09:52.519449 31091 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.519976 31133 consensus_queue.cc:237] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.521180 30645 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 2: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:52.524832 30493 raft_consensus.cc:2468] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:52.529187 30537 leader_election.cc:304] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:52.533247 30985 raft_consensus.cc:2804] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:52.533835 30985 raft_consensus.cc:697] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:52.534880 30985 consensus_queue.cc:237] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:09:52.505312 31207 raft_consensus.cc:515] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.551859 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:52.552580 30416 raft_consensus.cc:3060] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.555042 30611 leader_election.cc:304] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:09:52.564105 30570 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 2: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:52.547082 31091 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:52.581835 30416 raft_consensus.cc:2468] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
I20260709 13:09:52.583099 30610 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:52.588515 30908 raft_consensus.cc:2804] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:52.589082 30908 raft_consensus.cc:697] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:52.594460 31190 raft_consensus.cc:2749] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:52.604069 30908 consensus_queue.cc:237] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:52.605161 30611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 59515.7us
I20260709 13:09:52.606423 30611 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:52.607450 31087 raft_consensus.cc:2749] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:52.620493 31190 consensus_queue.cc:1048] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:52.623183 31213 log.cc:927] Time spent T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:52.636883 30421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:52.637817 30421 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 2: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:52.635861 31207 leader_election.cc:290] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:52.653092 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:52.654059 30564 raft_consensus.cc:2393] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 2: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:52.657711 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679bf6d4a8654a72b47b318645ab6fc0" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:52.659610 30985 raft_consensus.cc:493] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.664536 30535 leader_election.cc:304] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:52.666608 31207 raft_consensus.cc:2749] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:52.676038 30692 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: No bootstrap required, opened a new log
I20260709 13:09:52.676817 30692 ts_tablet_manager.cc:1403] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: Time spent bootstrapping tablet: real 0.417s	user 0.026s	sys 0.019s
I20260709 13:09:52.683565 30317 catalog_manager.cc:5697] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 0 to 2, leader changed from <none> to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 2 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:52.676852 30985 raft_consensus.cc:515] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.691205 30985 leader_election.cc:290] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:52.681134 30320 catalog_manager.cc:5697] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:52.693754 30692 raft_consensus.cc:359] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.694669 30692 raft_consensus.cc:385] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:52.695636 30416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:52.694959 30692 raft_consensus.cc:740] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Initialized, Role: FOLLOWER
I20260709 13:09:52.699819 30320 catalog_manager.cc:5697] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:52.712617 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bccb92fd917f4a97af88be6e7e8fed1f" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:52.722728 30535 leader_election.cc:304] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:52.731730 30692 consensus_queue.cc:260] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:52.733562 31207 raft_consensus.cc:2749] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:52.735054 30692 ts_tablet_manager.cc:1434] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.003s
I20260709 13:09:52.745398 30316 catalog_manager.cc:5697] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:52.747676 30316 catalog_manager.cc:5697] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:52.870901 30985 consensus_queue.cc:1048] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:52.890350 31207 consensus_queue.cc:1048] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:52.917663 31088 consensus_queue.cc:1048] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:52.949823 31088 consensus_queue.cc:1048] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:52.969281 30565 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:52.984416 31093 consensus_queue.cc:1048] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:52.992525 31133 consensus_queue.cc:1048] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:53.016636 31207 consensus_queue.cc:1048] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:53.031970 30415 raft_consensus.cc:1275] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:09:53.036407 31087 consensus_queue.cc:1048] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:53.076249 31207 consensus_queue.cc:1048] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:53.130429 30565 raft_consensus.cc:1275] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:09:53.137633 31087 consensus_queue.cc:1048] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:53.367448 30994 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:53.368052 30994 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:53.370461 30994 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:53.371575 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:53.371578 30415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:53.372653 30415 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:09:53.372822 30637 raft_consensus.cc:2393] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19b1445cece541d3b2abe177283fef44 in current term 2: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:53.374229 30462 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:09:53.375167 30994 raft_consensus.cc:2804] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:53.376019 30994 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:53.376477 30994 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:53.387943 30994 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:53.391552 30994 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:53.392550 30415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:53.392552 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:53.393723 30641 raft_consensus.cc:2393] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19b1445cece541d3b2abe177283fef44 in current term 2: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:53.400698 30415 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:09:53.402026 30462 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: b897660686b3465390a5bad32efea13c
I20260709 13:09:53.403237 30994 raft_consensus.cc:2804] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:53.403796 30994 raft_consensus.cc:697] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:53.404927 30994 consensus_queue.cc:237] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:53.424844 30318 catalog_manager.cc:5697] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.852655 30637 raft_consensus.cc:1275] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:53.854755 30994 consensus_queue.cc:1048] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:53.925787 30415 raft_consensus.cc:1275] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:53.928231 30994 consensus_queue.cc:1048] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:54.673707 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
W20260709 13:09:54.685971 30462 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:54.695245 30462 consensus_peers.cc:597] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:54.726678 30610 consensus_peers.cc:597] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.764927 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:54.767148 20746 tablet_replica.cc:333] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.770437 20746 raft_consensus.cc:2243] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.772166 20746 raft_consensus.cc:2272] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.777566 20746 tablet_replica.cc:333] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.779320 20746 raft_consensus.cc:2243] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.780786 20746 raft_consensus.cc:2272] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.786474 20746 tablet_replica.cc:333] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.788350 20746 raft_consensus.cc:2243] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.789646 20746 raft_consensus.cc:2272] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.799619 20746 tablet_replica.cc:333] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.800877 20746 raft_consensus.cc:2243] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.801615 20746 raft_consensus.cc:2272] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.807535 20746 tablet_replica.cc:333] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:54.807710 30536 consensus_peers.cc:597] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.809276 20746 raft_consensus.cc:2243] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.810870 20746 raft_consensus.cc:2272] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.817689 20746 tablet_replica.cc:333] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.818861 20746 raft_consensus.cc:2243] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.819635 20746 raft_consensus.cc:2272] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.825027 20746 tablet_replica.cc:333] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:54.825125 30462 consensus_peers.cc:597] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.826165 20746 raft_consensus.cc:2243] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.826977 20746 raft_consensus.cc:2272] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.830713 20746 tablet_replica.cc:333] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.831950 20746 raft_consensus.cc:2243] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.832768 20746 raft_consensus.cc:2272] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:54.834022 30610 consensus_peers.cc:597] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.837291 20746 tablet_replica.cc:333] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.838806 20746 raft_consensus.cc:2243] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.839720 20746 raft_consensus.cc:2272] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.845670 20746 tablet_replica.cc:333] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:54.845770 30462 consensus_peers.cc:597] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.848927 20746 raft_consensus.cc:2243] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.852522 20746 raft_consensus.cc:2272] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.857976 20746 tablet_replica.cc:333] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:54.858130 30610 consensus_peers.cc:597] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.859683 20746 raft_consensus.cc:2243] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.860548 20746 raft_consensus.cc:2272] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.864233 20746 tablet_replica.cc:333] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.865345 20746 raft_consensus.cc:2243] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:09:54.865497 30462 consensus_peers.cc:597] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.866073 20746 raft_consensus.cc:2272] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:54.866318 30462 consensus_peers.cc:597] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.871501 20746 tablet_replica.cc:333] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.872668 20746 raft_consensus.cc:2243] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.873404 20746 raft_consensus.cc:2272] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.880781 20746 tablet_replica.cc:333] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.882081 20746 raft_consensus.cc:2243] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.883520 20746 raft_consensus.cc:2272] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.887704 20746 tablet_replica.cc:333] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.888790 20746 raft_consensus.cc:2243] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:09:54.898708 30610 consensus_peers.cc:597] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.889541 20746 raft_consensus.cc:2272] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:54.899298 30536 consensus_peers.cc:597] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:54.900169 30536 consensus_peers.cc:597] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.904536 20746 tablet_replica.cc:333] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.905968 20746 raft_consensus.cc:2243] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.906814 20746 raft_consensus.cc:2272] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.910981 20746 tablet_replica.cc:333] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.912195 20746 raft_consensus.cc:2243] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.913509 20746 raft_consensus.cc:2272] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.917379 20746 tablet_replica.cc:333] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.918445 20746 raft_consensus.cc:2243] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.919950 20746 raft_consensus.cc:2272] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.923991 20746 tablet_replica.cc:333] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.925086 20746 raft_consensus.cc:2243] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.926815 20746 raft_consensus.cc:2272] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.930843 20746 tablet_replica.cc:333] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.931960 20746 raft_consensus.cc:2243] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.932809 20746 raft_consensus.cc:2272] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.937043 20746 tablet_replica.cc:333] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:54.939296 30610 consensus_peers.cc:597] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.940284 20746 raft_consensus.cc:2243] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.941373 20746 raft_consensus.cc:2272] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.945242 20746 tablet_replica.cc:333] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.947642 20746 raft_consensus.cc:2243] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.949334 20746 raft_consensus.cc:2272] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.953040 20746 tablet_replica.cc:333] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:54.953090 30462 consensus_peers.cc:597] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.953976 20746 raft_consensus.cc:2243] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.954705 20746 raft_consensus.cc:2272] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.958303 20746 tablet_replica.cc:333] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.959285 20746 raft_consensus.cc:2243] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.959985 20746 raft_consensus.cc:2272] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:54.961966 30462 consensus_peers.cc:597] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.965478 20746 tablet_replica.cc:333] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.966668 20746 raft_consensus.cc:2243] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:54.967717 20746 raft_consensus.cc:2272] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.971885 20746 tablet_replica.cc:333] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.973060 20746 raft_consensus.cc:2243] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.974779 20746 raft_consensus.cc:2272] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.978785 20746 tablet_replica.cc:333] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.981395 20746 raft_consensus.cc:2243] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.982939 20746 raft_consensus.cc:2272] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:54.987231 20746 tablet_replica.cc:333] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.991513 20746 raft_consensus.cc:2243] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.993076 20746 raft_consensus.cc:2272] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:54.995100 30610 consensus_peers.cc:597] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:54.996912 20746 tablet_replica.cc:333] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:54.998030 20746 raft_consensus.cc:2243] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:54.999287 20746 raft_consensus.cc:2272] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.003409 20746 tablet_replica.cc:333] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.004513 20746 raft_consensus.cc:2243] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:55.006215 20746 raft_consensus.cc:2272] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.011130 20746 tablet_replica.cc:333] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.012085 20746 raft_consensus.cc:2243] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.012862 20746 raft_consensus.cc:2272] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.016963 20746 tablet_replica.cc:333] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
W20260709 13:09:55.017915 30536 consensus_peers.cc:597] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:55.018119 30462 consensus_peers.cc:597] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.018666 20746 raft_consensus.cc:2243] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.019459 20746 raft_consensus.cc:2272] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.022806 20746 tablet_replica.cc:333] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.023574 20746 raft_consensus.cc:2243] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.024171 20746 raft_consensus.cc:2272] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.026808 30610 consensus_peers.cc:597] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.028121 20746 tablet_replica.cc:333] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.028955 20746 raft_consensus.cc:2243] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.029757 20746 raft_consensus.cc:2272] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.033752 20746 tablet_replica.cc:333] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.035193 20746 raft_consensus.cc:2243] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.036095 20746 raft_consensus.cc:2272] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.039970 20746 tablet_replica.cc:333] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.040913 20746 raft_consensus.cc:2243] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.041669 20746 raft_consensus.cc:2272] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.042704 30610 consensus_peers.cc:597] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.045049 20746 tablet_replica.cc:333] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.045995 20746 raft_consensus.cc:2243] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.046794 20746 raft_consensus.cc:2272] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.050379 20746 tablet_replica.cc:333] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.051328 20746 raft_consensus.cc:2243] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.052271 20746 raft_consensus.cc:2272] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.055831 20746 tablet_replica.cc:333] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.056658 20746 raft_consensus.cc:2243] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.057390 20746 raft_consensus.cc:2272] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.061650 20746 tablet_replica.cc:333] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.062609 20746 raft_consensus.cc:2243] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.063280 20746 raft_consensus.cc:2272] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:55.067179 20746 tablet_replica.cc:333] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.068111 20746 raft_consensus.cc:2243] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:55.069733 20746 raft_consensus.cc:2272] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.068585 30536 consensus_peers.cc:597] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.073676 20746 tablet_replica.cc:333] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.074854 20746 raft_consensus.cc:2243] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:09:55.075932 30536 consensus_peers.cc:597] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.076423 20746 raft_consensus.cc:2272] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.077368 30536 consensus_peers.cc:597] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.080749 20746 tablet_replica.cc:333] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.081872 20746 raft_consensus.cc:2243] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.082618 20746 raft_consensus.cc:2272] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.085489 30610 consensus_peers.cc:597] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.086822 20746 tablet_replica.cc:333] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.087790 20746 raft_consensus.cc:2243] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.088694 20746 raft_consensus.cc:2272] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.091460 30536 consensus_peers.cc:597] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.096379 20746 tablet_replica.cc:333] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: stopping tablet replica
I20260709 13:09:55.097419 20746 raft_consensus.cc:2243] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:55.098232 20746 raft_consensus.cc:2272] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:55.128029 30462 consensus_peers.cc:597] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:55.141515 30536 consensus_peers.cc:597] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:55.176832 30610 consensus_peers.cc:597] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.242197 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
W20260709 13:09:55.260895 30610 consensus_peers.cc:597] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.406081 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:55.416487 31258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:55.419171 31259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:55.425931 31262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:55.426438 20746 server_base.cc:1061] running on GCE node
I20260709 13:09:55.431493 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:55.431864 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:55.432106 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602595432088 us; error 0 us; skew 500 ppm
I20260709 13:09:55.432912 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:55.436872 20746 webserver.cc:533] Webserver started at http://127.20.66.129:36493/ using document root <none> and password file <none>
I20260709 13:09:55.437662 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:55.437948 20746 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:09:55.443506 20746 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:09:55.462316 31266 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:55.463255 20746 fs_manager.cc:730] Time spent opening block manager: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:09:55.463663 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "5825cf9d14b4469c9960cca89b0010f1"
format_stamp: "Formatted at 2026-07-09 13:09:39 on dist-test-slave-rgrd"
I20260709 13:09:55.464253 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:55.488904 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:55.490700 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:55.505792 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:55.512539 31273 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:09:55.819885 31245 raft_consensus.cc:493] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:55.820739 31245 raft_consensus.cc:515] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
W20260709 13:09:55.830051 30536 leader_election.cc:336] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:55.832576 31245 leader_election.cc:290] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:55.833366 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:55.834975 30537 leader_election.cc:304] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:55.836189 31245 raft_consensus.cc:2749] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.854617 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:09:55.855047 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.344s	user 0.009s	sys 0.001s
I20260709 13:09:55.855376 20746 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:09:55.861342 31273 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:55.872345 31240 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:55.872972 31240 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:55.891710 31273 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:55.892757 31273 tablet_bootstrap.cc:492] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
W20260709 13:09:55.892946 30462 leader_election.cc:336] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:55.893433 31273 ts_tablet_manager.cc:1403] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.012s
I20260709 13:09:55.896214 31273 raft_consensus.cc:359] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:55.896885 31273 raft_consensus.cc:740] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.897691 31273 consensus_queue.cc:260] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:55.924326 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:55.926355 30463 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:55.927538 31248 raft_consensus.cc:2749] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.930392 31240 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:55.950210 31103 raft_consensus.cc:493] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:55.952523 31103 raft_consensus.cc:515] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:55.953305 31245 raft_consensus.cc:493] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:55.953953 31245 raft_consensus.cc:515] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:55.957247 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:55.958920 31245 leader_election.cc:290] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:55.967790 31103 leader_election.cc:290] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:55.977128 31248 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:55.977927 31248 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:55.950368 31252 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:55.982220 31252 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:55.985826 31252 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:55.970016 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.002825 31273 ts_tablet_manager.cc:1434] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.109s	user 0.049s	sys 0.058s
I20260709 13:09:55.987263 31248 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:55.993184 30994 raft_consensus.cc:493] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.003741 31273 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:56.003995 30994 raft_consensus.cc:515] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.004441 30908 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.004968 30908 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.006045 30994 leader_election.cc:290] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:55.994074 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.006876 30641 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:56.008716 30461 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:56.010181 30994 raft_consensus.cc:2804] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.010653 30994 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.011143 30994 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.016535 31248 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.017766 31248 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.006788 30908 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.022854 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.031533 31273 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:09:56.002187 30610 leader_election.cc:336] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.052336 30610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 50251.5us
I20260709 13:09:56.032832 31273 tablet_bootstrap.cc:492] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:56.041858 31192 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.054361 31273 ts_tablet_manager.cc:1403] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.051s	user 0.032s	sys 0.007s
I20260709 13:09:56.058292 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.000113 31240 raft_consensus.cc:493] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.059196 31240 raft_consensus.cc:515] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.054047 31192 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.061484 31240 leader_election.cc:290] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:56.061959 31192 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.064788 31273 raft_consensus.cc:359] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.070334 31273 raft_consensus.cc:740] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:56.071161 31273 consensus_queue.cc:260] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.067728 30908 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.072217 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.072327 30908 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.074862 30908 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.072971 30493 raft_consensus.cc:2468] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
W20260709 13:09:56.069829 30610 leader_election.cc:336] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.079742 31248 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:56.080981 30537 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:56.081820 31103 raft_consensus.cc:2804] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.082201 31103 raft_consensus.cc:493] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.082588 31103 raft_consensus.cc:3060] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.083595 31273 ts_tablet_manager.cc:1434] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:09:56.087203 31273 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:56.051761 31279 raft_consensus.cc:493] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.090065 31279 raft_consensus.cc:515] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
W20260709 13:09:56.088881 30610 leader_election.cc:336] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.092264 30565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.092736 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.093405 30567 raft_consensus.cc:2468] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:56.088061 30994 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.103361 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:56.104182 30637 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.112092 30637 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:09:56.113343 30461 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:56.113461 30908 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.113590 31279 leader_election.cc:290] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.114004 30908 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.115092 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.115700 30493 raft_consensus.cc:2468] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.115859 30908 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.117084 30537 leader_election.cc:304] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:56.117841 31245 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.118307 31245 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.120530 31245 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:56.121649 30610 leader_election.cc:336] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.124955 31248 raft_consensus.cc:2804] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.125384 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.125736 31248 raft_consensus.cc:697] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:56.126708 31248 consensus_queue.cc:237] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.127676 30611 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:56.128052 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.128525 30908 raft_consensus.cc:2749] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.129627 31273 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:56.130697 31273 tablet_bootstrap.cc:492] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:56.131395 31273 ts_tablet_manager.cc:1403] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.044s	user 0.025s	sys 0.000s
I20260709 13:09:56.134373 31273 raft_consensus.cc:359] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.135095 31273 raft_consensus.cc:740] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:56.135947 31192 raft_consensus.cc:493] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.136257 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.136580 31192 raft_consensus.cc:515] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.136934 30493 raft_consensus.cc:2468] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.103693 30994 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.140501 30463 leader_election.cc:304] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:56.140779 31281 raft_consensus.cc:493] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.141695 31240 raft_consensus.cc:2804] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.141439 31281 raft_consensus.cc:515] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.142118 31240 raft_consensus.cc:493] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.142530 31240 raft_consensus.cc:3060] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.143764 31281 leader_election.cc:290] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.145905 30537 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
W20260709 13:09:56.138908 30536 leader_election.cc:336] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.146842 30536 leader_election.cc:304] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:56.147383 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.147646 31281 raft_consensus.cc:2804] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.148066 31281 raft_consensus.cc:493] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.148391 31281 raft_consensus.cc:3060] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.135780 31273 consensus_queue.cc:260] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.148669 30611 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:56.148797 31280 raft_consensus.cc:2749] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.150467 31273 ts_tablet_manager.cc:1434] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:09:56.150312 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.151095 30493 raft_consensus.cc:2468] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.151367 31273 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:56.157401 30536 leader_election.cc:336] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.157866 30536 leader_election.cc:304] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:56.158881 31280 raft_consensus.cc:2749] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:56.159617 30536 leader_election.cc:336] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.156633 31281 raft_consensus.cc:515] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.167299 31281 leader_election.cc:290] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.167733 31252 raft_consensus.cc:2749] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.170728 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.175596 31278 raft_consensus.cc:493] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.179605 31278 raft_consensus.cc:515] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.181910 31278 leader_election.cc:290] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:56.181228 30318 catalog_manager.cc:5697] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.183343 31240 raft_consensus.cc:515] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.142536 31245 raft_consensus.cc:2804] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.186031 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.186316 31245 raft_consensus.cc:493] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.186703 31245 raft_consensus.cc:3060] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.186673 30570 raft_consensus.cc:2468] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:56.189546 30463 leader_election.cc:304] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:56.190541 31278 raft_consensus.cc:2804] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.191138 31278 raft_consensus.cc:493] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.191612 31278 raft_consensus.cc:3060] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.190476 30462 leader_election.cc:336] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.193953 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a15b8193c82e4caeb06f0da37159665c" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
W20260709 13:09:56.194164 30462 leader_election.cc:336] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.194599 30491 raft_consensus.cc:3060] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.194875 30462 leader_election.cc:304] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:56.195477 30537 leader_election.cc:304] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
W20260709 13:09:56.197469 30536 leader_election.cc:336] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.198518 31240 leader_election.cc:290] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:56.138933 31192 leader_election.cc:290] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.199432 31281 raft_consensus.cc:2804] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.199873 31281 raft_consensus.cc:493] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.200245 31281 raft_consensus.cc:3060] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.205406 30536 leader_election.cc:336] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.212957 30491 raft_consensus.cc:2468] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.216571 31087 raft_consensus.cc:493] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.217118 31087 raft_consensus.cc:515] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.174153 31103 raft_consensus.cc:515] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.219256 30537 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:56.220134 31280 raft_consensus.cc:2804] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.220636 31280 raft_consensus.cc:697] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.221135 31103 leader_election.cc:290] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.221580 31280 consensus_queue.cc:237] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.226024 31248 raft_consensus.cc:2749] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.227567 31248 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.228071 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d204ad91a1c74eb9b3335582ee6717c7" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:56.228700 30567 raft_consensus.cc:3060] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.228346 31248 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:56.229629 30610 leader_election.cc:336] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.230435 31087 leader_election.cc:290] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.231288 31248 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.226356 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.233294 31279 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.234772 31279 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.236079 30611 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:56.236959 30908 raft_consensus.cc:2749] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.213339 31281 raft_consensus.cc:515] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.233389 30570 raft_consensus.cc:2468] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:56.238322 31279 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.231779 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.239547 31281 leader_election.cc:290] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:56.234041 30462 leader_election.cc:336] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.243752 30567 raft_consensus.cc:2468] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:09:56.244359 31273 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:09:56.215862 30536 leader_election.cc:336] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.245311 30463 leader_election.cc:304] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
W20260709 13:09:56.246068 30536 leader_election.cc:336] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.246559 30536 leader_election.cc:304] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.246639 31248 raft_consensus.cc:2804] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.248579 31279 raft_consensus.cc:2749] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.248777 31248 raft_consensus.cc:697] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:56.250211 31248 consensus_queue.cc:237] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.253302 30611 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.253446 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a6f0d4b4c848deb76000430c8bb0a5" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:56.245352 31273 tablet_bootstrap.cc:492] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:56.254195 30493 raft_consensus.cc:3060] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.254347 31285 raft_consensus.cc:2749] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.255909 30611 leader_election.cc:304] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.257052 31285 raft_consensus.cc:2804] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.257436 31285 raft_consensus.cc:493] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.257808 31285 raft_consensus.cc:3060] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.258766 30462 leader_election.cc:336] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.260607 30462 leader_election.cc:304] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:56.259213 31273 ts_tablet_manager.cc:1403] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.108s	user 0.038s	sys 0.050s
I20260709 13:09:56.261195 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.261868 30637 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.263980 31273 raft_consensus.cc:359] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.264760 30994 raft_consensus.cc:2749] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.277908 30994 raft_consensus.cc:493] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.278369 30994 raft_consensus.cc:515] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.264014 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9035baa441924d8b9218b26855598f19" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:56.279860 30491 raft_consensus.cc:3060] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.280135 30994 leader_election.cc:290] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.280895 31273 raft_consensus.cc:740] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:56.283113 31273 consensus_queue.cc:260] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.285967 31252 raft_consensus.cc:493] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.286567 31252 raft_consensus.cc:515] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:56.247646 30536 leader_election.cc:336] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.265501 30462 leader_election.cc:336] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.216073 31245 raft_consensus.cc:515] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.291204 31245 leader_election.cc:290] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.288811 31252 leader_election.cc:290] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.292934 31285 raft_consensus.cc:515] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.275919 31278 raft_consensus.cc:515] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
W20260709 13:09:56.296654 30462 leader_election.cc:336] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.298162 31273 ts_tablet_manager.cc:1434] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.002s
W20260709 13:09:56.300226 30610 leader_election.cc:336] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.309419 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.310235 31273 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:56.300928 30536 leader_election.cc:336] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.302670 30493 raft_consensus.cc:2468] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.267951 30535 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
W20260709 13:09:56.311717 30462 leader_election.cc:336] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.307770 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:09:56.312116 30536 leader_election.cc:336] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.312347 30567 raft_consensus.cc:2468] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:56.306689 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea53b9ee66e343b18d301fe650b12554" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:56.307853 30491 raft_consensus.cc:2468] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.306406 31285 leader_election.cc:290] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.296648 31278 leader_election.cc:290] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:09:56.319494 31087 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
W20260709 13:09:56.319731 30535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 51875.4us
I20260709 13:09:56.312906 30537 leader_election.cc:304] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
W20260709 13:09:56.318924 30462 leader_election.cc:336] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.322453 30462 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:56.323712 31087 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.326116 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e2ca1d8a742e1a0beb1710464b3bf" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:56.326330 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78e63f8484d24f8f9bc5c94fef442265" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:56.326877 30567 raft_consensus.cc:3060] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.327617 30316 catalog_manager.cc:5697] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.328299 30537 leader_election.cc:304] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.330238 31281 raft_consensus.cc:2804] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.330709 31281 raft_consensus.cc:697] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.331580 31281 consensus_queue.cc:237] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.326927 30570 raft_consensus.cc:3060] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.332702 31192 raft_consensus.cc:2804] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.335707 30567 raft_consensus.cc:2468] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:09:56.323161 30497 raft_consensus.cc:3060] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.323573 31245 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.345139 30463 leader_election.cc:304] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:09:56.350411 31087 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.327731 31279 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.352435 31292 raft_consensus.cc:2749] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.362666 31291 raft_consensus.cc:2804] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.362922 30497 raft_consensus.cc:2468] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.363574 30570 raft_consensus.cc:2468] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
I20260709 13:09:56.363482 31291 raft_consensus.cc:697] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:56.365383 31291 consensus_queue.cc:237] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
W20260709 13:09:56.361065 30462 leader_election.cc:336] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.345337 31245 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.381702 30462 leader_election.cc:304] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19b1445cece541d3b2abe177283fef44; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
W20260709 13:09:56.378363 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 67205us
W20260709 13:09:56.385426 30536 leader_election.cc:336] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.393983 31087 raft_consensus.cc:493] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.371523 31279 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.394541 31087 raft_consensus.cc:515] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.399145 31087 leader_election.cc:290] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.399633 31087 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.400215 31087 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.402338 31087 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.403313 30908 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.404542 30908 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.373392 31103 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.406579 30908 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.406275 31103 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.408087 31279 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.389498 31245 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.402117 31292 raft_consensus.cc:2749] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:56.405081 30610 leader_election.cc:336] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.413453 30610 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.372144 31192 raft_consensus.cc:697] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.416702 30994 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
W20260709 13:09:56.418166 30462 leader_election.cc:336] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.414680 31103 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.418599 30994 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.422271 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.423036 30637 raft_consensus.cc:2468] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 1.
I20260709 13:09:56.424286 30461 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:56.425166 31294 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.420799 30994 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.417095 31192 consensus_queue.cc:237] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.426954 31294 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:09:56.433132 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 104899us
W20260709 13:09:56.444725 30536 leader_election.cc:336] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.425230 31292 raft_consensus.cc:2804] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:09:56.449247 30610 leader_election.cc:336] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.450587 30610 leader_election.cc:336] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.455578 30611 leader_election.cc:304] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.450160 31292 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.461621 31294 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.464802 31292 raft_consensus.cc:3060] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.477977 30611 leader_election.cc:304] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.479800 31087 raft_consensus.cc:2804] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.480330 31087 raft_consensus.cc:697] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:56.482183 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:56.483009 30567 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:56.483956 31087 consensus_queue.cc:237] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.465251 31245 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.487775 31245 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:09:56.493104 30610 leader_election.cc:336] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.502702 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.503669 30641 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.505667 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.506408 30497 raft_consensus.cc:2468] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:56.506579 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.507539 30641 raft_consensus.cc:2468] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.513746 31298 raft_consensus.cc:2804] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.514282 31298 raft_consensus.cc:493] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.514676 31298 raft_consensus.cc:3060] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.522683 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.533751 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:56.534871 30637 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:56.537822 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:56.538663 30637 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.544507 30462 leader_election.cc:336] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.548173 30611 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.550578 31252 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.551223 31252 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.551702 31252 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.559037 30611 leader_election.cc:304] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.560184 30908 raft_consensus.cc:2804] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.560541 30908 raft_consensus.cc:493] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.560897 30908 raft_consensus.cc:3060] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.565208 30537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:35751, user_credentials={real_user=slave}} blocked reactor thread for 115705us
I20260709 13:09:56.566233 30537 leader_election.cc:304] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.569818 31280 raft_consensus.cc:2804] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.574203 31280 raft_consensus.cc:697] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.574694 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
I20260709 13:09:56.575353 30497 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 1.
I20260709 13:09:56.575120 31280 consensus_queue.cc:237] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.579030 30535 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:56.579901 30535 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:56.582118 31192 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.582906 31273 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:56.584903 31273 tablet_bootstrap.cc:492] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:56.583012 31281 raft_consensus.cc:2804] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.583112 30535 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:56.585572 31281 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
W20260709 13:09:56.583839 30536 leader_election.cc:336] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.585788 31273 ts_tablet_manager.cc:1403] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.276s	user 0.020s	sys 0.016s
I20260709 13:09:56.589233 31301 raft_consensus.cc:2804] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.590003 31301 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.590389 31301 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.592423 30536 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.582492 31292 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.582525 31192 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.593828 31192 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.594352 30491 raft_consensus.cc:2393] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 2: Already voted for candidate 19b1445cece541d3b2abe177283fef44 in this term.
I20260709 13:09:56.586033 31281 raft_consensus.cc:3060] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.605867 31292 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.611130 30637 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.606546 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6664fcb5ba4f487b957d5cc98ec559de" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:56.645664 30645 raft_consensus.cc:3060] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.647289 31192 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.649726 31192 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.654910 31281 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.657451 30908 raft_consensus.cc:515] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.659313 30491 raft_consensus.cc:1275] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.659459 30908 leader_election.cc:290] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.605677 31273 raft_consensus.cc:359] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.630546 30611 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.661509 30535 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.662281 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:56.662567 31087 raft_consensus.cc:2804] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:56.662983 30641 raft_consensus.cc:3060] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.663142 31087 raft_consensus.cc:493] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:09:56.663614 31087 raft_consensus.cc:3060] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.664351 31301 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.665556 30611 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.662596 31192 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.667536 31192 raft_consensus.cc:697] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.667618 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:56.668303 30643 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:56.629419 30536 leader_election.cc:336] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.636237 30462 leader_election.cc:336] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.669657 30536 leader_election.cc:336] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.638789 31298 raft_consensus.cc:515] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.643023 31252 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.644266 30644 raft_consensus.cc:1275] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.674520 31252 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.675782 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "289ca1e3324041acb6ac3410039210be" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:56.585666 30318 catalog_manager.cc:5697] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:56.676455 30491 raft_consensus.cc:3060] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.602092 30316 catalog_manager.cc:5697] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.676697 30994 consensus_queue.cc:1048] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.677592 30316 catalog_manager.cc:5697] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.682541 31298 leader_election.cc:290] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.683635 30645 raft_consensus.cc:2468] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:09:56.684937 30461 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.685668 31304 raft_consensus.cc:2749] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.686244 30994 raft_consensus.cc:2804] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.686782 30994 raft_consensus.cc:697] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:09:56.687593 30994 consensus_queue.cc:237] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:56.689039 30610 leader_election.cc:336] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.692050 30641 raft_consensus.cc:2468] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.668419 31192 consensus_queue.cc:237] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.693307 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:56.694267 30637 raft_consensus.cc:2393] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 955d2e588381448981f85fadf225aa05 in current term 2: Already voted for candidate b897660686b3465390a5bad32efea13c in this term.
I20260709 13:09:56.696029 31301 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.696264 31087 raft_consensus.cc:515] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.696969 30491 raft_consensus.cc:2468] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
I20260709 13:09:56.698283 31087 leader_election.cc:290] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:09:56.692696 30318 catalog_manager.cc:5697] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:56.685720 30536 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.700888 30536 leader_election.cc:336] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:09:56.701231 30610 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.703147 31279 consensus_queue.cc:1048] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.704802 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc578fa661c4935aae4aece586f4271" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:56.705457 30567 raft_consensus.cc:3060] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.707055 30643 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:56.712401 30567 raft_consensus.cc:2468] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
W20260709 13:09:56.713116 30610 leader_election.cc:336] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.656893 31281 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:56.662564 31273 raft_consensus.cc:740] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:56.729655 31273 consensus_queue.cc:260] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.731602 31273 ts_tablet_manager.cc:1434] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.144s	user 0.014s	sys 0.002s
I20260709 13:09:56.732398 31273 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:56.712972 30536 leader_election.cc:336] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.714768 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:09:56.734915 30567 raft_consensus.cc:2393] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b897660686b3465390a5bad32efea13c in current term 2: Already voted for candidate 955d2e588381448981f85fadf225aa05 in this term.
I20260709 13:09:56.714779 30535 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.716529 30497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17363c79974c6a863b51a45eef6e0e" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:09:56.716722 30611 leader_election.cc:304] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.735822 30497 raft_consensus.cc:3060] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.736892 30908 raft_consensus.cc:2804] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.737424 30908 raft_consensus.cc:697] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:56.737540 30611 leader_election.cc:304] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.738204 30908 consensus_queue.cc:237] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:56.738751 30611 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:09:56.739705 31298 raft_consensus.cc:2804] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.740202 31298 raft_consensus.cc:697] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
W20260709 13:09:56.727217 30462 consensus_peers.cc:597] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.740989 31298 consensus_queue.cc:237] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.741140 31252 raft_consensus.cc:2749] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:09:56.743346 30536 consensus_peers.cc:597] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.746626 30497 raft_consensus.cc:2468] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
I20260709 13:09:56.749817 30611 leader_election.cc:304] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
W20260709 13:09:56.750344 30610 leader_election.cc:336] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:09:56.751348 31304 raft_consensus.cc:2804] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.751854 31304 raft_consensus.cc:697] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:09:56.752692 31304 consensus_queue.cc:237] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:56.766501 30535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.132:45099, user_credentials={real_user=slave}} blocked reactor thread for 51795.8us
I20260709 13:09:56.767997 30535 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c
I20260709 13:09:56.769266 31296 raft_consensus.cc:2749] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.767454 31279 raft_consensus.cc:2804] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.771188 31279 raft_consensus.cc:697] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.771391 30535 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:09:56.772666 31301 raft_consensus.cc:2804] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.773237 31301 raft_consensus.cc:697] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:56.772500 31279 consensus_queue.cc:237] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.774092 31301 consensus_queue.cc:237] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:56.778585 20746 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:09:56.778991 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.924s	user 0.318s	sys 0.005s
I20260709 13:09:56.786480 30571 raft_consensus.cc:1275] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.788717 31240 consensus_queue.cc:1048] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:56.829026 30318 catalog_manager.cc:5697] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 2 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:56.841265 30491 raft_consensus.cc:1275] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.843281 31192 consensus_queue.cc:1048] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.856701 30316 catalog_manager.cc:5697] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.858213 30317 catalog_manager.cc:5697] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.862349 30317 catalog_manager.cc:5697] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:56.893744 30462 consensus_peers.cc:597] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.902822 31273 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:09:56.907976 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 56755.3us
I20260709 13:09:56.912370 30571 raft_consensus.cc:1275] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.914768 30908 consensus_queue.cc:1048] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:56.938956 30610 consensus_peers.cc:597] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.942070 31273 tablet_bootstrap.cc:492] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:56.944200 31273 ts_tablet_manager.cc:1403] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.212s	user 0.025s	sys 0.014s
I20260709 13:09:56.947332 31273 raft_consensus.cc:359] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.948227 31273 raft_consensus.cc:740] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
W20260709 13:09:56.951861 30536 consensus_peers.cc:597] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.952011 31273 consensus_queue.cc:260] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:56.957799 31273 ts_tablet_manager.cc:1434] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:09:56.958601 31273 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:56.963284 30462 consensus_peers.cc:597] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.970564 30493 raft_consensus.cc:1275] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.973907 30318 catalog_manager.cc:5697] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 2 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:56.974699 30570 raft_consensus.cc:1275] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.976994 31321 consensus_queue.cc:1048] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:56.977334 31279 consensus_queue.cc:1048] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:09:56.982894 30318 catalog_manager.cc:5697] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 2 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:56.989481 30318 catalog_manager.cc:5697] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 2 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:57.016485 30490 raft_consensus.cc:1275] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.034475 30317 catalog_manager.cc:5697] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:57.035420 31279 consensus_queue.cc:1048] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:57.036484 30317 catalog_manager.cc:5697] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 2, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:57.051781 30536 consensus_peers.cc:597] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:57.058586 30536 consensus_peers.cc:597] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:57.083911 31273 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.085704 30643 raft_consensus.cc:1275] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.086282 31273 tablet_bootstrap.cc:492] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.087298 31273 ts_tablet_manager.cc:1403] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.129s	user 0.017s	sys 0.000s
I20260709 13:09:57.089808 31322 consensus_queue.cc:1048] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:57.135452 30493 raft_consensus.cc:1275] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.138023 31304 consensus_queue.cc:1048] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:57.141093 31273 raft_consensus.cc:359] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.141878 31273 raft_consensus.cc:740] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.142549 31273 consensus_queue.cc:260] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.144546 31273 ts_tablet_manager.cc:1434] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.000s
I20260709 13:09:57.145380 31273 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:57.160218 30610 consensus_peers.cc:597] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:57.171846 30491 raft_consensus.cc:1275] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.173880 31304 consensus_queue.cc:1048] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:57.182880 30462 consensus_peers.cc:597] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.183661 30462 consensus_peers.cc:597] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:57.199365 30570 raft_consensus.cc:1275] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:57.201265 30536 consensus_peers.cc:597] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:57.203317 30908 consensus_queue.cc:1048] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:57.219564 30637 raft_consensus.cc:1275] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.231446 31301 consensus_queue.cc:1048] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:57.239735 30462 consensus_peers.cc:597] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.274685 31273 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.274820 31276 raft_consensus.cc:493] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.275691 31273 tablet_bootstrap.cc:492] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.275417 31276 raft_consensus.cc:515] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.277375 31273 ts_tablet_manager.cc:1403] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.132s	user 0.034s	sys 0.049s
I20260709 13:09:57.280723 31273 raft_consensus.cc:359] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.281524 31273 raft_consensus.cc:740] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.282231 31273 consensus_queue.cc:260] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.282661 31276 leader_election.cc:290] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:57.332959 30637 raft_consensus.cc:1275] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.359172 31273 ts_tablet_manager.cc:1434] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.081s	user 0.013s	sys 0.005s
I20260709 13:09:57.360216 31273 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:57.355345 31301 consensus_queue.cc:1048] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
W20260709 13:09:57.371977 30536 consensus_peers.cc:597] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:57.420112 31346 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.420950 31346 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.425783 31346 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:09:57.356266 30610 consensus_peers.cc:597] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:57.431295 30610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 75065.5us
W20260709 13:09:57.431859 30610 consensus_peers.cc:597] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.454272 30462 consensus_peers.cc:597] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.454366 30610 consensus_peers.cc:597] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.457442 30610 consensus_peers.cc:597] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:57.455946 30536 consensus_peers.cc:597] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.471320 30610 consensus_peers.cc:597] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.494743 30645 raft_consensus.cc:1275] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:57.497583 31280 consensus_queue.cc:1048] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:57.506585 30536 consensus_peers.cc:597] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:57.507243 30536 consensus_peers.cc:597] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.514779 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:57.515623 30610 consensus_peers.cc:597] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.516963 30610 consensus_peers.cc:597] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.517735 30645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
W20260709 13:09:57.521344 30610 consensus_peers.cc:597] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.522807 30462 consensus_peers.cc:597] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.536952 31273 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.545300 31273 tablet_bootstrap.cc:492] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.546065 31273 ts_tablet_manager.cc:1403] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.186s	user 0.027s	sys 0.032s
I20260709 13:09:57.548424 31273 raft_consensus.cc:359] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.549455 31273 raft_consensus.cc:740] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.549367 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47885caab3e14ff2ae88840a9dfc0944" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:57.550370 31273 consensus_queue.cc:260] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:09:57.552290 30462 consensus_peers.cc:597] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.554315 31273 ts_tablet_manager.cc:1434] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:09:57.555137 31273 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:57.558530 31270 leader_election.cc:304] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:57.559803 31346 raft_consensus.cc:2749] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:09:57.578415 30536 consensus_peers.cc:597] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.578974 30536 consensus_peers.cc:597] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.580034 30462 consensus_peers.cc:597] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.589901 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19b1445cece541d3b2abe177283fef44" is_pre_election: true
W20260709 13:09:57.602008 30610 consensus_peers.cc:597] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.607937 31270 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c
I20260709 13:09:57.640081 31346 raft_consensus.cc:2749] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:57.648324 30610 consensus_peers.cc:597] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.649478 30536 consensus_peers.cc:597] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.651206 30462 consensus_peers.cc:597] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.657742 31273 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.658763 31273 tablet_bootstrap.cc:492] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.659444 31273 ts_tablet_manager.cc:1403] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.104s	user 0.028s	sys 0.042s
I20260709 13:09:57.662052 31273 raft_consensus.cc:359] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.663108 31273 raft_consensus.cc:740] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.663844 31273 consensus_queue.cc:260] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.666585 31273 ts_tablet_manager.cc:1434] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:57.667445 31273 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:57.668579 30462 consensus_peers.cc:597] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.670377 30462 consensus_peers.cc:597] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.686866 31273 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.688467 31273 tablet_bootstrap.cc:492] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.689211 31273 ts_tablet_manager.cc:1403] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.010s
I20260709 13:09:57.692471 31273 raft_consensus.cc:359] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.693420 31273 raft_consensus.cc:740] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.694465 31273 consensus_queue.cc:260] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.697176 31273 ts_tablet_manager.cc:1434] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:09:57.698217 31273 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:57.715581 31273 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.716640 31273 tablet_bootstrap.cc:492] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.717411 31273 ts_tablet_manager.cc:1403] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:09:57.720310 31273 raft_consensus.cc:359] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.721180 31273 raft_consensus.cc:740] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.722105 31273 consensus_queue.cc:260] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.724577 31273 ts_tablet_manager.cc:1434] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:57.725481 31273 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:57.750583 31273 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.751693 31273 tablet_bootstrap.cc:492] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.752417 31273 ts_tablet_manager.cc:1403] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.006s
I20260709 13:09:57.755101 31273 raft_consensus.cc:359] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.755867 31273 raft_consensus.cc:740] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.756569 31273 consensus_queue.cc:260] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.759267 31273 ts_tablet_manager.cc:1434] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:09:57.760151 31273 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:57.764521 30536 consensus_peers.cc:597] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.765533 31346 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.766055 31346 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.768229 31346 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:57.768935 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:57.769706 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:57.771195 31268 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:57.772341 31346 raft_consensus.cc:2749] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:57.795162 30610 consensus_peers.cc:597] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.797478 31273 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.798547 31273 tablet_bootstrap.cc:492] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.799257 31273 ts_tablet_manager.cc:1403] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.011s
I20260709 13:09:57.803200 31273 raft_consensus.cc:359] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.803910 31273 raft_consensus.cc:740] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.805827 31273 consensus_queue.cc:260] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.808833 31273 ts_tablet_manager.cc:1434] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:09:57.820055 31273 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:57.823542 30536 consensus_peers.cc:597] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.837991 30610 consensus_peers.cc:597] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.842226 30610 consensus_peers.cc:597] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:57.847625 30536 consensus_peers.cc:597] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:57.859578 31273 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.860705 31273 tablet_bootstrap.cc:492] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.861528 31273 ts_tablet_manager.cc:1403] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.008s
I20260709 13:09:57.864658 31273 raft_consensus.cc:359] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.865407 31273 raft_consensus.cc:740] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.866466 31273 consensus_queue.cc:260] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:57.868860 31273 ts_tablet_manager.cc:1434] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:57.869721 31273 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:57.889322 31346 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.889971 31346 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:57.892230 31346 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:57.899966 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:57.912379 31273 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:57.913527 31273 tablet_bootstrap.cc:492] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:57.914270 31273 ts_tablet_manager.cc:1403] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.012s
I20260709 13:09:57.917352 31273 raft_consensus.cc:359] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.918236 31273 raft_consensus.cc:740] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:57.919171 31273 consensus_queue.cc:260] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:57.921553 31273 ts_tablet_manager.cc:1434] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:09:57.922406 31273 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:57.937733 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:57.954546 31270 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:57.956269 31346 raft_consensus.cc:2749] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:58.056399 30536 consensus_peers.cc:597] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:58.156703 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 90647.7us
I20260709 13:09:58.161913 31273 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.162993 31273 tablet_bootstrap.cc:492] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.163646 31273 ts_tablet_manager.cc:1403] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.241s	user 0.039s	sys 0.071s
I20260709 13:09:58.166240 31273 raft_consensus.cc:359] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.174539 31273 raft_consensus.cc:740] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.175379 31273 consensus_queue.cc:260] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.186614 31273 ts_tablet_manager.cc:1434] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 13:09:58.187445 31273 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.203181 31273 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.204171 31273 tablet_bootstrap.cc:492] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.209354 31273 ts_tablet_manager.cc:1403] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.001s
I20260709 13:09:58.211944 31273 raft_consensus.cc:359] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.212940 31273 raft_consensus.cc:740] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.213644 31273 consensus_queue.cc:260] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.215250 31273 ts_tablet_manager.cc:1434] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:58.216040 31273 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.225555 31414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:34915 every 8 connection(s)
I20260709 13:09:58.226720 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:34915
I20260709 13:09:58.266358 31381 raft_consensus.cc:3060] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.290644 31273 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.292646 31273 tablet_bootstrap.cc:492] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.293565 31273 ts_tablet_manager.cc:1403] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.003s
I20260709 13:09:58.297049 31273 raft_consensus.cc:359] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.297971 31273 raft_consensus.cc:740] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.298585 31273 consensus_queue.cc:260] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.301292 31273 ts_tablet_manager.cc:1434] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:58.302142 31273 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.321936 31383 raft_consensus.cc:3060] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.371760 31301 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.372363 31301 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.373525 31273 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.375906 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:58.376549 30637 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:09:58.379359 30535 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:58.381865 31273 tablet_bootstrap.cc:492] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.382752 31273 ts_tablet_manager.cc:1403] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.081s	user 0.026s	sys 0.020s
I20260709 13:09:58.384250 31192 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:58.384652 31192 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.384940 31192 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.388288 31273 raft_consensus.cc:359] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.389051 31273 raft_consensus.cc:740] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.389801 31273 consensus_queue.cc:260] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.392951 31273 ts_tablet_manager.cc:1434] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:09:58.394075 31273 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.413393 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1" is_pre_election: true
I20260709 13:09:58.414886 31378 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 1.
I20260709 13:09:58.405128 31381 raft_consensus.cc:3060] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.438673 31377 raft_consensus.cc:3060] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.444193 31416 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41433
I20260709 13:09:58.444819 31416 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:58.424170 31301 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:58.445861 31416 heartbeater.cc:507] Master 127.20.66.190:41433 requested a full tablet report, sending...
I20260709 13:09:58.460934 31378 raft_consensus.cc:3060] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.463168 31376 raft_consensus.cc:3060] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.458822 31192 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.487741 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:09:58.488381 30637 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.498279 31346 raft_consensus.cc:493] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.498970 31346 raft_consensus.cc:515] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.500986 31192 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:58.501407 31273 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.501986 31346 leader_election.cc:290] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:09:58.502351 31273 tablet_bootstrap.cc:492] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.502518 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
I20260709 13:09:58.503074 31273 ts_tablet_manager.cc:1403] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.109s	user 0.016s	sys 0.003s
I20260709 13:09:58.503681 31375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
I20260709 13:09:58.504658 31375 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.504971 30637 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:09:58.506609 30535 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:09:58.508459 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1213ccbc173c43f683d18d933293e4de" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:09:58.508476 31273 raft_consensus.cc:359] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:58.511301 31273 raft_consensus.cc:740] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.512423 31273 consensus_queue.cc:260] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:58.514055 31192 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:58.515913 31192 raft_consensus.cc:697] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:09:58.518710 31273 ts_tablet_manager.cc:1434] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:09:58.519552 31273 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.523713 31268 leader_election.cc:304] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1; no voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c
I20260709 13:09:58.527971 31192 consensus_queue.cc:237] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.539112 31375 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:09:58.530125 31383 raft_consensus.cc:3060] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.635442 30320 catalog_manager.cc:5697] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 reported cstate change: term changed from 1 to 3, leader changed from 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129) to 955d2e588381448981f85fadf225aa05 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "955d2e588381448981f85fadf225aa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:58.699781 31273 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.701288 31273 tablet_bootstrap.cc:492] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.701989 31273 ts_tablet_manager.cc:1403] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.183s	user 0.031s	sys 0.026s
I20260709 13:09:58.725132 31346 raft_consensus.cc:2749] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:58.727051 31273 raft_consensus.cc:359] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.727886 31273 raft_consensus.cc:740] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.728600 31273 consensus_queue.cc:260] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:58.731161 31273 ts_tablet_manager.cc:1434] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:09:58.732046 31273 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.738940 30317 ts_manager.cc:194] Re-registered known tserver with Master: 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:09:58.767819 31273 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.768908 31273 tablet_bootstrap.cc:492] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.770605 31273 ts_tablet_manager.cc:1403] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.002s
I20260709 13:09:58.774070 31273 raft_consensus.cc:359] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.774789 31273 raft_consensus.cc:740] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.775585 31273 consensus_queue.cc:260] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.777933 31273 ts_tablet_manager.cc:1434] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:58.778743 31273 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.801910 31273 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.802721 31273 tablet_bootstrap.cc:492] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.803270 31273 ts_tablet_manager.cc:1403] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.000s
I20260709 13:09:58.807214 31273 raft_consensus.cc:359] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:58.808019 31273 raft_consensus.cc:740] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.808698 31273 consensus_queue.cc:260] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:58.811058 31273 ts_tablet_manager.cc:1434] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:58.811900 31273 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.829049 31273 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.830166 31273 tablet_bootstrap.cc:492] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.830965 31273 ts_tablet_manager.cc:1403] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:09:58.833747 31273 raft_consensus.cc:359] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:58.834475 31273 raft_consensus.cc:740] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.835157 31273 consensus_queue.cc:260] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:58.836992 31273 ts_tablet_manager.cc:1434] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:58.837778 31273 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.842801 30317 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39664
I20260709 13:09:58.851152 31416 heartbeater.cc:499] Master 127.20.66.190:41433 was elected leader, sending a full tablet report...
I20260709 13:09:58.861881 31273 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.862848 31273 tablet_bootstrap.cc:492] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.863560 31273 ts_tablet_manager.cc:1403] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.002s
I20260709 13:09:58.866405 31273 raft_consensus.cc:359] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.867218 31273 raft_consensus.cc:740] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.867882 31273 consensus_queue.cc:260] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.871622 31273 ts_tablet_manager.cc:1434] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:09:58.872469 31273 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.902719 31273 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.903765 31273 tablet_bootstrap.cc:492] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.904484 31273 ts_tablet_manager.cc:1403] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.004s
I20260709 13:09:58.907202 31273 raft_consensus.cc:359] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.907990 31273 raft_consensus.cc:740] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.908763 31273 consensus_queue.cc:260] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.922384 31273 ts_tablet_manager.cc:1434] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:09:58.923244 31273 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.957428 31273 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.959934 31273 tablet_bootstrap.cc:492] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:58.960700 31273 ts_tablet_manager.cc:1403] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.005s
I20260709 13:09:58.963462 31273 raft_consensus.cc:359] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.965329 31273 raft_consensus.cc:740] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:58.966365 31273 consensus_queue.cc:260] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:58.970857 31273 ts_tablet_manager.cc:1434] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 13:09:58.971916 31273 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:58.978763 31378 raft_consensus.cc:3060] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.055390 31383 raft_consensus.cc:1275] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:59.058393 31192 consensus_queue.cc:1048] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:59.158756 31273 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.159349 30636 raft_consensus.cc:1275] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Refusing update from remote peer 955d2e588381448981f85fadf225aa05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:59.160007 31273 tablet_bootstrap.cc:492] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.160895 31273 ts_tablet_manager.cc:1403] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.189s	user 0.065s	sys 0.071s
I20260709 13:09:59.163617 31429 consensus_queue.cc:1048] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:59.165738 31273 raft_consensus.cc:359] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:59.166622 31273 raft_consensus.cc:740] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.167434 31273 consensus_queue.cc:260] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:59.169618 31273 ts_tablet_manager.cc:1434] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:09:59.170440 31273 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.206382 31273 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.207507 31273 tablet_bootstrap.cc:492] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.209453 31273 ts_tablet_manager.cc:1403] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.000s
I20260709 13:09:59.212218 31273 raft_consensus.cc:359] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.213147 31273 raft_consensus.cc:740] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.214152 31273 consensus_queue.cc:260] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.216820 31273 ts_tablet_manager.cc:1434] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:59.217713 31273 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:09:59.226171 30462 consensus_peers.cc:597] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:59.271979 31273 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.273154 31273 tablet_bootstrap.cc:492] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.273952 31273 ts_tablet_manager.cc:1403] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.012s
I20260709 13:09:59.276523 31273 raft_consensus.cc:359] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.277264 31273 raft_consensus.cc:740] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.278090 31273 consensus_queue.cc:260] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.280085 31273 ts_tablet_manager.cc:1434] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:59.280897 31273 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.298915 31273 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.301241 31273 tablet_bootstrap.cc:492] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.301929 31273 ts_tablet_manager.cc:1403] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:09:59.302240 31375 raft_consensus.cc:3060] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.304399 31273 raft_consensus.cc:359] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.305464 31273 raft_consensus.cc:740] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.306494 31273 consensus_queue.cc:260] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.309553 31273 ts_tablet_manager.cc:1434] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:09:59.313632 31273 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.371687 31273 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.372709 31273 tablet_bootstrap.cc:492] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.373436 31273 ts_tablet_manager.cc:1403] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.060s	user 0.027s	sys 0.026s
I20260709 13:09:59.375950 31273 raft_consensus.cc:359] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.376585 31273 raft_consensus.cc:740] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.377283 31273 consensus_queue.cc:260] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:09:59.379482 31273 ts_tablet_manager.cc:1434] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:09:59.380595 31273 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.399438 31273 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.400445 31273 tablet_bootstrap.cc:492] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.401212 31273 ts_tablet_manager.cc:1403] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.001s
I20260709 13:09:59.403985 31273 raft_consensus.cc:359] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.404770 31273 raft_consensus.cc:740] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.405730 31273 consensus_queue.cc:260] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.408085 31273 ts_tablet_manager.cc:1434] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:09:59.424832 31273 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.475469 31273 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.476606 31273 tablet_bootstrap.cc:492] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.478377 31273 ts_tablet_manager.cc:1403] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.054s	user 0.020s	sys 0.009s
I20260709 13:09:59.481637 31273 raft_consensus.cc:359] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.483222 31273 raft_consensus.cc:740] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.484078 31273 consensus_queue.cc:260] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.486649 31273 ts_tablet_manager.cc:1434] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:59.487484 31377 raft_consensus.cc:3060] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.492026 31273 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.542881 31273 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.543999 31273 tablet_bootstrap.cc:492] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.544860 31273 ts_tablet_manager.cc:1403] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.003s
I20260709 13:09:59.548372 31273 raft_consensus.cc:359] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.549475 31273 raft_consensus.cc:740] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.550335 31273 consensus_queue.cc:260] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.552361 31273 ts_tablet_manager.cc:1434] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:59.553601 31273 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.742967 31273 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.744318 31273 tablet_bootstrap.cc:492] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.745262 31273 ts_tablet_manager.cc:1403] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.192s	user 0.050s	sys 0.103s
I20260709 13:09:59.748422 31273 raft_consensus.cc:359] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:59.749277 31273 raft_consensus.cc:740] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.750258 31273 consensus_queue.cc:260] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:59.755077 31273 ts_tablet_manager.cc:1434] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:09:59.757438 31273 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.776870 31377 raft_consensus.cc:3060] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.836982 31273 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.838451 31273 tablet_bootstrap.cc:492] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.839346 31273 ts_tablet_manager.cc:1403] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.082s	user 0.044s	sys 0.024s
I20260709 13:09:59.843084 31273 raft_consensus.cc:359] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:59.843990 31273 raft_consensus.cc:740] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.844856 31273 consensus_queue.cc:260] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:09:59.850981 31273 ts_tablet_manager.cc:1434] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:09:59.852145 31273 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:09:59.895905 31383 raft_consensus.cc:3060] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:59.902833 30536 consensus_peers.cc:597] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:59.913589 31375 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:59.927628 30536 consensus_peers.cc:597] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:59.964986 31273 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:59.966342 31273 tablet_bootstrap.cc:492] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:09:59.967247 31273 ts_tablet_manager.cc:1403] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.115s	user 0.026s	sys 0.032s
I20260709 13:09:59.972297 31273 raft_consensus.cc:359] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.973282 31273 raft_consensus.cc:740] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:09:59.974164 31273 consensus_queue.cc:260] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:09:59.975989 31273 ts_tablet_manager.cc:1434] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:59.977028 31273 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:10:00.040416 31273 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:00.041666 31273 tablet_bootstrap.cc:492] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:10:00.042527 31273 ts_tablet_manager.cc:1403] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.026s
I20260709 13:10:00.045934 31377 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:00.045727 31273 raft_consensus.cc:359] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:00.046681 31273 raft_consensus.cc:740] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:10:00.047783 31273 consensus_queue.cc:260] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:00.050032 31273 ts_tablet_manager.cc:1434] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:00.051055 31273 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:10:00.081696 31273 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:00.083401 31273 tablet_bootstrap.cc:492] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:10:00.087713 31273 ts_tablet_manager.cc:1403] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.001s
I20260709 13:10:00.094448 31273 raft_consensus.cc:359] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:00.095348 31273 raft_consensus.cc:740] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:10:00.096469 31273 consensus_queue.cc:260] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:00.098954 31273 ts_tablet_manager.cc:1434] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:10:00.100003 31273 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
W20260709 13:10:00.111195 30536 consensus_peers.cc:597] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:00.120285 30462 consensus_peers.cc:597] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:00.125362 31273 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:00.126857 31273 tablet_bootstrap.cc:492] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:10:00.127750 31273 ts_tablet_manager.cc:1403] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.028s	user 0.025s	sys 0.000s
I20260709 13:10:00.136138 31273 raft_consensus.cc:359] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:00.137130 31273 raft_consensus.cc:740] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:10:00.138101 31273 consensus_queue.cc:260] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:00.140373 31273 ts_tablet_manager.cc:1434] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:10:00.141389 31273 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:10:00.177584 31273 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:00.178834 31273 tablet_bootstrap.cc:492] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:10:00.179721 31273 ts_tablet_manager.cc:1403] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.008s
I20260709 13:10:00.182689 31273 raft_consensus.cc:359] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:00.183475 31273 raft_consensus.cc:740] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:10:00.184391 31273 consensus_queue.cc:260] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:00.188499 31273 ts_tablet_manager.cc:1434] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:00.189767 31273 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:10:00.222615 31273 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:00.226543 31273 tablet_bootstrap.cc:492] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:10:00.231853 31273 ts_tablet_manager.cc:1403] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.042s	user 0.029s	sys 0.003s
I20260709 13:10:00.235356 31273 raft_consensus.cc:359] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:00.236158 31273 raft_consensus.cc:740] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:10:00.237099 31273 consensus_queue.cc:260] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:00.246462 31273 ts_tablet_manager.cc:1434] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:10:00.247684 31273 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap starting.
I20260709 13:10:00.289646 31273 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:00.291404 31273 tablet_bootstrap.cc:492] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Bootstrap complete.
I20260709 13:10:00.292533 31273 ts_tablet_manager.cc:1403] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Time spent bootstrapping tablet: real 0.045s	user 0.027s	sys 0.007s
I20260709 13:10:00.296475 31273 raft_consensus.cc:359] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:00.297482 31273 raft_consensus.cc:740] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Initialized, Role: FOLLOWER
I20260709 13:10:00.298354 31273 consensus_queue.cc:260] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:00.307991 31273 ts_tablet_manager.cc:1434] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:10:00.390353 31377 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.301915 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:08.362092 30567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:08.363056 30567 raft_consensus.cc:606] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Received request to transfer leadership to TS 19b1445cece541d3b2abe177283fef44
I20260709 13:10:08.378719 30490 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "730e52dc15a842e8a606385755e5558e"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:08.379221 30490 raft_consensus.cc:606] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:08.390857 30637 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee"
dest_uuid: "b897660686b3465390a5bad32efea13c"
mode: GRACEFUL
new_leader_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:47154
I20260709 13:10:08.391539 30637 raft_consensus.cc:606] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Received request to transfer leadership to TS 19b1445cece541d3b2abe177283fef44
I20260709 13:10:08.394572 30490 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5b071d1d18845b8828943fac633def7"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:08.395262 30490 raft_consensus.cc:606] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:08.397994 30567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:08.398633 30567 raft_consensus.cc:606] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Received request to transfer leadership to TS 19b1445cece541d3b2abe177283fef44
I20260709 13:10:08.402545 30567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:08.403115 30567 raft_consensus.cc:606] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:08.405525 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:08.409209 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:08.524879 31494 raft_consensus.cc:993] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:08.525629 31502 raft_consensus.cc:1081] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:08.527268 31376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:10:08.527745 31376 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.528034 31376 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.534854 31376 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:08.537974 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:08.538888 30570 raft_consensus.cc:3055] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.539259 30570 raft_consensus.cc:740] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:08.539932 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:08.540521 30637 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.540164 30570 consensus_queue.cc:260] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:08.541569 30570 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.537391 31376 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:08.547971 30637 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:08.549620 31268 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:08.550101 30570 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:08.551668 31510 raft_consensus.cc:2804] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.552865 31510 raft_consensus.cc:697] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:08.553920 31510 consensus_queue.cc:237] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:08.565868 30317 catalog_manager.cc:5697] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:08.726953 31489 raft_consensus.cc:993] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:08.727584 31504 raft_consensus.cc:1081] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:08.729525 31376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5b071d1d18845b8828943fac633def7"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:10:08.730089 31376 raft_consensus.cc:493] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.730508 31376 raft_consensus.cc:3060] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.730909 31489 raft_consensus.cc:993] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:08.731389 31504 raft_consensus.cc:1081] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:08.733192 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "730e52dc15a842e8a606385755e5558e"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:10:08.733667 31383 raft_consensus.cc:493] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.734011 31383 raft_consensus.cc:3060] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.737211 31376 raft_consensus.cc:515] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:08.738878 31376 leader_election.cc:290] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:08.739552 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:08.740159 30637 raft_consensus.cc:3060] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.740487 30490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b071d1d18845b8828943fac633def7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:08.741168 30490 raft_consensus.cc:3055] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.741459 30490 raft_consensus.cc:740] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:08.740368 31383 raft_consensus.cc:515] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:08.742406 30490 consensus_queue.cc:260] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:08.743551 30490 raft_consensus.cc:3060] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.744019 31383 leader_election.cc:290] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:10:08.745695 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "730e52dc15a842e8a606385755e5558e" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:08.746241 30491 raft_consensus.cc:3055] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.746503 30491 raft_consensus.cc:740] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:08.747272 30491 consensus_queue.cc:260] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:08.748052 30637 raft_consensus.cc:2468] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:08.748404 30491 raft_consensus.cc:3060] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.749344 31268 leader_election.cc:304] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:08.749635 30567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "730e52dc15a842e8a606385755e5558e" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:08.750231 30567 raft_consensus.cc:3060] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.750227 31510 raft_consensus.cc:2804] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.750974 31510 raft_consensus.cc:697] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:08.751832 31510 consensus_queue.cc:237] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:08.751852 30490 raft_consensus.cc:2468] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:08.755417 30491 raft_consensus.cc:2468] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:08.759210 30567 raft_consensus.cc:2468] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:08.760560 31270 leader_election.cc:304] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:10:08.762526 31510 raft_consensus.cc:2804] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.763016 31510 raft_consensus.cc:697] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:08.763871 31510 consensus_queue.cc:237] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:08.773746 31494 raft_consensus.cc:993] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: : Instructing follower 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:08.774353 31502 raft_consensus.cc:1081] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Signalling peer 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:08.775853 30491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:43876
I20260709 13:10:08.776345 30491 raft_consensus.cc:493] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.776705 30491 raft_consensus.cc:3060] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.776480 30317 catalog_manager.cc:5697] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.780812 30317 catalog_manager.cc:5697] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.783272 30491 raft_consensus.cc:515] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.786739 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:08.787350 30570 raft_consensus.cc:3055] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.787684 30570 raft_consensus.cc:740] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:08.788419 30570 consensus_queue.cc:260] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.789556 30570 raft_consensus.cc:3060] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.788863 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9c275f1b334c7db06956d0941b04ed" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:08.790611 30637 raft_consensus.cc:3060] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.796037 30570 raft_consensus.cc:2468] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:08.786741 30491 leader_election.cc:290] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:08.797359 30463 leader_election.cc:304] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:10:08.798228 31504 raft_consensus.cc:2804] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.798659 31504 raft_consensus.cc:697] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:08.799525 31504 consensus_queue.cc:237] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.802820 30637 raft_consensus.cc:2468] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:08.810791 30317 catalog_manager.cc:5697] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.858625 31494 raft_consensus.cc:993] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: : Instructing follower 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:08.859108 31494 raft_consensus.cc:1081] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Signalling peer 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:08.860836 30490 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:43876
I20260709 13:10:08.865252 30490 raft_consensus.cc:493] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.865701 30490 raft_consensus.cc:3060] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.871413 30490 raft_consensus.cc:515] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.874276 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:08.873732 30490 leader_election.cc:290] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:08.874922 30570 raft_consensus.cc:3055] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.875316 30570 raft_consensus.cc:740] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:08.876226 30570 consensus_queue.cc:260] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.877417 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37455107e57b47c8978a434d17f3c76c" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:08.879776 30637 raft_consensus.cc:3060] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.877667 30570 raft_consensus.cc:3060] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.886338 30637 raft_consensus.cc:2468] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:08.887530 30570 raft_consensus.cc:2468] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:08.888197 30461 leader_election.cc:304] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:08.889326 31504 raft_consensus.cc:2804] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.905599 31504 raft_consensus.cc:697] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:08.906678 31504 consensus_queue.cc:237] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.919667 30317 catalog_manager.cc:5697] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.941327 31500 raft_consensus.cc:993] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: : Instructing follower 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:08.941910 31509 raft_consensus.cc:1081] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Signalling peer 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:08.943550 30490 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:43862
I20260709 13:10:08.944087 30490 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.944377 30490 raft_consensus.cc:3060] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.950989 30490 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.952625 30490 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:08.953313 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:08.953894 30570 raft_consensus.cc:3060] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.953768 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5ed6aaa36704114809147b7efabcaee" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:08.954468 30637 raft_consensus.cc:3055] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.954684 30637 raft_consensus.cc:740] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:10:08.955582 30637 consensus_queue.cc:260] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.957041 30637 raft_consensus.cc:3060] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.960475 30570 raft_consensus.cc:2468] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:08.962361 30463 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:10:08.963792 31504 raft_consensus.cc:2804] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.964318 31504 raft_consensus.cc:697] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:08.965366 31504 consensus_queue.cc:237] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:08.966674 30637 raft_consensus.cc:2468] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:08.968003 30570 raft_consensus.cc:1275] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.969413 31510 consensus_queue.cc:1048] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.981355 30637 raft_consensus.cc:1275] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.980840 30317 catalog_manager.cc:5697] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from b897660686b3465390a5bad32efea13c (127.20.66.132) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.982685 31510 consensus_queue.cc:1048] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:09.218559 30570 raft_consensus.cc:1275] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.220060 31511 consensus_queue.cc:1048] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:09.229377 30490 raft_consensus.cc:1275] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.231153 31511 consensus_queue.cc:1048] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:09.292297 30567 raft_consensus.cc:1275] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.293373 30637 raft_consensus.cc:1275] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.293735 31489 consensus_queue.cc:1048] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:09.294647 31511 consensus_queue.cc:1048] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:09.302538 30643 raft_consensus.cc:1275] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.304015 31504 consensus_queue.cc:1048] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:09.332566 30493 raft_consensus.cc:1275] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.334477 31530 consensus_queue.cc:1048] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:09.340569 30643 raft_consensus.cc:1275] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.342679 31520 consensus_queue.cc:1048] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:10:09.371024 30567 raft_consensus.cc:1275] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.385066 31504 consensus_queue.cc:1048] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:09.410383 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:09.444761 30643 raft_consensus.cc:1275] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.447539 31540 consensus_queue.cc:1048] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:09.461637 30570 raft_consensus.cc:1275] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:09.464450 31540 consensus_queue.cc:1048] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:09.480324 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:09.523156 30570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:09.523792 30570 raft_consensus.cc:606] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Received request to transfer leadership to TS 19b1445cece541d3b2abe177283fef44
I20260709 13:10:09.526772 30493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:09.527348 30493 raft_consensus.cc:606] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:09.529865 30493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:09.530426 30493 raft_consensus.cc:606] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:09.532853 30567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4871109603643fe996c7bfd323672c7"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:09.533425 30567 raft_consensus.cc:606] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:09.535952 30493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "b897660686b3465390a5bad32efea13c"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:09.536410 30493 raft_consensus.cc:606] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Received request to transfer leadership to TS b897660686b3465390a5bad32efea13c
I20260709 13:10:09.538885 30643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313"
dest_uuid: "b897660686b3465390a5bad32efea13c"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:47154
I20260709 13:10:09.539572 30643 raft_consensus.cc:606] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:09.540937 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:09.542649 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:09.631307 31540 raft_consensus.cc:993] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:09.631950 31504 raft_consensus.cc:1081] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:09.633592 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:10:09.634168 31383 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:09.634491 31383 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:09.639521 31383 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:09.641305 31383 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:09.641968 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:09.642618 30643 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:09.642539 30493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a97b4e180f4b618f578bfcc4d841a1" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:09.643349 30493 raft_consensus.cc:3055] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:09.643663 30493 raft_consensus.cc:740] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:09.644572 30493 consensus_queue.cc:260] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:09.646004 30493 raft_consensus.cc:3060] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:09.658288 30643 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:09.659490 30493 raft_consensus.cc:2468] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:09.661664 31268 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:09.662580 31510 raft_consensus.cc:2804] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:09.663166 31510 raft_consensus.cc:697] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:09.664078 31510 consensus_queue.cc:237] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:09.692840 31509 raft_consensus.cc:993] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:09.693485 31509 raft_consensus.cc:1081] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:09.719213 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57316
I20260709 13:10:09.720322 31383 raft_consensus.cc:493] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:09.720727 31383 raft_consensus.cc:3060] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.737898 31383 raft_consensus.cc:515] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:09.740345 31383 leader_election.cc:290] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:09.800478 30490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:09.801211 30490 raft_consensus.cc:3060] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.819099 30490 raft_consensus.cc:2468] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:09.820334 31270 leader_election.cc:304] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:10:09.821326 31511 raft_consensus.cc:2804] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:09.821985 31511 raft_consensus.cc:697] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:09.823027 31511 consensus_queue.cc:237] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:09.828649 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb6f3b113b34e3c8bce234b33434313" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:09.829324 30643 raft_consensus.cc:3055] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:09.829614 30643 raft_consensus.cc:740] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:10:09.830292 30643 consensus_queue.cc:260] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:09.831132 30643 raft_consensus.cc:3060] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.841569 30643 raft_consensus.cc:2468] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:09.853406 31541 raft_consensus.cc:993] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: : Instructing follower b897660686b3465390a5bad32efea13c to start an election
I20260709 13:10:09.854043 31504 raft_consensus.cc:1081] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Signalling peer b897660686b3465390a5bad32efea13c to start an election
I20260709 13:10:09.859228 30636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389"
dest_uuid: "b897660686b3465390a5bad32efea13c"
 from {username='slave'} at 127.0.0.1:51026
I20260709 13:10:09.859845 30636 raft_consensus.cc:493] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:09.860281 30636 raft_consensus.cc:3060] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.863670 31526 raft_consensus.cc:993] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: : Instructing follower 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:09.864379 31549 raft_consensus.cc:1081] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Signalling peer 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:09.871402 30636 raft_consensus.cc:515] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:09.874231 30636 leader_election.cc:290] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Requested vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:10:09.880486 30490 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:43876
I20260709 13:10:09.881141 30490 raft_consensus.cc:493] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:09.881572 30490 raft_consensus.cc:3060] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.882961 31549 raft_consensus.cc:993] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:09.883589 31502 raft_consensus.cc:1081] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:09.886785 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4871109603643fe996c7bfd323672c7"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:10:09.887406 31378 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:09.887794 31378 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:09.895965 30490 raft_consensus.cc:515] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:09.901881 30490 leader_election.cc:290] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:09.902202 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:09.902861 30570 raft_consensus.cc:3055] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:09.903139 30570 raft_consensus.cc:740] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:09.904223 30570 consensus_queue.cc:260] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:09.905370 30570 raft_consensus.cc:3060] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.905537 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4b70f2baf64ae8a943a8bb8aa24385" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:09.906280 30636 raft_consensus.cc:3060] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.913528 30565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:09.914302 30565 raft_consensus.cc:3060] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.920362 30636 raft_consensus.cc:2468] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:09.921168 30637 raft_consensus.cc:1240] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Rejecting Update request from peer 955d2e588381448981f85fadf225aa05 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:09.924444 31378 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:09.927655 31378 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 4 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:09.932351 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:09.933069 30564 raft_consensus.cc:3055] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:09.933216 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:09.933490 30564 raft_consensus.cc:740] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:09.933970 30641 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:09.934532 30564 consensus_queue.cc:260] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:09.939003 30564 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:09.959861 30565 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
I20260709 13:10:09.961488 30611 leader_election.cc:304] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:09.961544 30320 catalog_manager.cc:5697] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 2 to 3, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 3 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:09.962599 31551 raft_consensus.cc:2804] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:09.965068 31551 raft_consensus.cc:697] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Becoming Leader. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:10:09.966293 31551 consensus_queue.cc:237] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:09.969758 30461 leader_election.cc:304] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:09.971328 31520 raft_consensus.cc:2804] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:09.971921 31520 raft_consensus.cc:697] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:09.975847 30490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78ade21dd6d46a3a18152943df69389" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:09.975694 31520 consensus_queue.cc:237] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:09.976564 30490 raft_consensus.cc:3055] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:09.976935 30490 raft_consensus.cc:740] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:09.981086 30570 raft_consensus.cc:2468] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:09.983650 30641 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 4.
I20260709 13:10:09.986271 30564 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 4.
I20260709 13:10:09.989174 30490 consensus_queue.cc:260] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:09.991283 30490 raft_consensus.cc:3060] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.992121 31270 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:10:09.993589 31530 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:09.994227 31530 raft_consensus.cc:697] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 4 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:09.995368 31530 consensus_queue.cc:237] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:10:09.999274 30610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 61339.1us
I20260709 13:10:10.025641 30490 raft_consensus.cc:2468] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897660686b3465390a5bad32efea13c in term 2.
I20260709 13:10:10.047196 30317 catalog_manager.cc:5697] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from b897660686b3465390a5bad32efea13c (127.20.66.132) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:10.050271 30317 catalog_manager.cc:5697] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 3 to 4, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 4 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.029695 30320 catalog_manager.cc:5697] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c reported cstate change: term changed from 1 to 2, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to b897660686b3465390a5bad32efea13c (127.20.66.132). New cstate: current_term: 2 leader_uuid: "b897660686b3465390a5bad32efea13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.086283 30317 catalog_manager.cc:5697] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:10.143872 31520 raft_consensus.cc:993] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:10.144411 31541 raft_consensus.cc:1081] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:10.145839 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:10:10.146360 31383 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:10.146612 31383 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.151430 31383 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:10.153028 31383 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:10.153919 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:10.154172 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44a0e227c0ca4247b0ba8fbf2d005fec" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:10.154610 30643 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.154817 30491 raft_consensus.cc:3055] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:10.155208 30491 raft_consensus.cc:740] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:10.156028 30491 consensus_queue.cc:260] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:10.157467 30491 raft_consensus.cc:3060] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.159053 30490 raft_consensus.cc:1275] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:10.161762 30643 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:10.162081 31510 consensus_queue.cc:1048] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:10.162945 31268 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:10.163692 31510 raft_consensus.cc:2804] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:10.164144 31510 raft_consensus.cc:697] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:10.164363 30491 raft_consensus.cc:2468] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:10.164983 31510 consensus_queue.cc:237] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:10.175726 30643 raft_consensus.cc:1275] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:10.177511 31511 consensus_queue.cc:1048] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:10.178340 30320 catalog_manager.cc:5697] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 2 to 3, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 3 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:10.350797 30490 raft_consensus.cc:1275] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:10.352083 31510 consensus_queue.cc:1048] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.363082 30641 raft_consensus.cc:1275] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:10.364462 31511 consensus_queue.cc:1048] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.391691 30641 raft_consensus.cc:1275] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:10.393160 31520 consensus_queue.cc:1048] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.402714 30570 raft_consensus.cc:1275] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:10.404021 31540 consensus_queue.cc:1048] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.409412 30564 raft_consensus.cc:1275] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:10.412071 31552 consensus_queue.cc:1048] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:10.437222 30490 raft_consensus.cc:1275] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer b897660686b3465390a5bad32efea13c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:10.441815 31551 consensus_queue.cc:1048] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.445166 30564 raft_consensus.cc:1275] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:10.446815 31511 consensus_queue.cc:1048] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:10.475203 30641 raft_consensus.cc:1275] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:10.478108 31530 consensus_queue.cc:1048] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:10.657660 30490 raft_consensus.cc:1275] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:10.658885 31530 consensus_queue.cc:1048] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:10.668890 30643 raft_consensus.cc:1275] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:10.670522 31510 consensus_queue.cc:1048] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:10.693343 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:10.745806 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:10.801203 30564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:10.801870 30564 raft_consensus.cc:606] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:10.804970 30564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:10.805606 30564 raft_consensus.cc:606] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:10.808324 30643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba"
dest_uuid: "b897660686b3465390a5bad32efea13c"
mode: GRACEFUL
new_leader_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:47154
I20260709 13:10:10.808882 30643 raft_consensus.cc:606] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Received request to transfer leadership to TS 19b1445cece541d3b2abe177283fef44
I20260709 13:10:10.811865 30564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9"
dest_uuid: "955d2e588381448981f85fadf225aa05"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:10:10.812513 30564 raft_consensus.cc:606] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:10.815167 30490 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:10.815729 30490 raft_consensus.cc:606] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:10.817026 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:10.819088 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:10.825740 31502 raft_consensus.cc:993] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:10.826582 31502 raft_consensus.cc:1081] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:10.828562 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:10:10.829211 31378 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:10.829614 31378 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.837617 31378 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:10.839978 31378 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:10.841279 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:10.841763 30564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca8dcce43e444f8ada68dae5eecc1f6" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:10.842289 30643 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.842540 30564 raft_consensus.cc:3055] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:10.842939 30564 raft_consensus.cc:740] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:10.844398 30564 consensus_queue.cc:260] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:10.845827 30564 raft_consensus.cc:3060] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.849591 30643 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:10.850908 31268 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:10.851891 31530 raft_consensus.cc:2804] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:10.852662 31530 raft_consensus.cc:697] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:10.853559 31530 consensus_queue.cc:237] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:10.859193 30564 raft_consensus.cc:2468] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:10.872633 30317 catalog_manager.cc:5697] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 2 to 3, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 3 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:10.931294 30666 debug-util.cc:398] Leaking SignalData structure 0x7b08001ddb40 after lost signal to thread 20749
W20260709 13:10:10.932881 30666 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca440 after lost signal to thread 30354
W20260709 13:10:10.934618 30666 debug-util.cc:398] Leaking SignalData structure 0x7b080032eb20 after lost signal to thread 30521
W20260709 13:10:10.936182 30666 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5bc0 after lost signal to thread 30595
W20260709 13:10:10.937459 30666 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf280 after lost signal to thread 30669
W20260709 13:10:10.939098 30666 debug-util.cc:398] Leaking SignalData structure 0x7b08001dc0a0 after lost signal to thread 31414
I20260709 13:10:11.137336 31494 raft_consensus.cc:993] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:11.139096 31581 raft_consensus.cc:1081] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:11.143244 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:10:11.144043 31383 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.144445 31383 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.146284 31552 raft_consensus.cc:993] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: : Instructing follower 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:11.146951 31552 raft_consensus.cc:1081] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Signalling peer 19b1445cece541d3b2abe177283fef44 to start an election
I20260709 13:10:11.151918 30491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
 from {username='slave'} at 127.0.0.1:43862
I20260709 13:10:11.152673 30491 raft_consensus.cc:493] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.153124 30491 raft_consensus.cc:3060] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.177212 31502 raft_consensus.cc:993] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:11.177983 31502 raft_consensus.cc:1081] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:11.180333 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:10:11.181140 31378 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.180828 30491 raft_consensus.cc:515] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:11.181659 31378 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:11.185081 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:11.185889 30636 raft_consensus.cc:3055] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:11.186209 30636 raft_consensus.cc:740] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:10:11.187155 30636 consensus_queue.cc:260] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:11.188438 30636 raft_consensus.cc:3060] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.190147 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0df1d226d24b358d6d328eb07552ba" candidate_uuid: "19b1445cece541d3b2abe177283fef44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:11.190841 30570 raft_consensus.cc:3060] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.193934 30491 leader_election.cc:290] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:11.218984 31541 raft_consensus.cc:993] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:11.219803 31541 raft_consensus.cc:1081] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:11.222251 31376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:10:11.217916 30636 raft_consensus.cc:2468] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:11.230662 30461 leader_election.cc:304] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:11.231278 31378 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:11.231914 31580 raft_consensus.cc:2804] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:11.232959 31580 raft_consensus.cc:697] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Becoming Leader. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:11.234076 31580 consensus_queue.cc:237] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:11.218926 30570 raft_consensus.cc:2468] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b1445cece541d3b2abe177283fef44 in term 2.
I20260709 13:10:11.228660 31383 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:11.235451 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:11.222895 31376 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.237138 30636 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:11.237334 31376 raft_consensus.cc:3060] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.234159 31378 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:11.245313 30565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:11.246120 30565 raft_consensus.cc:3055] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:11.246505 30565 raft_consensus.cc:740] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:11.246628 30636 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:11.248175 31268 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:11.249361 31510 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:11.249959 31510 raft_consensus.cc:697] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:11.251121 31510 consensus_queue.cc:237] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:11.247907 30565 consensus_queue.cc:260] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:11.265204 30565 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:11.262120 30643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:11.273818 30643 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.284526 30318 catalog_manager.cc:5697] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 2 to 3, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 3 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:11.288077 30643 raft_consensus.cc:2468] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:11.289664 31268 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:11.263808 31383 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:11.290681 31530 raft_consensus.cc:2804] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:11.291281 31530 raft_consensus.cc:697] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:11.292387 31530 consensus_queue.cc:237] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:11.294715 31376 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:11.259547 30570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:11.295887 30570 raft_consensus.cc:3055] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:11.296234 30570 raft_consensus.cc:740] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:11.297977 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:11.299299 30636 raft_consensus.cc:3060] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.310333 30565 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 3.
I20260709 13:10:11.313897 31376 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:11.313964 30570 consensus_queue.cc:260] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:11.320411 30570 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.321774 30636 raft_consensus.cc:2468] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:11.323808 31268 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:11.325066 31510 raft_consensus.cc:2804] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:11.325691 31510 raft_consensus.cc:697] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:11.326864 31510 consensus_queue.cc:237] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:11.332413 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ea982486944f2fbb7ce66208002c94" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:11.333621 30491 raft_consensus.cc:3055] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:11.334013 30491 raft_consensus.cc:740] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:11.343415 30570 raft_consensus.cc:2468] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:11.348459 30491 consensus_queue.cc:260] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:11.350373 30491 raft_consensus.cc:3060] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.360760 30318 catalog_manager.cc:5697] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 reported cstate change: term changed from 1 to 2, leader changed from b897660686b3465390a5bad32efea13c (127.20.66.132) to 19b1445cece541d3b2abe177283fef44 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "19b1445cece541d3b2abe177283fef44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:11.393720 30636 raft_consensus.cc:1275] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:11.399358 31530 consensus_queue.cc:1048] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:10:11.414176 30317 catalog_manager.cc:5697] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from 955d2e588381448981f85fadf225aa05 (127.20.66.131) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:11.443753 30491 raft_consensus.cc:2468] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:11.455560 30570 raft_consensus.cc:1275] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:11.460630 31510 consensus_queue.cc:1048] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:11.465358 30317 catalog_manager.cc:5697] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:11.684381 30570 raft_consensus.cc:1275] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.684742 30564 raft_consensus.cc:1275] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.687155 31510 consensus_queue.cc:1048] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.689394 31580 consensus_queue.cc:1048] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:11.711921 30636 raft_consensus.cc:1275] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.713716 31586 consensus_queue.cc:1048] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.722329 30643 raft_consensus.cc:1275] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 19b1445cece541d3b2abe177283fef44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.723861 31541 consensus_queue.cc:1048] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.739717 30570 raft_consensus.cc:1275] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:11.741554 31586 consensus_queue.cc:1048] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:11.764705 30643 raft_consensus.cc:1275] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.780815 31530 consensus_queue.cc:1048] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:10:11.820246 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:11.822104 30643 raft_consensus.cc:1275] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:11.839360 31511 consensus_queue.cc:1048] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:10:11.886158 30491 raft_consensus.cc:1275] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.910065 31608 consensus_queue.cc:1048] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.941821 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:11.952881 30536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:34915, user_credentials={real_user=slave}} blocked reactor thread for 58057.3us
W20260709 13:10:12.071597 30463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:36261, user_credentials={real_user=slave}} blocked reactor thread for 53792.7us
I20260709 13:10:12.105377 30490 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14"
dest_uuid: "19b1445cece541d3b2abe177283fef44"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:36650
I20260709 13:10:12.105997 30490 raft_consensus.cc:606] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:12.143534 30637 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf"
dest_uuid: "b897660686b3465390a5bad32efea13c"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:47154
I20260709 13:10:12.144165 30637 raft_consensus.cc:606] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:12.159665 30637 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90"
dest_uuid: "b897660686b3465390a5bad32efea13c"
mode: GRACEFUL
new_leader_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:47154
I20260709 13:10:12.160248 30637 raft_consensus.cc:606] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Received request to transfer leadership to TS 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:12.161677 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:10:12.167152 31600 raft_consensus.cc:993] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:12.167696 31600 raft_consensus.cc:1081] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:12.168452 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:12.171990 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57316
I20260709 13:10:12.172610 31383 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.172945 31383 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.187290 31383 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:12.189448 31383 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), b897660686b3465390a5bad32efea13c (127.20.66.132:45099)
I20260709 13:10:12.190452 30565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955d2e588381448981f85fadf225aa05"
I20260709 13:10:12.191143 30565 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.193567 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:12.194866 30636 raft_consensus.cc:3055] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:12.195178 30636 raft_consensus.cc:740] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:10:12.196244 30636 consensus_queue.cc:260] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:12.197577 30636 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.198814 30565 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:12.200292 31270 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05; no voters: 
I20260709 13:10:12.201365 31511 raft_consensus.cc:2804] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:12.201969 31511 raft_consensus.cc:697] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:12.202831 31511 consensus_queue.cc:237] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:12.234205 30320 catalog_manager.cc:5697] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from b897660686b3465390a5bad32efea13c (127.20.66.132) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:12.245769 30636 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:12.254138 31600 raft_consensus.cc:993] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:12.254803 31600 raft_consensus.cc:1081] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:12.261662 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57316
I20260709 13:10:12.262279 31378 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.262686 31378 raft_consensus.cc:3060] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.273564 31378 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:12.301317 31378 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:12.302656 30491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:12.303369 30491 raft_consensus.cc:3060] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.311092 30491 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:12.311640 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59eef71464c40618e5e26de6a40aa90" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:12.312399 30636 raft_consensus.cc:3055] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:12.312680 31270 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1; no voters: 
I20260709 13:10:12.313778 31511 raft_consensus.cc:2804] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:12.314316 31511 raft_consensus.cc:697] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:12.312727 30636 raft_consensus.cc:740] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b897660686b3465390a5bad32efea13c, State: Running, Role: LEADER
I20260709 13:10:12.315295 31511 consensus_queue.cc:237] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:12.315855 30636 consensus_queue.cc:260] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:12.317335 30636 raft_consensus.cc:3060] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.325723 30636 raft_consensus.cc:2468] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:12.380059 30317 catalog_manager.cc:5697] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from b897660686b3465390a5bad32efea13c (127.20.66.132) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.674260 31602 raft_consensus.cc:993] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: : Instructing follower 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:12.675303 31541 raft_consensus.cc:1081] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Signalling peer 5825cf9d14b4469c9960cca89b0010f1 to start an election
I20260709 13:10:12.678313 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14"
dest_uuid: "5825cf9d14b4469c9960cca89b0010f1"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:10:12.679165 31378 raft_consensus.cc:493] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.679620 31378 raft_consensus.cc:3060] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.691457 31378 raft_consensus.cc:515] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:12.695998 30636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:12.696743 30636 raft_consensus.cc:3060] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.696949 30490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726e825b67364fccb3b417c19afe5e14" candidate_uuid: "5825cf9d14b4469c9960cca89b0010f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19b1445cece541d3b2abe177283fef44"
I20260709 13:10:12.697675 30490 raft_consensus.cc:3055] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:12.698005 30490 raft_consensus.cc:740] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19b1445cece541d3b2abe177283fef44, State: Running, Role: LEADER
I20260709 13:10:12.699136 30490 consensus_queue.cc:260] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:12.700877 30490 raft_consensus.cc:3060] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.707672 31378 leader_election.cc:290] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:12.717808 30636 raft_consensus.cc:2468] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:12.720141 31268 leader_election.cc:304] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5825cf9d14b4469c9960cca89b0010f1, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:12.721755 31530 raft_consensus.cc:2804] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:12.722276 31530 raft_consensus.cc:697] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Becoming Leader. State: Replica: 5825cf9d14b4469c9960cca89b0010f1, State: Running, Role: LEADER
I20260709 13:10:12.723344 31530 consensus_queue.cc:237] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:12.728259 30636 raft_consensus.cc:1275] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:12.732523 31586 consensus_queue.cc:1048] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:12.759140 30317 catalog_manager.cc:5697] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 reported cstate change: term changed from 1 to 2, leader changed from 19b1445cece541d3b2abe177283fef44 (127.20.66.130) to 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5825cf9d14b4469c9960cca89b0010f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.771302 30490 raft_consensus.cc:2468] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5825cf9d14b4469c9960cca89b0010f1 in term 2.
I20260709 13:10:12.792812 30570 raft_consensus.cc:1275] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:12.795149 31603 consensus_queue.cc:1048] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:12.865149 30490 raft_consensus.cc:1275] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:12.867069 31586 consensus_queue.cc:1048] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:12.885015 30641 raft_consensus.cc:1275] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:12.887514 31610 consensus_queue.cc:1048] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:13.169801 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:13.208246 30490 raft_consensus.cc:1275] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:13.210095 31610 consensus_queue.cc:1048] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:13.229172 30641 raft_consensus.cc:1275] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Refusing update from remote peer 5825cf9d14b4469c9960cca89b0010f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:13.230818 31511 consensus_queue.cc:1048] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:13.268087 20746 tablet_server.cc:179] TabletServer@127.20.66.129:34915 shutting down...
W20260709 13:10:13.271129 30536 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 283 similar messages]
W20260709 13:10:13.275517 30536 consensus_peers.cc:597] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.299098 30610 consensus_peers.cc:597] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.315586 30610 consensus_peers.cc:597] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.326756 30610 consensus_peers.cc:597] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.351878 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:13.353427 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.354653 20746 raft_consensus.cc:2243] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.355947 20746 raft_consensus.cc:2272] T b4a97b4e180f4b618f578bfcc4d841a1 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.360729 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.361946 20746 raft_consensus.cc:2243] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.363373 20746 raft_consensus.cc:2272] T c09468889ceb4dd490e2cbd2c245b3d9 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.367090 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.368119 20746 raft_consensus.cc:2243] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.368822 20746 raft_consensus.cc:2272] T bbc578fa661c4935aae4aece586f4271 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.370797 30462 consensus_peers.cc:597] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.372682 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.374033 20746 raft_consensus.cc:2243] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:10:13.374490 30536 consensus_peers.cc:597] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.375295 20746 raft_consensus.cc:2272] T 2908e68999c34a76b7343444d882b4e9 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.379247 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.380298 20746 raft_consensus.cc:2243] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.380924 20746 raft_consensus.cc:2272] T 679bf6d4a8654a72b47b318645ab6fc0 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.385515 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.386545 20746 raft_consensus.cc:2243] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.387159 20746 raft_consensus.cc:2272] T 95d00bccf5ce41f4b601bba7d235ffbb P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.389465 30462 consensus_peers.cc:597] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.390769 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.391949 20746 raft_consensus.cc:2243] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.392627 20746 raft_consensus.cc:2272] T 24a265170a904a89a2801145c1a61172 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.394793 30536 consensus_peers.cc:597] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.396093 20746 tablet_replica.cc:333] stopping tablet replica
W20260709 13:10:13.396359 30462 consensus_peers.cc:597] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.397145 20746 raft_consensus.cc:2243] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.397830 20746 raft_consensus.cc:2272] T 55b4252704f04cae82d10d8b1bb4c88d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.401197 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.402074 20746 raft_consensus.cc:2243] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.402691 20746 raft_consensus.cc:2272] T 6664fcb5ba4f487b957d5cc98ec559de P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.406240 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.407228 20746 raft_consensus.cc:2243] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.407795 20746 raft_consensus.cc:2272] T 040a6253e94b4c19952ed01cb9d8c6a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.411947 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.413486 20746 raft_consensus.cc:2243] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.414252 20746 raft_consensus.cc:2272] T ea53b9ee66e343b18d301fe650b12554 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.417758 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.418746 20746 raft_consensus.cc:2243] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.419438 20746 raft_consensus.cc:2272] T 9035baa441924d8b9218b26855598f19 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.423866 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.424825 20746 raft_consensus.cc:2243] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.425508 20746 raft_consensus.cc:2272] T 9b1c1fe97f07496c92182b3065536c15 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.429325 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.430284 20746 raft_consensus.cc:2243] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.431505 20746 raft_consensus.cc:2272] T d5b071d1d18845b8828943fac633def7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.435124 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.436178 20746 raft_consensus.cc:2243] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.437430 20746 raft_consensus.cc:2272] T 730e52dc15a842e8a606385755e5558e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.440958 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.441977 20746 raft_consensus.cc:2243] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.443369 20746 raft_consensus.cc:2272] T efba0c8d26eb438ea2ff27b4bb66b20a P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.446936 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.447902 20746 raft_consensus.cc:2243] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.448514 20746 raft_consensus.cc:2272] T ba4cf9aa835b48038aa80c194df56bd6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.452025 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.452931 20746 raft_consensus.cc:2243] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.454134 20746 raft_consensus.cc:2272] T 726e825b67364fccb3b417c19afe5e14 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.459452 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.460351 20746 raft_consensus.cc:2243] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.461508 20746 raft_consensus.cc:2272] T fdb6f3b113b34e3c8bce234b33434313 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.470234 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.471518 20746 raft_consensus.cc:2243] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.472312 20746 raft_consensus.cc:2272] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.473379 30536 consensus_peers.cc:597] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.475575 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.476513 20746 raft_consensus.cc:2243] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.477626 20746 raft_consensus.cc:2272] T b59eef71464c40618e5e26de6a40aa90 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.483222 30610 consensus_peers.cc:597] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.483956 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.485746 20746 raft_consensus.cc:2243] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.486305 20746 raft_consensus.cc:2272] T 6ac46eb166db448cb0c6f504819da525 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.489040 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.489804 20746 raft_consensus.cc:2243] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.490289 20746 raft_consensus.cc:2272] T c69a30e3a4a8473a9f1f8652ea9463d8 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.493270 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.493991 20746 raft_consensus.cc:2243] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.494580 20746 raft_consensus.cc:2272] T a15b8193c82e4caeb06f0da37159665c P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.497843 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.498937 20746 raft_consensus.cc:2243] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.499433 20746 raft_consensus.cc:2272] T 86637a5e467340ae9ac4f57d22c68bf6 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.502230 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.502993 20746 raft_consensus.cc:2243] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.503417 20746 raft_consensus.cc:2272] T 6e52d69d8f9044099d83595faf8a7c88 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.506323 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.507071 20746 raft_consensus.cc:2243] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.507602 20746 raft_consensus.cc:2272] T bccb92fd917f4a97af88be6e7e8fed1f P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.510856 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.514360 20746 raft_consensus.cc:2243] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.515159 20746 raft_consensus.cc:2272] T 78e63f8484d24f8f9bc5c94fef442265 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.519311 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.520260 20746 raft_consensus.cc:2243] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.520897 20746 raft_consensus.cc:2272] T 289ca1e3324041acb6ac3410039210be P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.522957 30610 consensus_peers.cc:597] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.524581 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.525490 20746 raft_consensus.cc:2243] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.527197 20746 raft_consensus.cc:2272] T d4871109603643fe996c7bfd323672c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.530939 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.531510 20746 raft_consensus.cc:2243] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.532052 20746 raft_consensus.cc:2272] T 93a6f0d4b4c848deb76000430c8bb0a5 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.534827 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.535730 20746 raft_consensus.cc:2243] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.536370 20746 raft_consensus.cc:2272] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.539458 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.540294 20746 raft_consensus.cc:2243] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:13.540381 30536 consensus_peers.cc:597] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.541066 20746 raft_consensus.cc:2272] T d204ad91a1c74eb9b3335582ee6717c7 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.544273 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.545118 20746 raft_consensus.cc:2243] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.546021 20746 raft_consensus.cc:2272] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.549098 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.549911 20746 raft_consensus.cc:2243] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.551393 20746 raft_consensus.cc:2272] T 44a0e227c0ca4247b0ba8fbf2d005fec P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.555384 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.556212 20746 raft_consensus.cc:2243] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.557210 20746 raft_consensus.cc:2272] T 6f17363c79974c6a863b51a45eef6e0e P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.560276 30536 consensus_peers.cc:597] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.560491 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.561450 20746 raft_consensus.cc:2243] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.562011 20746 raft_consensus.cc:2272] T 8391e1197cab41039a7daf01b4cfb72b P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.565768 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.566711 20746 raft_consensus.cc:2243] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.567345 20746 raft_consensus.cc:2272] T c678eac1173e4cdaa75a8279bc764f29 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.571077 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.572032 20746 raft_consensus.cc:2243] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.572701 20746 raft_consensus.cc:2272] T 06e3f31e48cd40e890350bcb72063bf7 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.576270 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.577198 20746 raft_consensus.cc:2243] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.577874 20746 raft_consensus.cc:2272] T 47885caab3e14ff2ae88840a9dfc0944 P 5825cf9d14b4469c9960cca89b0010f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.581605 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.582522 20746 raft_consensus.cc:2243] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.583171 20746 raft_consensus.cc:2272] T 1213ccbc173c43f683d18d933293e4de P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.586659 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.587455 20746 raft_consensus.cc:2243] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.588030 20746 raft_consensus.cc:2272] T 366b9fff4017464a9b99ab4878dbb119 P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.591472 30610 consensus_peers.cc:597] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.591624 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.592882 20746 raft_consensus.cc:2243] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.593607 20746 raft_consensus.cc:2272] T 0f3b4d4c8fc44213807c39418459447d P 5825cf9d14b4469c9960cca89b0010f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.597561 20746 tablet_replica.cc:333] stopping tablet replica
W20260709 13:10:13.592947 30610 consensus_peers.cc:597] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.599718 20746 raft_consensus.cc:2243] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:10:13.600162 30462 consensus_peers.cc:597] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.600865 20746 raft_consensus.cc:2272] T d7ea982486944f2fbb7ce66208002c94 P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.600963 30462 consensus_peers.cc:597] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.601758 30536 consensus_peers.cc:597] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.604121 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:13.604991 20746 raft_consensus.cc:2243] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.606199 20746 raft_consensus.cc:2272] T 8b2c6fa20b2b4325804ef77b1d2172bf P 5825cf9d14b4469c9960cca89b0010f1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:13.607132 30536 consensus_peers.cc:597] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.613251 30462 consensus_peers.cc:597] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.616489 30610 consensus_peers.cc:597] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.617318 30610 consensus_peers.cc:597] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.618505 30536 consensus_peers.cc:597] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.635288 30462 consensus_peers.cc:597] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.640466 30610 consensus_peers.cc:597] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.662935 30536 consensus_peers.cc:597] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.669059 30610 consensus_peers.cc:597] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.701232 30536 consensus_peers.cc:597] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.710799 20746 tablet_server.cc:196] TabletServer@127.20.66.129:34915 shutdown complete.
W20260709 13:10:13.730705 30536 consensus_peers.cc:597] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.823779 30610 consensus_peers.cc:597] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c -> Peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Couldn't send request to peer 5825cf9d14b4469c9960cca89b0010f1. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.850725 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
W20260709 13:10:13.871675 30537 consensus_peers.cc:597] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.908854 30611 consensus_peers.cc:597] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.916726 30537 consensus_peers.cc:597] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:13.934424 30537 consensus_peers.cc:597] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:13.961386 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:13.962817 20746 tablet_replica.cc:333] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.964234 20746 raft_consensus.cc:2243] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.964843 20746 raft_consensus.cc:2272] T eba54fb250484a4e8102345d2a02e5cb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.968125 20746 tablet_replica.cc:333] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.974915 20746 raft_consensus.cc:2243] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.975548 20746 raft_consensus.cc:2272] T b4a97b4e180f4b618f578bfcc4d841a1 P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.978324 20746 tablet_replica.cc:333] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.979717 20746 raft_consensus.cc:2243] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.980242 20746 raft_consensus.cc:2272] T 55b4252704f04cae82d10d8b1bb4c88d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.982527 20746 tablet_replica.cc:333] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.983223 20746 raft_consensus.cc:2243] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:13.984037 20746 raft_consensus.cc:2272] T d204ad91a1c74eb9b3335582ee6717c7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.986410 20746 tablet_replica.cc:333] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.987118 20746 raft_consensus.cc:2243] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.987574 20746 raft_consensus.cc:2272] T 0f3b4d4c8fc44213807c39418459447d P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.990144 20746 tablet_replica.cc:333] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.990952 20746 raft_consensus.cc:2243] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.994621 20746 raft_consensus.cc:2272] T 366b9fff4017464a9b99ab4878dbb119 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.998854 20746 tablet_replica.cc:333] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:13.999796 20746 raft_consensus.cc:2243] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.001927 20746 raft_consensus.cc:2272] T 040a6253e94b4c19952ed01cb9d8c6a5 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.006201 30537 consensus_peers.cc:597] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.006372 20746 tablet_replica.cc:333] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.007689 20746 raft_consensus.cc:2243] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.009146 20746 raft_consensus.cc:2272] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.014760 30537 consensus_peers.cc:597] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.014871 20746 tablet_replica.cc:333] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.015967 20746 raft_consensus.cc:2243] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.017227 20746 raft_consensus.cc:2272] T 37455107e57b47c8978a434d17f3c76c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.020529 20746 tablet_replica.cc:333] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.021440 20746 raft_consensus.cc:2243] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:14.026659 30611 consensus_peers.cc:597] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.027724 20746 raft_consensus.cc:2272] T e78ade21dd6d46a3a18152943df69389 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.031167 20746 tablet_replica.cc:333] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
W20260709 13:10:14.033908 30537 consensus_peers.cc:597] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.034673 20746 raft_consensus.cc:2243] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.035773 20746 raft_consensus.cc:2272] T 93a6f0d4b4c848deb76000430c8bb0a5 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.039206 20746 tablet_replica.cc:333] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.040128 20746 raft_consensus.cc:2243] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.040748 20746 raft_consensus.cc:2272] T a15b8193c82e4caeb06f0da37159665c P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.044595 20746 tablet_replica.cc:333] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.046265 20746 raft_consensus.cc:2243] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.047427 20746 raft_consensus.cc:2272] T c69a30e3a4a8473a9f1f8652ea9463d8 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.051425 20746 tablet_replica.cc:333] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.052325 20746 raft_consensus.cc:2243] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.053475 20746 raft_consensus.cc:2272] T 86637a5e467340ae9ac4f57d22c68bf6 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.057147 20746 tablet_replica.cc:333] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.058072 20746 raft_consensus.cc:2243] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.058706 20746 raft_consensus.cc:2272] T 6ac46eb166db448cb0c6f504819da525 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.062549 20746 tablet_replica.cc:333] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.063453 20746 raft_consensus.cc:2243] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.064786 20746 raft_consensus.cc:2272] T dc9c275f1b334c7db06956d0941b04ed P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.067783 20746 tablet_replica.cc:333] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.068697 20746 raft_consensus.cc:2243] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.069351 20746 raft_consensus.cc:2272] T 92c56097e8594d298daed39079975dd1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.070502 30611 consensus_peers.cc:597] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.073884 20746 tablet_replica.cc:333] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.075435 20746 raft_consensus.cc:2243] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.076450 20746 raft_consensus.cc:2272] T b59eef71464c40618e5e26de6a40aa90 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.079737 20746 tablet_replica.cc:333] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.081022 20746 raft_consensus.cc:2243] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.081805 20746 raft_consensus.cc:2272] T 107dd7c115d0402a9fd4e3312f621930 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.083328 30537 consensus_peers.cc:597] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.085453 20746 tablet_replica.cc:333] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.086522 20746 raft_consensus.cc:2243] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.087817 20746 raft_consensus.cc:2272] T a5ed6aaa36704114809147b7efabcaee P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.094200 20746 tablet_replica.cc:333] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.095288 20746 raft_consensus.cc:2243] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.096665 20746 raft_consensus.cc:2272] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.100066 20746 tablet_replica.cc:333] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.100945 20746 raft_consensus.cc:2243] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:14.095376 30611 consensus_peers.cc:597] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.101567 20746 raft_consensus.cc:2272] T d5b071d1d18845b8828943fac633def7 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.105074 20746 tablet_replica.cc:333] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.105988 20746 raft_consensus.cc:2243] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.106590 20746 raft_consensus.cc:2272] T 9035baa441924d8b9218b26855598f19 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.107154 30537 consensus_peers.cc:597] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.107879 30537 consensus_peers.cc:597] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.110015 20746 tablet_replica.cc:333] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.111045 20746 raft_consensus.cc:2243] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.114248 20746 raft_consensus.cc:2272] T 9b1c1fe97f07496c92182b3065536c15 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.117259 20746 tablet_replica.cc:333] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.119328 20746 raft_consensus.cc:2243] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.124013 20746 raft_consensus.cc:2272] T 95d00bccf5ce41f4b601bba7d235ffbb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.127185 20746 tablet_replica.cc:333] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.128072 20746 raft_consensus.cc:2243] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.129257 20746 raft_consensus.cc:2272] T 845d59ad223a4f6c9c388017a170527b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.132119 20746 tablet_replica.cc:333] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.132915 20746 raft_consensus.cc:2243] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.133502 20746 raft_consensus.cc:2272] T 3ad2905df5e84f04bcc21f6c9713c744 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.136798 20746 tablet_replica.cc:333] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.137625 20746 raft_consensus.cc:2243] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.138156 20746 raft_consensus.cc:2272] T 24a265170a904a89a2801145c1a61172 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.140946 20746 tablet_replica.cc:333] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.141949 20746 raft_consensus.cc:2243] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.142495 20746 raft_consensus.cc:2272] T d7ea982486944f2fbb7ce66208002c94 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.145406 20746 tablet_replica.cc:333] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.146173 20746 raft_consensus.cc:2243] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.147280 20746 raft_consensus.cc:2272] T 6664fcb5ba4f487b957d5cc98ec559de P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.150449 20746 tablet_replica.cc:333] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.151367 20746 raft_consensus.cc:2243] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.152603 20746 raft_consensus.cc:2272] T e003633f9bf24b09b3789ddc3ac48cc1 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.156024 20746 tablet_replica.cc:333] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.156836 20746 raft_consensus.cc:2243] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.157503 20746 raft_consensus.cc:2272] T 44a0e227c0ca4247b0ba8fbf2d005fec P 19b1445cece541d3b2abe177283fef44 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.160480 20746 tablet_replica.cc:333] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.161594 20746 raft_consensus.cc:2243] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.162220 20746 raft_consensus.cc:2272] T 6f17363c79974c6a863b51a45eef6e0e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.165335 20746 tablet_replica.cc:333] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.166139 20746 raft_consensus.cc:2243] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.166770 20746 raft_consensus.cc:2272] T c678eac1173e4cdaa75a8279bc764f29 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.169878 20746 tablet_replica.cc:333] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.170680 20746 raft_consensus.cc:2243] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.171864 20746 raft_consensus.cc:2272] T 06e3f31e48cd40e890350bcb72063bf7 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.175400 20746 tablet_replica.cc:333] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.176270 20746 raft_consensus.cc:2243] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.176813 20746 raft_consensus.cc:2272] T 8391e1197cab41039a7daf01b4cfb72b P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.179728 20746 tablet_replica.cc:333] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.180552 20746 raft_consensus.cc:2243] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.181787 20746 raft_consensus.cc:2272] T bd0df1d226d24b358d6d328eb07552ba P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.185083 20746 tablet_replica.cc:333] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.185923 20746 raft_consensus.cc:2243] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.186533 20746 raft_consensus.cc:2272] T ea53b9ee66e343b18d301fe650b12554 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.189564 20746 tablet_replica.cc:333] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.190340 20746 raft_consensus.cc:2243] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.191475 20746 raft_consensus.cc:2272] T 309754d072834ef381a0c8a6121b05cb P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.194926 20746 tablet_replica.cc:333] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.195803 20746 raft_consensus.cc:2243] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.196412 20746 raft_consensus.cc:2272] T 730e52dc15a842e8a606385755e5558e P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.199564 20746 tablet_replica.cc:333] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.200385 20746 raft_consensus.cc:2243] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.200992 20746 raft_consensus.cc:2272] T 7a582514dd6e459fa48675d41a9ff078 P 19b1445cece541d3b2abe177283fef44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.204077 20746 tablet_replica.cc:333] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.204802 20746 raft_consensus.cc:2243] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.205396 20746 raft_consensus.cc:2272] T 726e825b67364fccb3b417c19afe5e14 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.208904 20746 tablet_replica.cc:333] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.209738 20746 raft_consensus.cc:2243] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.210374 20746 raft_consensus.cc:2272] T fdb6f3b113b34e3c8bce234b33434313 P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.213405 20746 tablet_replica.cc:333] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44: stopping tablet replica
I20260709 13:10:14.214100 20746 raft_consensus.cc:2243] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.214684 20746 raft_consensus.cc:2272] T 289ca1e3324041acb6ac3410039210be P 19b1445cece541d3b2abe177283fef44 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.221678 30537 consensus_peers.cc:597] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.244433 30611 consensus_peers.cc:597] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.284175 30611 consensus_peers.cc:597] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.287729 30611 consensus_peers.cc:597] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.295279 30537 connection.cc:573] client connection to 127.20.66.130:35751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:10:14.295807 30537 consensus_peers.cc:597] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.322417 30611 consensus_peers.cc:597] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.357879 31647 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.358599 31647 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.361418 31647 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.362316 30565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "b897660686b3465390a5bad32efea13c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955d2e588381448981f85fadf225aa05" is_pre_election: true
W20260709 13:10:14.367043 30610 leader_election.cc:336] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.367787 30610 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:10:14.370182 31647 raft_consensus.cc:2749] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:14.374747 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
W20260709 13:10:14.408368 30537 consensus_peers.cc:597] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.431747 30537 consensus_peers.cc:597] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 -> Peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Couldn't send request to peer 19b1445cece541d3b2abe177283fef44. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.495941 31605 raft_consensus.cc:493] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.496537 31605 raft_consensus.cc:515] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } }
I20260709 13:10:14.500164 31605 leader_election.cc:290] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:14.506489 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:10:14.506490 30537 leader_election.cc:336] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
W20260709 13:10:14.512490 30536 leader_election.cc:336] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.512835 30536 leader_election.cc:304] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955d2e588381448981f85fadf225aa05; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.513451 31605 raft_consensus.cc:2749] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.559299 31647 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.560861 31647 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.565482 31647 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:10:14.579984 30611 leader_election.cc:336] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:14.580781 30611 consensus_peers.cc:597] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.591324 30610 leader_election.cc:336] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.591915 30610 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:10:14.594336 31605 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.594980 31605 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.600677 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:10:14.601421 30641 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:10:14.602993 31647 raft_consensus.cc:2749] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.603116 30535 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:14.605223 31641 raft_consensus.cc:2804] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:14.605350 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:14.605676 31641 raft_consensus.cc:493] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.606154 31641 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.608014 31647 raft_consensus.cc:493] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.608588 31647 raft_consensus.cc:515] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:14.614637 20746 tablet_replica.cc:333] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.615700 20746 raft_consensus.cc:2243] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.619094 31641 raft_consensus.cc:515] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:10:14.621994 30610 leader_election.cc:336] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.616832 20746 raft_consensus.cc:2272] T 0f3b4d4c8fc44213807c39418459447d P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.629016 30536 leader_election.cc:336] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.630137 31605 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.611059 31647 leader_election.cc:290] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:10:14.633983 30611 consensus_peers.cc:597] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.649380 30611 leader_election.cc:336] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.650175 30611 leader_election.cc:304] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.654382 31647 raft_consensus.cc:2749] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.654608 31650 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.651485 20746 tablet_replica.cc:333] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.655234 31650 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.657316 20746 raft_consensus.cc:2243] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.658214 31597 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.667917 31597 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.670495 31597 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.671883 20746 raft_consensus.cc:2272] T eba54fb250484a4e8102345d2a02e5cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.672655 31641 leader_election.cc:290] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.675114 20746 tablet_replica.cc:333] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
W20260709 13:10:14.675005 30610 leader_election.cc:336] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.676172 20746 raft_consensus.cc:2243] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.676785 20746 raft_consensus.cc:2272] T d204ad91a1c74eb9b3335582ee6717c7 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.678567 31597 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.678607 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efba0c8d26eb438ea2ff27b4bb66b20a" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:14.688212 30641 raft_consensus.cc:3060] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.687879 31597 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.690541 20746 tablet_replica.cc:333] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.691615 20746 raft_consensus.cc:2243] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.691795 31647 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.692668 20746 raft_consensus.cc:2272] T 040a6253e94b4c19952ed01cb9d8c6a5 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.692384 31647 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.694868 31647 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.690604 31597 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:10:14.687799 30536 leader_election.cc:336] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:10:14.700949 30610 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:10:14.701606 30610 leader_election.cc:336] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.702208 31650 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.697513 30641 raft_consensus.cc:2468] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:10:14.702994 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:10:14.703760 30637 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 4.
I20260709 13:10:14.704272 30535 leader_election.cc:304] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.704517 31581 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.705199 31581 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.705838 30535 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:14.707046 20746 tablet_replica.cc:333] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.707602 31581 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.708053 20746 raft_consensus.cc:2243] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.708187 31581 raft_consensus.cc:2804] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:14.708592 20746 raft_consensus.cc:2272] T 3c4b70f2baf64ae8a943a8bb8aa24385 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.709192 31605 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:14.709550 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:10:14.709632 31605 raft_consensus.cc:493] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.710338 30637 raft_consensus.cc:2468] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:10:14.710479 31605 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:14.711491 20746 tablet_replica.cc:333] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.711423 30535 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:14.712260 31641 raft_consensus.cc:2804] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:14.712468 20746 raft_consensus.cc:2243] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.712667 31641 raft_consensus.cc:493] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.713136 20746 raft_consensus.cc:2272] T 679bf6d4a8654a72b47b318645ab6fc0 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.715574 20746 tablet_replica.cc:333] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05: stopping tablet replica
W20260709 13:10:14.715986 30611 consensus_peers.cc:597] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.716408 20746 raft_consensus.cc:2243] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.716795 31641 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.717140 20746 raft_consensus.cc:2272] T 37455107e57b47c8978a434d17f3c76c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.717404 31650 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.717958 31650 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.720095 20746 tablet_replica.cc:333] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.720193 31650 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.721130 20746 raft_consensus.cc:2243] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.722198 20746 raft_consensus.cc:2272] T 93a6f0d4b4c848deb76000430c8bb0a5 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.725325 30611 leader_election.cc:336] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:14.725402 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:10:14.725960 30611 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:10:14.726125 30637 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:10:14.726670 31597 raft_consensus.cc:2749] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.732128 31597 raft_consensus.cc:493] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.732903 31597 raft_consensus.cc:515] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:14.734941 31647 raft_consensus.cc:493] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.735745 31597 leader_election.cc:290] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:14.738070 20746 tablet_replica.cc:333] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.735550 31647 raft_consensus.cc:515] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:10:14.740267 30611 consensus_peers.cc:597] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.741067 30611 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:14.741323 31647 leader_election.cc:290] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:14.741585 30611 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:10:14.742559 31647 raft_consensus.cc:2749] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:14.743945 30611 leader_election.cc:336] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:14.744449 30611 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:10:14.739425 20746 raft_consensus.cc:2243] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:14.744463 30610 leader_election.cc:336] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.745805 20746 raft_consensus.cc:2272] T 6ca8dcce43e444f8ada68dae5eecc1f6 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.748121 30610 leader_election.cc:336] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.748399 31647 raft_consensus.cc:2749] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.708765 31581 raft_consensus.cc:697] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:14.750941 30535 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:14.751055 31581 consensus_queue.cc:237] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.753211 31652 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:14.753607 31652 raft_consensus.cc:493] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.754000 31652 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:14.755743 20746 tablet_replica.cc:333] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.758379 20746 raft_consensus.cc:2243] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.762468 31641 raft_consensus.cc:515] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:10:14.767452 30611 leader_election.cc:336] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.783165 30611 leader_election.cc:304] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
W20260709 13:10:14.784241 30611 leader_election.cc:336] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.784494 31641 leader_election.cc:290] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.784683 30611 leader_election.cc:304] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.786234 31643 raft_consensus.cc:2749] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.767995 20746 raft_consensus.cc:2272] T 7a582514dd6e459fa48675d41a9ff078 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.786939 31643 raft_consensus.cc:2749] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.786875 31652 raft_consensus.cc:515] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.789223 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c09468889ceb4dd490e2cbd2c245b3d9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:14.789779 20746 tablet_replica.cc:333] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.789928 30637 raft_consensus.cc:3060] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:14.789181 30611 consensus_peers.cc:597] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.790712 20746 raft_consensus.cc:2243] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.791282 20746 raft_consensus.cc:2272] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.770555 31605 raft_consensus.cc:515] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.794723 31605 leader_election.cc:290] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 5 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.796403 20746 tablet_replica.cc:333] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.797353 20746 raft_consensus.cc:2243] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.797942 20746 raft_consensus.cc:2272] T bccb92fd917f4a97af88be6e7e8fed1f P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.798326 30637 raft_consensus.cc:2468] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
I20260709 13:10:14.799566 31643 raft_consensus.cc:493] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.800305 31643 raft_consensus.cc:515] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
W20260709 13:10:14.764928 30536 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.800922 31652 leader_election.cc:290] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 4 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.800971 20746 tablet_replica.cc:333] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.805416 20746 raft_consensus.cc:2243] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.806005 20746 raft_consensus.cc:2272] T 6e52d69d8f9044099d83595faf8a7c88 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.808696 30610 leader_election.cc:336] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:10:14.802294 30611 consensus_peers.cc:597] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.810250 31643 leader_election.cc:290] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:10:14.802883 30536 leader_election.cc:336] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:10:14.811858 30536 leader_election.cc:336] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.803074 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2908e68999c34a76b7343444d882b4e9" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:14.815953 30641 raft_consensus.cc:3060] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:14.803256 30535 leader_election.cc:304] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:14.803575 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4871109603643fe996c7bfd323672c7" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
W20260709 13:10:14.819204 30536 leader_election.cc:336] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.819766 30637 raft_consensus.cc:3060] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:14.820077 30536 leader_election.cc:336] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.808923 20746 tablet_replica.cc:333] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.823194 31652 raft_consensus.cc:2804] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:14.823585 30641 raft_consensus.cc:2468] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 4.
I20260709 13:10:14.824086 20746 raft_consensus.cc:2243] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.824107 31652 raft_consensus.cc:697] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:14.825320 20746 raft_consensus.cc:2272] T a15b8193c82e4caeb06f0da37159665c P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.827564 30536 leader_election.cc:336] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.828976 20746 tablet_replica.cc:333] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.828462 31652 consensus_queue.cc:237] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:10:14.829787 30611 leader_election.cc:336] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.829715 30535 leader_election.cc:304] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.829969 20746 raft_consensus.cc:2243] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.830320 30611 leader_election.cc:304] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.831094 31605 raft_consensus.cc:2804] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:14.831126 20746 raft_consensus.cc:2272] T 6ac46eb166db448cb0c6f504819da525 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.831723 31605 raft_consensus.cc:697] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 4 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:14.832686 31605 consensus_queue.cc:237] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.833174 31643 raft_consensus.cc:2749] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:14.834632 30611 consensus_peers.cc:597] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:14.837090 30611 consensus_peers.cc:597] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.839092 20746 tablet_replica.cc:333] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.840160 20746 raft_consensus.cc:2243] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.842187 31641 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.842731 31641 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.843976 20746 raft_consensus.cc:2272] T e78ade21dd6d46a3a18152943df69389 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.845160 31641 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.846266 30637 raft_consensus.cc:2468] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 5.
I20260709 13:10:14.849669 30641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c" is_pre_election: true
I20260709 13:10:14.849908 30535 leader_election.cc:304] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.850414 30641 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 2.
I20260709 13:10:14.851032 31641 raft_consensus.cc:2804] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:14.852200 30535 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 
I20260709 13:10:14.847251 20746 tablet_replica.cc:333] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.853685 31581 raft_consensus.cc:2804] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:14.854197 31581 raft_consensus.cc:493] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.854650 31581 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.857511 20746 raft_consensus.cc:2243] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.858076 31641 raft_consensus.cc:697] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 5 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:14.859661 31641 consensus_queue.cc:237] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
W20260709 13:10:14.865820 30536 leader_election.cc:336] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.863443 31581 raft_consensus.cc:515] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.861267 20746 pending_rounds.cc:70] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: Trying to abort 1 pending ops.
I20260709 13:10:14.867578 20746 pending_rounds.cc:77] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305636310373101568 op_type: NO_OP noop_request { }
I20260709 13:10:14.867995 20746 raft_consensus.cc:2889] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:14.868351 20746 raft_consensus.cc:2272] T 2908e68999c34a76b7343444d882b4e9 P 955d2e588381448981f85fadf225aa05 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.869351 30637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2c6fa20b2b4325804ef77b1d2172bf" candidate_uuid: "955d2e588381448981f85fadf225aa05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b897660686b3465390a5bad32efea13c"
I20260709 13:10:14.870054 30637 raft_consensus.cc:3060] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.881209 20746 tablet_replica.cc:333] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.882282 20746 raft_consensus.cc:2243] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.882937 20746 raft_consensus.cc:2272] T bbc578fa661c4935aae4aece586f4271 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.885923 20746 tablet_replica.cc:333] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
W20260709 13:10:14.886127 30611 consensus_peers.cc:597] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.886788 20746 raft_consensus.cc:2243] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.887446 20746 raft_consensus.cc:2272] T 3ad2905df5e84f04bcc21f6c9713c744 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.888355 31616 raft_consensus.cc:493] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.888928 31616 raft_consensus.cc:515] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:14.889823 31581 leader_election.cc:290] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Requested vote from peers b897660686b3465390a5bad32efea13c (127.20.66.132:45099), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
I20260709 13:10:14.891251 31616 leader_election.cc:290] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:14.881209 30637 raft_consensus.cc:2468] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955d2e588381448981f85fadf225aa05 in term 3.
W20260709 13:10:14.891850 30536 leader_election.cc:336] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.893560 20746 tablet_replica.cc:333] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.894486 20746 raft_consensus.cc:2243] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.895067 30535 leader_election.cc:304] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955d2e588381448981f85fadf225aa05, b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.896032 31581 raft_consensus.cc:2804] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:14.896739 31581 raft_consensus.cc:697] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Becoming Leader. State: Replica: 955d2e588381448981f85fadf225aa05, State: Running, Role: LEADER
I20260709 13:10:14.895697 20746 pending_rounds.cc:70] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: Trying to abort 1 pending ops.
W20260709 13:10:14.897497 30610 leader_election.cc:336] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.897624 20746 pending_rounds.cc:77] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636310082228224 op_type: NO_OP noop_request { }
I20260709 13:10:14.897644 31581 consensus_queue.cc:237] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:14.898394 20746 raft_consensus.cc:2889] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:14.898872 20746 raft_consensus.cc:2272] T efba0c8d26eb438ea2ff27b4bb66b20a P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.900202 31616 raft_consensus.cc:493] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
W20260709 13:10:14.900951 30611 leader_election.cc:336] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.900791 31616 raft_consensus.cc:515] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:14.901508 30611 leader_election.cc:304] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.902235 31647 raft_consensus.cc:2749] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:14.903116 31616 leader_election.cc:290] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:14.905937 20746 tablet_replica.cc:333] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.907058 20746 raft_consensus.cc:2243] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.907660 20746 raft_consensus.cc:2272] T ba4cf9aa835b48038aa80c194df56bd6 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.909646 30610 leader_election.cc:336] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.910907 20746 tablet_replica.cc:333] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.912024 20746 raft_consensus.cc:2243] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.912847 20746 raft_consensus.cc:2272] T 730e52dc15a842e8a606385755e5558e P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.914832 30611 leader_election.cc:336] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.915238 30611 leader_election.cc:304] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.916565 31616 raft_consensus.cc:2749] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.917850 20746 tablet_replica.cc:333] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.918720 20746 raft_consensus.cc:2243] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.919245 20746 raft_consensus.cc:2272] T e003633f9bf24b09b3789ddc3ac48cc1 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.922132 20746 tablet_replica.cc:333] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.922932 20746 raft_consensus.cc:2243] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.923593 20746 pending_rounds.cc:70] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: Trying to abort 1 pending ops.
I20260709 13:10:14.923805 20746 pending_rounds.cc:77] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05: Aborting op as it isn't in flight: id { term: 5 index: 4 } timestamp: 7305636310490058752 op_type: NO_OP noop_request { }
I20260709 13:10:14.924103 20746 raft_consensus.cc:2889] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 5 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:14.924467 20746 raft_consensus.cc:2272] T d4871109603643fe996c7bfd323672c7 P 955d2e588381448981f85fadf225aa05 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.926767 20746 tablet_replica.cc:333] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.927350 20746 raft_consensus.cc:2243] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.927789 20746 raft_consensus.cc:2272] T bd0df1d226d24b358d6d328eb07552ba P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.929932 20746 tablet_replica.cc:333] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.930451 20746 raft_consensus.cc:2243] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.930864 20746 raft_consensus.cc:2272] T 845d59ad223a4f6c9c388017a170527b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.932834 20746 tablet_replica.cc:333] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.933382 20746 raft_consensus.cc:2243] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.933804 20746 raft_consensus.cc:2272] T 78e63f8484d24f8f9bc5c94fef442265 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.936426 20746 tablet_replica.cc:333] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.937188 20746 raft_consensus.cc:2243] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.938217 20746 raft_consensus.cc:2272] T 47885caab3e14ff2ae88840a9dfc0944 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.941149 20746 tablet_replica.cc:333] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.941946 20746 raft_consensus.cc:2243] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.942929 20746 raft_consensus.cc:2272] T 1213ccbc173c43f683d18d933293e4de P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.945927 20746 tablet_replica.cc:333] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.946689 20746 raft_consensus.cc:2243] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.946916 31647 raft_consensus.cc:493] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:14.947717 20746 raft_consensus.cc:2272] T 8391e1197cab41039a7daf01b4cfb72b P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.947444 31647 raft_consensus.cc:515] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:14.949772 31647 leader_election.cc:290] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:14.950730 20746 tablet_replica.cc:333] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.951565 20746 raft_consensus.cc:2243] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.952582 20746 raft_consensus.cc:2272] T c678eac1173e4cdaa75a8279bc764f29 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:14.954631 30611 leader_election.cc:336] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:14.955638 20746 tablet_replica.cc:333] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
W20260709 13:10:14.955812 30610 leader_election.cc:336] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:14.956265 30610 leader_election.cc:304] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:14.956475 20746 raft_consensus.cc:2243] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.957055 20746 raft_consensus.cc:2272] T 06e3f31e48cd40e890350bcb72063bf7 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.957020 31647 raft_consensus.cc:2749] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.959739 20746 tablet_replica.cc:333] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.960354 20746 raft_consensus.cc:2243] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.960969 20746 pending_rounds.cc:70] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: Trying to abort 1 pending ops.
I20260709 13:10:14.961251 20746 pending_rounds.cc:77] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636310641537024 op_type: NO_OP noop_request { }
I20260709 13:10:14.961524 20746 raft_consensus.cc:2889] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:14.961787 20746 raft_consensus.cc:2272] T 8b2c6fa20b2b4325804ef77b1d2172bf P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.964023 20746 tablet_replica.cc:333] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.964586 20746 raft_consensus.cc:2243] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.965456 20746 raft_consensus.cc:2272] T 95d00bccf5ce41f4b601bba7d235ffbb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.967482 20746 tablet_replica.cc:333] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.968014 20746 raft_consensus.cc:2243] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.968628 20746 pending_rounds.cc:70] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: Trying to abort 1 pending ops.
I20260709 13:10:14.968845 20746 pending_rounds.cc:77] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636310396628992 op_type: NO_OP noop_request { }
I20260709 13:10:14.969170 20746 raft_consensus.cc:2889] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:14.969442 20746 raft_consensus.cc:2272] T c09468889ceb4dd490e2cbd2c245b3d9 P 955d2e588381448981f85fadf225aa05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.971742 20746 tablet_replica.cc:333] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.972364 20746 raft_consensus.cc:2243] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.972811 20746 raft_consensus.cc:2272] T dc9c275f1b334c7db06956d0941b04ed P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.974973 20746 tablet_replica.cc:333] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.975539 20746 raft_consensus.cc:2243] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.976366 20746 raft_consensus.cc:2272] T 9035baa441924d8b9218b26855598f19 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.978524 20746 tablet_replica.cc:333] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.979048 20746 raft_consensus.cc:2243] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.979507 20746 raft_consensus.cc:2272] T 9b1c1fe97f07496c92182b3065536c15 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.981572 20746 tablet_replica.cc:333] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.982076 20746 raft_consensus.cc:2243] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.982513 20746 raft_consensus.cc:2272] T 92c56097e8594d298daed39079975dd1 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.984472 20746 tablet_replica.cc:333] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.984990 20746 raft_consensus.cc:2243] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.985868 20746 raft_consensus.cc:2272] T 107dd7c115d0402a9fd4e3312f621930 P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.987833 20746 tablet_replica.cc:333] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.988366 20746 raft_consensus.cc:2243] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.988830 20746 raft_consensus.cc:2272] T a5ed6aaa36704114809147b7efabcaee P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.991626 20746 tablet_replica.cc:333] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.992352 20746 raft_consensus.cc:2243] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.992945 20746 raft_consensus.cc:2272] T 7a4e2ca1d8a742e1a0beb1710464b3bf P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.995494 20746 tablet_replica.cc:333] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.995993 20746 raft_consensus.cc:2243] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:14.996807 20746 raft_consensus.cc:2272] T ea53b9ee66e343b18d301fe650b12554 P 955d2e588381448981f85fadf225aa05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:14.998939 20746 tablet_replica.cc:333] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05: stopping tablet replica
I20260709 13:10:14.999455 20746 raft_consensus.cc:2243] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:14.999887 20746 raft_consensus.cc:2272] T 309754d072834ef381a0c8a6121b05cb P 955d2e588381448981f85fadf225aa05 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:15.005935 30611 consensus_peers.cc:597] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c -> Peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Couldn't send request to peer 955d2e588381448981f85fadf225aa05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:15.050074 31647 raft_consensus.cc:493] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5825cf9d14b4469c9960cca89b0010f1)
I20260709 13:10:15.050644 31647 raft_consensus.cc:515] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } }
I20260709 13:10:15.053061 31647 leader_election.cc:290] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261), 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915)
W20260709 13:10:15.058892 30611 leader_election.cc:336] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:15.060926 30610 leader_election.cc:336] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:15.063290 30610 leader_election.cc:304] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 5825cf9d14b4469c9960cca89b0010f1, 955d2e588381448981f85fadf225aa05
I20260709 13:10:15.064126 31647 raft_consensus.cc:2749] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:15.091490 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:10:15.178598 31647 raft_consensus.cc:493] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19b1445cece541d3b2abe177283fef44)
I20260709 13:10:15.179415 31647 raft_consensus.cc:515] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:15.182155 31647 leader_election.cc:290] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:10:15.189810 30611 leader_election.cc:336] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
W20260709 13:10:15.191401 30611 leader_election.cc:336] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36261: connect: Connection refused (error 111)
I20260709 13:10:15.191908 30611 leader_election.cc:304] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:10:15.192746 31647 raft_consensus.cc:2749] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:15.199523 20746 tablet_server.cc:179] TabletServer@127.20.66.132:0 shutting down...
I20260709 13:10:15.200829 31647 raft_consensus.cc:493] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19b1445cece541d3b2abe177283fef44)
I20260709 13:10:15.202742 31647 raft_consensus.cc:515] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:15.205238 31647 leader_election.cc:290] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
W20260709 13:10:15.213627 30611 leader_election.cc:336] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
W20260709 13:10:15.214524 30611 leader_election.cc:336] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36261: connect: Connection refused (error 111)
I20260709 13:10:15.215054 30611 leader_election.cc:304] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:10:15.215924 31647 raft_consensus.cc:2749] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:15.255550 31647 raft_consensus.cc:493] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19b1445cece541d3b2abe177283fef44)
I20260709 13:10:15.256136 31647 raft_consensus.cc:515] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:15.258191 31647 leader_election.cc:290] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
W20260709 13:10:15.262951 30610 leader_election.cc:336] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
W20260709 13:10:15.263796 30611 leader_election.cc:336] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:15.264252 30611 leader_election.cc:304] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:15.265079 31647 raft_consensus.cc:2749] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:15.293064 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:15.294195 20746 tablet_replica.cc:333] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.294883 20746 raft_consensus.cc:2243] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.295408 20746 raft_consensus.cc:2272] T eba54fb250484a4e8102345d2a02e5cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.297832 20746 tablet_replica.cc:333] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.298448 20746 raft_consensus.cc:2243] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.298878 20746 raft_consensus.cc:2272] T b4a97b4e180f4b618f578bfcc4d841a1 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.302670 20746 tablet_replica.cc:333] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.303512 20746 raft_consensus.cc:2243] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.304163 20746 raft_consensus.cc:2272] T a5ed6aaa36704114809147b7efabcaee P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.307200 20746 tablet_replica.cc:333] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.307925 20746 raft_consensus.cc:2243] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.308911 20746 raft_consensus.cc:2272] T 55b4252704f04cae82d10d8b1bb4c88d P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.311611 20746 tablet_replica.cc:333] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.312436 20746 raft_consensus.cc:2243] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.313445 20746 raft_consensus.cc:2272] T 366b9fff4017464a9b99ab4878dbb119 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.315658 20746 tablet_replica.cc:333] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.316272 20746 raft_consensus.cc:2243] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.316728 20746 raft_consensus.cc:2272] T 3c4b70f2baf64ae8a943a8bb8aa24385 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.318908 20746 tablet_replica.cc:333] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.319741 20746 raft_consensus.cc:2243] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.320699 20746 raft_consensus.cc:2272] T 679bf6d4a8654a72b47b318645ab6fc0 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.323053 20746 tablet_replica.cc:333] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.323628 20746 raft_consensus.cc:2243] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.324064 20746 raft_consensus.cc:2272] T 37455107e57b47c8978a434d17f3c76c P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.326087 20746 tablet_replica.cc:333] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.326619 20746 raft_consensus.cc:2243] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.327032 20746 raft_consensus.cc:2272] T efba0c8d26eb438ea2ff27b4bb66b20a P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.329133 20746 tablet_replica.cc:333] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.329686 20746 raft_consensus.cc:2243] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.330461 20746 raft_consensus.cc:2272] T ba4cf9aa835b48038aa80c194df56bd6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.333098 20746 tablet_replica.cc:333] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.333819 20746 raft_consensus.cc:2243] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.334774 20746 raft_consensus.cc:2272] T e78ade21dd6d46a3a18152943df69389 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.337690 20746 tablet_replica.cc:333] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.338265 20746 raft_consensus.cc:2243] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.338696 20746 raft_consensus.cc:2272] T 7a582514dd6e459fa48675d41a9ff078 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.341187 20746 tablet_replica.cc:333] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.341770 20746 raft_consensus.cc:2243] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.342201 20746 raft_consensus.cc:2272] T 726e825b67364fccb3b417c19afe5e14 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.344204 20746 tablet_replica.cc:333] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.344729 20746 raft_consensus.cc:2243] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.345216 20746 raft_consensus.cc:2272] T fdb6f3b113b34e3c8bce234b33434313 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.347594 20746 tablet_replica.cc:333] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.348379 20746 raft_consensus.cc:2243] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.349262 20746 raft_consensus.cc:2272] T ec6ae4a76cb84d41a9dcc0c1cce9cba2 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.351398 20746 tablet_replica.cc:333] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.351980 20746 raft_consensus.cc:2243] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.352780 20746 raft_consensus.cc:2272] T bccb92fd917f4a97af88be6e7e8fed1f P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.354947 20746 tablet_replica.cc:333] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.355525 20746 raft_consensus.cc:2243] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.356031 20746 raft_consensus.cc:2272] T c69a30e3a4a8473a9f1f8652ea9463d8 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.358260 20746 tablet_replica.cc:333] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.358814 20746 raft_consensus.cc:2243] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.359578 20746 raft_consensus.cc:2272] T 6e52d69d8f9044099d83595faf8a7c88 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.361665 20746 tablet_replica.cc:333] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.362212 20746 raft_consensus.cc:2243] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.362663 20746 raft_consensus.cc:2272] T 86637a5e467340ae9ac4f57d22c68bf6 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.364583 20746 tablet_replica.cc:333] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.365207 20746 raft_consensus.cc:2243] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.365618 20746 raft_consensus.cc:2272] T dc9c275f1b334c7db06956d0941b04ed P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.367695 20746 tablet_replica.cc:333] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.368245 20746 raft_consensus.cc:2243] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.369169 20746 raft_consensus.cc:2272] T 92c56097e8594d298daed39079975dd1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.371382 20746 tablet_replica.cc:333] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.371987 20746 raft_consensus.cc:2243] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.372479 20746 raft_consensus.cc:2272] T b59eef71464c40618e5e26de6a40aa90 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.372892 31597 raft_consensus.cc:493] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19b1445cece541d3b2abe177283fef44)
I20260709 13:10:15.373533 31597 raft_consensus.cc:515] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } } peers { permanent_uuid: "5825cf9d14b4469c9960cca89b0010f1" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 34915 } } peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } }
I20260709 13:10:15.374969 20746 tablet_replica.cc:333] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.375679 20746 raft_consensus.cc:2243] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.376080 31597 leader_election.cc:290] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915), 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751)
I20260709 13:10:15.376231 20746 raft_consensus.cc:2272] T 107dd7c115d0402a9fd4e3312f621930 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.379156 20746 tablet_replica.cc:333] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.380039 20746 raft_consensus.cc:2243] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.381172 20746 raft_consensus.cc:2272] T 289ca1e3324041acb6ac3410039210be P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:15.381418 30611 leader_election.cc:336] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:15.384318 20746 tablet_replica.cc:333] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c: stopping tablet replica
W20260709 13:10:15.384419 30610 leader_election.cc:336] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5825cf9d14b4469c9960cca89b0010f1 (127.20.66.129:34915): Network error: Client connection negotiation failed: client connection to 127.20.66.129:34915: connect: Connection refused (error 111)
I20260709 13:10:15.385234 30610 leader_election.cc:304] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 5825cf9d14b4469c9960cca89b0010f1
I20260709 13:10:15.385640 20746 raft_consensus.cc:2243] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.385993 31597 raft_consensus.cc:2749] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:15.386605 20746 raft_consensus.cc:2272] T d4871109603643fe996c7bfd323672c7 P b897660686b3465390a5bad32efea13c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.390107 20746 tablet_replica.cc:333] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.390642 31597 raft_consensus.cc:493] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19b1445cece541d3b2abe177283fef44)
I20260709 13:10:15.390957 20746 raft_consensus.cc:2243] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.391661 20746 raft_consensus.cc:2272] T 2908e68999c34a76b7343444d882b4e9 P b897660686b3465390a5bad32efea13c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.391233 31597 raft_consensus.cc:515] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b1445cece541d3b2abe177283fef44" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 35751 } } peers { permanent_uuid: "955d2e588381448981f85fadf225aa05" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 36261 } } peers { permanent_uuid: "b897660686b3465390a5bad32efea13c" member_type: VOTER last_known_addr { host: "127.20.66.132" port: 45099 } }
I20260709 13:10:15.393770 31597 leader_election.cc:290] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751), 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261)
I20260709 13:10:15.394392 20746 tablet_replica.cc:333] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.395270 20746 raft_consensus.cc:2243] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.396270 20746 raft_consensus.cc:2272] T bbc578fa661c4935aae4aece586f4271 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.399240 20746 tablet_replica.cc:333] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.399828 20746 raft_consensus.cc:2243] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:10:15.400336 30611 leader_election.cc:336] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 19b1445cece541d3b2abe177283fef44 (127.20.66.130:35751): Network error: Client connection negotiation failed: client connection to 127.20.66.130:35751: connect: Connection refused (error 111)
I20260709 13:10:15.400642 20746 raft_consensus.cc:2272] T 3ad2905df5e84f04bcc21f6c9713c744 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:15.401093 30611 leader_election.cc:336] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955d2e588381448981f85fadf225aa05 (127.20.66.131:36261): Network error: Client connection negotiation failed: client connection to 127.20.66.131:36261: connect: Connection refused (error 111)
I20260709 13:10:15.401543 30611 leader_election.cc:304] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b897660686b3465390a5bad32efea13c; no voters: 19b1445cece541d3b2abe177283fef44, 955d2e588381448981f85fadf225aa05
I20260709 13:10:15.402303 31597 raft_consensus.cc:2749] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:15.403466 20746 tablet_replica.cc:333] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.404114 20746 raft_consensus.cc:2243] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.405174 20746 raft_consensus.cc:2272] T 24a265170a904a89a2801145c1a61172 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.407816 20746 tablet_replica.cc:333] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.408538 20746 raft_consensus.cc:2243] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.409184 20746 raft_consensus.cc:2272] T d7ea982486944f2fbb7ce66208002c94 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.411178 20746 tablet_replica.cc:333] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.411692 20746 raft_consensus.cc:2243] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.412106 20746 raft_consensus.cc:2272] T 44a0e227c0ca4247b0ba8fbf2d005fec P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.414134 20746 tablet_replica.cc:333] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.414657 20746 raft_consensus.cc:2243] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.415045 20746 raft_consensus.cc:2272] T 6ca8dcce43e444f8ada68dae5eecc1f6 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.416963 20746 tablet_replica.cc:333] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.417536 20746 raft_consensus.cc:2243] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.418322 20746 raft_consensus.cc:2272] T 6f17363c79974c6a863b51a45eef6e0e P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.420442 20746 tablet_replica.cc:333] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.420976 20746 raft_consensus.cc:2243] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.421483 20746 raft_consensus.cc:2272] T 6664fcb5ba4f487b957d5cc98ec559de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.423664 20746 tablet_replica.cc:333] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.424181 20746 raft_consensus.cc:2243] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.424625 20746 raft_consensus.cc:2272] T e003633f9bf24b09b3789ddc3ac48cc1 P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.426759 20746 tablet_replica.cc:333] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.427274 20746 raft_consensus.cc:2243] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.427701 20746 raft_consensus.cc:2272] T 845d59ad223a4f6c9c388017a170527b P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.429761 20746 tablet_replica.cc:333] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.430264 20746 raft_consensus.cc:2243] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:15.431026 20746 raft_consensus.cc:2272] T 78e63f8484d24f8f9bc5c94fef442265 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.433332 20746 tablet_replica.cc:333] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.433912 20746 raft_consensus.cc:2243] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.434356 20746 raft_consensus.cc:2272] T bd0df1d226d24b358d6d328eb07552ba P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.436514 20746 tablet_replica.cc:333] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.437071 20746 raft_consensus.cc:2243] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.437664 20746 raft_consensus.cc:2272] T 47885caab3e14ff2ae88840a9dfc0944 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.439596 20746 tablet_replica.cc:333] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.440152 20746 raft_consensus.cc:2243] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.440634 20746 raft_consensus.cc:2272] T 1213ccbc173c43f683d18d933293e4de P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.442811 20746 tablet_replica.cc:333] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.443320 20746 raft_consensus.cc:2243] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.443832 20746 raft_consensus.cc:2272] T 309754d072834ef381a0c8a6121b05cb P b897660686b3465390a5bad32efea13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.445902 20746 tablet_replica.cc:333] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.446405 20746 raft_consensus.cc:2243] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.446846 20746 raft_consensus.cc:2272] T d5b071d1d18845b8828943fac633def7 P b897660686b3465390a5bad32efea13c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.448894 20746 tablet_replica.cc:333] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.449458 20746 raft_consensus.cc:2243] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.449889 20746 raft_consensus.cc:2272] T 8b2c6fa20b2b4325804ef77b1d2172bf P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:15.451787 20746 tablet_replica.cc:333] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c: stopping tablet replica
I20260709 13:10:15.452257 20746 raft_consensus.cc:2243] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:15.452692 20746 raft_consensus.cc:2272] T c09468889ceb4dd490e2cbd2c245b3d9 P b897660686b3465390a5bad32efea13c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.523231 20746 tablet_server.cc:196] TabletServer@127.20.66.132:0 shutdown complete.
I20260709 13:10:16.589182 20746 master.cc:562] Master@127.20.66.190:41433 shutting down...
I20260709 13:10:16.617085 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.617628 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.617910 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2230c368179f4041943f07933371a9ad: stopping tablet replica
I20260709 13:10:16.640483 20746 master.cc:584] Master@127.20.66.190:41433 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (37949 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:10:16.732107 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:41769
I20260709 13:10:16.733294 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:16.739506 31661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.739595 31660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.741734 31664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:16.741952 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:16.743419 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:16.743598 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:16.743779 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602616743764 us; error 0 us; skew 500 ppm
I20260709 13:10:16.744361 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:16.747093 20746 webserver.cc:533] Webserver started at http://127.20.66.190:44443/ using document root <none> and password file <none>
I20260709 13:10:16.747617 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:16.747802 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:16.748096 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:16.749315 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "b7256215a205464eab3332b0c8659cfc"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-rgrd"
I20260709 13:10:16.754031 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:10:16.757529 31668 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.758520 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:10:16.758978 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "b7256215a205464eab3332b0c8659cfc"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-rgrd"
I20260709 13:10:16.759336 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:16.778854 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:16.780241 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:16.822540 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:41769
I20260709 13:10:16.822655 31719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:41769 every 8 connection(s)
I20260709 13:10:16.826722 31720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:16.837625 31720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc: Bootstrap starting.
I20260709 13:10:16.843122 31720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:16.847517 31720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc: No bootstrap required, opened a new log
I20260709 13:10:16.849907 31720 raft_consensus.cc:359] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7256215a205464eab3332b0c8659cfc" member_type: VOTER }
I20260709 13:10:16.850306 31720 raft_consensus.cc:385] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:16.850580 31720 raft_consensus.cc:740] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7256215a205464eab3332b0c8659cfc, State: Initialized, Role: FOLLOWER
I20260709 13:10:16.851282 31720 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7256215a205464eab3332b0c8659cfc" member_type: VOTER }
I20260709 13:10:16.851759 31720 raft_consensus.cc:399] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:16.851987 31720 raft_consensus.cc:493] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:16.852254 31720 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:16.857055 31720 raft_consensus.cc:515] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7256215a205464eab3332b0c8659cfc" member_type: VOTER }
I20260709 13:10:16.857630 31720 leader_election.cc:304] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7256215a205464eab3332b0c8659cfc; no voters: 
I20260709 13:10:16.858817 31720 leader_election.cc:290] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:16.859328 31723 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:16.860894 31723 raft_consensus.cc:697] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 1 LEADER]: Becoming Leader. State: Replica: b7256215a205464eab3332b0c8659cfc, State: Running, Role: LEADER
I20260709 13:10:16.861752 31723 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7256215a205464eab3332b0c8659cfc" member_type: VOTER }
I20260709 13:10:16.862211 31720 sys_catalog.cc:565] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:16.867621 31725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7256215a205464eab3332b0c8659cfc. Latest consensus state: current_term: 1 leader_uuid: "b7256215a205464eab3332b0c8659cfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7256215a205464eab3332b0c8659cfc" member_type: VOTER } }
I20260709 13:10:16.867657 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7256215a205464eab3332b0c8659cfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7256215a205464eab3332b0c8659cfc" member_type: VOTER } }
I20260709 13:10:16.869163 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:16.868928 31725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:16.871397 31732 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:16.877405 31732 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:16.878597 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:16.887486 31732 catalog_manager.cc:1383] Generated new cluster ID: 32bc0815ef204979ab3cd586f59b1edc
I20260709 13:10:16.887789 31732 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:16.924021 31732 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:16.925426 31732 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:16.940773 31732 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc: Generated new TSK 0
I20260709 13:10:16.941471 31732 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:17.010098 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:17.016938 31741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:17.018939 31742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:17.020519 31744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:17.020853 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:17.021931 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:17.022127 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:17.022248 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602617022238 us; error 0 us; skew 500 ppm
I20260709 13:10:17.022722 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:17.025453 20746 webserver.cc:533] Webserver started at http://127.20.66.129:35263/ using document root <none> and password file <none>
I20260709 13:10:17.025913 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:17.026067 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:17.026286 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:17.027408 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:17.031912 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:10:17.035751 31749 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.036692 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:10:17.037065 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:17.037376 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:17.053221 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:17.054468 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:17.055986 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:17.058358 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:17.058583 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.058815 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:17.059000 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.101797 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:44397
I20260709 13:10:17.101904 31811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:44397 every 8 connection(s)
I20260709 13:10:17.107146 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:17.115882 31816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:17.119722 31817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:17.121428 31819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:17.122605 31812 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41769
I20260709 13:10:17.123045 31812 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:17.123770 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:17.123950 31812 heartbeater.cc:507] Master 127.20.66.190:41769 requested a full tablet report, sending...
I20260709 13:10:17.124890 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:17.125238 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:17.125459 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602617125441 us; error 0 us; skew 500 ppm
I20260709 13:10:17.126196 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:17.126353 31685 ts_manager.cc:194] Registered new tserver with Master: 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397)
I20260709 13:10:17.128383 31685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52202
I20260709 13:10:17.129190 20746 webserver.cc:533] Webserver started at http://127.20.66.130:45309/ using document root <none> and password file <none>
I20260709 13:10:17.129859 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:17.130137 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:17.130409 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:17.131503 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "142ed2f6af504e9583ab5c467be2b4d7"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:17.136183 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:17.140066 31824 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.140949 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:10:17.141347 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "142ed2f6af504e9583ab5c467be2b4d7"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:17.141654 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:17.166064 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:17.167208 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:17.168640 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:17.170965 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:17.171171 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.171428 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:17.171594 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.212087 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:33599
I20260709 13:10:17.212210 31886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:33599 every 8 connection(s)
I20260709 13:10:17.217052 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:17.223467 31890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:17.226099 31891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:17.227864 31893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:17.228044 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:17.229154 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:17.229446 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:17.229651 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602617229634 us; error 0 us; skew 500 ppm
I20260709 13:10:17.229658 31887 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41769
I20260709 13:10:17.230161 31887 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:17.230394 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:17.230968 31887 heartbeater.cc:507] Master 127.20.66.190:41769 requested a full tablet report, sending...
I20260709 13:10:17.233002 20746 webserver.cc:533] Webserver started at http://127.20.66.131:34867/ using document root <none> and password file <none>
I20260709 13:10:17.233440 31685 ts_manager.cc:194] Registered new tserver with Master: 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:17.233738 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:17.234033 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:17.234350 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:17.235321 31685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52214
I20260709 13:10:17.235884 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "c37cf16aa11340c79c452627e7d8c6f4"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:17.241250 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:10:17.244874 31898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.245791 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:17.246117 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "c37cf16aa11340c79c452627e7d8c6f4"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:17.246439 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:17.256610 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:17.258088 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:17.259481 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:17.262151 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:17.262408 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.262683 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:17.262890 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:17.307245 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:46117
I20260709 13:10:17.307380 31960 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:46117 every 8 connection(s)
I20260709 13:10:17.322019 31961 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41769
I20260709 13:10:17.322489 31961 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:17.323261 31961 heartbeater.cc:507] Master 127.20.66.190:41769 requested a full tablet report, sending...
I20260709 13:10:17.325519 31685 ts_manager.cc:194] Registered new tserver with Master: c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:17.325690 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015142482s
I20260709 13:10:17.327389 31685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52224
I20260709 13:10:17.642480 31685 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52226:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:17.645804 31685 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:17.722818 31777 tablet_service.cc:1511] Processing CreateTablet for tablet 832d464d4b3a43b0a1652b7c1e81e9aa (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:17.724280 31777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 832d464d4b3a43b0a1652b7c1e81e9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.726236 31851 tablet_service.cc:1511] Processing CreateTablet for tablet 62978121cded40708c2fe686051eafa4 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:17.726816 31776 tablet_service.cc:1511] Processing CreateTablet for tablet 62978121cded40708c2fe686051eafa4 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:17.727069 31852 tablet_service.cc:1511] Processing CreateTablet for tablet 832d464d4b3a43b0a1652b7c1e81e9aa (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:17.727788 31851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62978121cded40708c2fe686051eafa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.728490 31852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 832d464d4b3a43b0a1652b7c1e81e9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.730051 31849 tablet_service.cc:1511] Processing CreateTablet for tablet 5b654b3d4ccb4ab9b4a5f4c58a14964a (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:17.730219 31774 tablet_service.cc:1511] Processing CreateTablet for tablet 5b654b3d4ccb4ab9b4a5f4c58a14964a (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:17.730665 31773 tablet_service.cc:1511] Processing CreateTablet for tablet 706b58c86e5a48ddac9daafa32041514 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:17.731439 31849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b654b3d4ccb4ab9b4a5f4c58a14964a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.732470 31850 tablet_service.cc:1511] Processing CreateTablet for tablet 93ac731386824507bff9466dfed7f865 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:17.733742 31850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ac731386824507bff9466dfed7f865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.735917 31775 tablet_service.cc:1511] Processing CreateTablet for tablet 93ac731386824507bff9466dfed7f865 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:17.736640 31848 tablet_service.cc:1511] Processing CreateTablet for tablet 706b58c86e5a48ddac9daafa32041514 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:17.737457 31776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62978121cded40708c2fe686051eafa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.737888 31848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 706b58c86e5a48ddac9daafa32041514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.741408 31773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 706b58c86e5a48ddac9daafa32041514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.745365 31774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b654b3d4ccb4ab9b4a5f4c58a14964a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.749466 31775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ac731386824507bff9466dfed7f865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.771436 31926 tablet_service.cc:1511] Processing CreateTablet for tablet 832d464d4b3a43b0a1652b7c1e81e9aa (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:17.772521 31925 tablet_service.cc:1511] Processing CreateTablet for tablet 62978121cded40708c2fe686051eafa4 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:17.773226 31926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 832d464d4b3a43b0a1652b7c1e81e9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.774453 31925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62978121cded40708c2fe686051eafa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.780570 31924 tablet_service.cc:1511] Processing CreateTablet for tablet 93ac731386824507bff9466dfed7f865 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:17.781066 31849 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa2541d650649f6969c9ac41ac82c83 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:17.782449 31849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa2541d650649f6969c9ac41ac82c83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.782644 31924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ac731386824507bff9466dfed7f865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.787426 31923 tablet_service.cc:1511] Processing CreateTablet for tablet 5b654b3d4ccb4ab9b4a5f4c58a14964a (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:17.789532 31923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b654b3d4ccb4ab9b4a5f4c58a14964a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.795897 31922 tablet_service.cc:1511] Processing CreateTablet for tablet 706b58c86e5a48ddac9daafa32041514 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:17.798151 31922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 706b58c86e5a48ddac9daafa32041514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.798285 31848 tablet_service.cc:1511] Processing CreateTablet for tablet 872ddbd361a64e06a386a4f1b6d8da8d (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:17.799716 31848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872ddbd361a64e06a386a4f1b6d8da8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.812803 31847 tablet_service.cc:1511] Processing CreateTablet for tablet 51c0b90cccc2423f9160c57f32682bfc (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:17.814354 31847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c0b90cccc2423f9160c57f32682bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.819975 31772 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa2541d650649f6969c9ac41ac82c83 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:17.823616 31921 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa2541d650649f6969c9ac41ac82c83 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:17.825058 31921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa2541d650649f6969c9ac41ac82c83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.827224 31771 tablet_service.cc:1511] Processing CreateTablet for tablet 872ddbd361a64e06a386a4f1b6d8da8d (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:17.831211 31920 tablet_service.cc:1511] Processing CreateTablet for tablet 872ddbd361a64e06a386a4f1b6d8da8d (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:17.833446 31920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872ddbd361a64e06a386a4f1b6d8da8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.834303 31772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa2541d650649f6969c9ac41ac82c83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.837561 31771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872ddbd361a64e06a386a4f1b6d8da8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.849692 31846 tablet_service.cc:1511] Processing CreateTablet for tablet 48121309c18d4fad8d28f098002b9d02 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:17.851569 31846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48121309c18d4fad8d28f098002b9d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.861894 31851 tablet_service.cc:1511] Processing CreateTablet for tablet 4214d17086964585905299fae79f0bca (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:17.863834 31851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4214d17086964585905299fae79f0bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.867802 31919 tablet_service.cc:1511] Processing CreateTablet for tablet 51c0b90cccc2423f9160c57f32682bfc (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:17.870386 31919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c0b90cccc2423f9160c57f32682bfc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:17.873800 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:17.864056 31773 tablet_service.cc:1511] Processing CreateTablet for tablet 51c0b90cccc2423f9160c57f32682bfc (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:17.878115 31773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c0b90cccc2423f9160c57f32682bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.880766 31981 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:17.887782 31981 tablet_bootstrap.cc:654] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.889514 31980 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:17.901360 31981 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:17.901942 31981 ts_tablet_manager.cc:1403] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:10:17.904706 31926 tablet_service.cc:1511] Processing CreateTablet for tablet 4214d17086964585905299fae79f0bca (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:17.908154 31777 tablet_service.cc:1511] Processing CreateTablet for tablet 4214d17086964585905299fae79f0bca (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:17.908172 31921 tablet_service.cc:1511] Processing CreateTablet for tablet 48121309c18d4fad8d28f098002b9d02 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:17.910080 31777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4214d17086964585905299fae79f0bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.910317 31926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4214d17086964585905299fae79f0bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.913313 31921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48121309c18d4fad8d28f098002b9d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.920958 31980 tablet_bootstrap.cc:654] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.905599 31981 raft_consensus.cc:359] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:17.906585 31770 tablet_service.cc:1511] Processing CreateTablet for tablet 48121309c18d4fad8d28f098002b9d02 (DEFAULT_TABLE table=test-workload [id=38a831aa13cf4b788794e6905c3509d5]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:17.936951 31770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48121309c18d4fad8d28f098002b9d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.932809 31981 raft_consensus.cc:385] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:17.954145 31979 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:17.952039 31981 raft_consensus.cc:740] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:17.958907 31981 consensus_queue.cc:260] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:17.975474 31979 tablet_bootstrap.cc:654] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.982623 31980 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:17.983225 31980 ts_tablet_manager.cc:1403] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.003s
I20260709 13:10:17.986670 31980 raft_consensus.cc:359] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:17.987528 31980 raft_consensus.cc:385] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:17.987869 31980 raft_consensus.cc:740] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:17.988855 31980 consensus_queue.cc:260] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.000944 31981 ts_tablet_manager.cc:1434] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.099s	user 0.016s	sys 0.005s
I20260709 13:10:18.001832 31961 heartbeater.cc:499] Master 127.20.66.190:41769 was elected leader, sending a full tablet report...
I20260709 13:10:18.002607 31981 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.012454 31981 tablet_bootstrap.cc:654] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.013600 31980 ts_tablet_manager.cc:1434] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.003s
I20260709 13:10:18.014443 31812 heartbeater.cc:499] Master 127.20.66.190:41769 was elected leader, sending a full tablet report...
I20260709 13:10:18.014766 31980 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.025790 31986 raft_consensus.cc:493] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.029537 31980 tablet_bootstrap.cc:654] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.026520 31986 raft_consensus.cc:515] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.038828 31979 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.039405 31979 ts_tablet_manager.cc:1403] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.086s	user 0.012s	sys 0.009s
I20260709 13:10:18.042503 31986 leader_election.cc:290] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.043751 31979 raft_consensus.cc:359] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.044772 31979 raft_consensus.cc:385] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.045092 31979 raft_consensus.cc:740] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.046343 31979 consensus_queue.cc:260] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.062546 31981 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.063302 31981 ts_tablet_manager.cc:1403] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.061s	user 0.034s	sys 0.020s
I20260709 13:10:18.067098 31981 raft_consensus.cc:359] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.068079 31981 raft_consensus.cc:385] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.068425 31981 raft_consensus.cc:740] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.069530 31981 consensus_queue.cc:260] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.073721 31887 heartbeater.cc:499] Master 127.20.66.190:41769 was elected leader, sending a full tablet report...
I20260709 13:10:18.074203 31981 ts_tablet_manager.cc:1434] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:10:18.076584 31984 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.074467 31979 ts_tablet_manager.cc:1434] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.010s
I20260709 13:10:18.077134 31984 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.078408 31979 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.079903 31984 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:18.089777 31980 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.090634 31980 ts_tablet_manager.cc:1403] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.008s
I20260709 13:10:18.090811 31981 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.092401 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.101646 31936 raft_consensus.cc:2468] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.103093 31753 leader_election.cc:304] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:18.104048 31987 raft_consensus.cc:2804] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.104506 31987 raft_consensus.cc:493] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.104311 31980 raft_consensus.cc:359] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.104907 31987 raft_consensus.cc:3060] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.105207 31980 raft_consensus.cc:385] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.105518 31980 raft_consensus.cc:740] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
W20260709 13:10:18.106948 31813 tablet.cc:2406] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:18.112722 31979 tablet_bootstrap.cc:654] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.119925 31987 raft_consensus.cc:515] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.124711 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.125504 31862 raft_consensus.cc:2468] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.129581 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.126973 31980 consensus_queue.cc:260] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.131076 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:18.131808 31861 raft_consensus.cc:3060] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.131951 31980 ts_tablet_manager.cc:1434] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.031s	user 0.001s	sys 0.004s
I20260709 13:10:18.133183 31980 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.128513 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.133865 31987 leader_election.cc:290] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Requested vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
W20260709 13:10:18.137135 31902 leader_election.cc:343] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.138206 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.138908 31936 raft_consensus.cc:3060] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.140825 31981 tablet_bootstrap.cc:654] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.147168 31981 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.147330 31980 tablet_bootstrap.cc:654] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.147675 31981 ts_tablet_manager.cc:1403] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.000s
W20260709 13:10:18.149081 31900 leader_election.cc:343] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.149641 31900 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:18.150988 31984 raft_consensus.cc:2749] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.151921 31981 raft_consensus.cc:359] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.152724 31981 raft_consensus.cc:385] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.153050 31981 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.152745 31936 raft_consensus.cc:2468] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:18.153900 31981 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.154798 31979 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.155386 31979 ts_tablet_manager.cc:1403] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.010s
I20260709 13:10:18.155284 31753 leader_election.cc:304] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:18.156246 31987 raft_consensus.cc:2804] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.157454 31987 raft_consensus.cc:697] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Becoming Leader. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Running, Role: LEADER
I20260709 13:10:18.158309 31987 consensus_queue.cc:237] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.159765 31979 raft_consensus.cc:359] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.161172 31979 raft_consensus.cc:385] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.161613 31979 raft_consensus.cc:740] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.163228 31981 ts_tablet_manager.cc:1434] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:10:18.163362 31861 raft_consensus.cc:2468] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:18.162885 31979 consensus_queue.cc:260] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.166913 31979 ts_tablet_manager.cc:1434] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 13:10:18.168391 31979 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.173085 31981 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.178660 31979 tablet_bootstrap.cc:654] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.180315 31981 tablet_bootstrap.cc:654] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.187041 31682 catalog_manager.cc:5697] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.192481 31980 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.193377 31980 ts_tablet_manager.cc:1403] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.060s	user 0.025s	sys 0.021s
I20260709 13:10:18.196945 31979 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.197458 31981 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.197899 31979 ts_tablet_manager.cc:1403] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.007s
I20260709 13:10:18.197751 31980 raft_consensus.cc:359] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.198024 31981 ts_tablet_manager.cc:1403] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.009s
I20260709 13:10:18.198721 31980 raft_consensus.cc:385] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.199062 31980 raft_consensus.cc:740] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.199983 31980 consensus_queue.cc:260] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.201622 31981 raft_consensus.cc:359] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.202605 31981 raft_consensus.cc:385] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.202790 31980 ts_tablet_manager.cc:1434] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:10:18.202963 31981 raft_consensus.cc:740] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.203034 31979 raft_consensus.cc:359] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.203931 31980 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.204313 31979 raft_consensus.cc:385] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.204772 31979 raft_consensus.cc:740] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.203930 31981 consensus_queue.cc:260] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.205742 31979 consensus_queue.cc:260] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.210027 31990 raft_consensus.cc:493] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.210564 31990 raft_consensus.cc:515] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.211326 31981 ts_tablet_manager.cc:1434] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:10:18.213045 31981 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.213773 31980 tablet_bootstrap.cc:654] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.216569 31990 leader_election.cc:290] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.221304 31981 tablet_bootstrap.cc:654] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.223459 31979 ts_tablet_manager.cc:1434] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.003s
I20260709 13:10:18.226512 31979 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.236835 31984 raft_consensus.cc:493] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.237465 31984 raft_consensus.cc:515] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.239626 31984 leader_election.cc:290] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:18.242868 31980 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.243443 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
W20260709 13:10:18.246354 31902 leader_election.cc:343] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.243490 31980 ts_tablet_manager.cc:1403] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.021s
I20260709 13:10:18.249344 31979 tablet_bootstrap.cc:654] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.253618 31980 raft_consensus.cc:359] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.254437 31981 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.256484 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.257369 31980 raft_consensus.cc:385] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.257833 31980 raft_consensus.cc:740] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
W20260709 13:10:18.258795 31900 leader_election.cc:343] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.259145 31979 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.259279 31900 leader_election.cc:304] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:18.259723 31979 ts_tablet_manager.cc:1403] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.010s
I20260709 13:10:18.259919 31981 ts_tablet_manager.cc:1403] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.012s
I20260709 13:10:18.260545 31984 raft_consensus.cc:2749] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.258668 31980 consensus_queue.cc:260] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.263449 31981 raft_consensus.cc:359] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.264328 31981 raft_consensus.cc:385] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.264226 31979 raft_consensus.cc:359] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.264855 31981 raft_consensus.cc:740] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.265137 31979 raft_consensus.cc:385] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.265486 31979 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.266049 31981 consensus_queue.cc:260] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.266255 31979 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.267369 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.268638 31979 ts_tablet_manager.cc:1434] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:10:18.268770 31981 ts_tablet_manager.cc:1434] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:18.269758 31979 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.271772 31981 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.273409 31980 ts_tablet_manager.cc:1434] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:10:18.273838 31787 raft_consensus.cc:2468] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 0.
I20260709 13:10:18.274515 31980 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.275717 31828 leader_election.cc:304] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:18.276796 31990 raft_consensus.cc:2804] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.277271 31990 raft_consensus.cc:493] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.277617 31990 raft_consensus.cc:3060] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.278988 31981 tablet_bootstrap.cc:654] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.281132 31980 tablet_bootstrap.cc:654] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.285293 31979 tablet_bootstrap.cc:654] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.287645 31990 raft_consensus.cc:515] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.291874 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:18.292673 31787 raft_consensus.cc:3060] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.304363 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
W20260709 13:10:18.306167 31828 leader_election.cc:343] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.306313 31981 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.307370 31981 ts_tablet_manager.cc:1403] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.014s
I20260709 13:10:18.308123 31980 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.308390 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.308892 31980 ts_tablet_manager.cc:1403] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.007s
W20260709 13:10:18.309950 31828 leader_election.cc:343] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.310621 31990 leader_election.cc:290] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.310672 31981 raft_consensus.cc:359] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.311506 31981 raft_consensus.cc:385] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.311813 31981 raft_consensus.cc:740] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.312745 31787 raft_consensus.cc:2468] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:18.313071 31980 raft_consensus.cc:359] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.312582 31981 consensus_queue.cc:260] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.314224 31980 raft_consensus.cc:385] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.314851 31980 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.316035 31981 ts_tablet_manager.cc:1434] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:18.314244 31828 leader_election.cc:304] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:18.315991 31980 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.319235 31980 ts_tablet_manager.cc:1434] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:10:18.320196 31979 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.320276 31980 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.320823 31990 raft_consensus.cc:2804] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.323026 31981 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.320902 31979 ts_tablet_manager.cc:1403] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.017s
I20260709 13:10:18.328109 31980 tablet_bootstrap.cc:654] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.327697 31979 raft_consensus.cc:359] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.328629 31979 raft_consensus.cc:385] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.328948 31979 raft_consensus.cc:740] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.329908 31979 consensus_queue.cc:260] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.334545 31981 tablet_bootstrap.cc:654] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.342314 31990 raft_consensus.cc:697] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:18.343477 31990 consensus_queue.cc:237] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.344396 32012 raft_consensus.cc:493] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.344947 32012 raft_consensus.cc:515] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.348079 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.349200 31981 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
W20260709 13:10:18.349646 31828 leader_election.cc:343] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.349803 31981 ts_tablet_manager.cc:1403] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.002s
I20260709 13:10:18.352367 31979 ts_tablet_manager.cc:1434] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.007s
I20260709 13:10:18.353564 31979 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.353639 31980 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.354861 31980 ts_tablet_manager.cc:1403] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.017s
I20260709 13:10:18.355062 31981 raft_consensus.cc:359] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.359416 31980 raft_consensus.cc:359] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.356683 32012 leader_election.cc:290] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.360167 31981 raft_consensus.cc:385] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.357985 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.360592 31980 raft_consensus.cc:385] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.360873 31936 raft_consensus.cc:2468] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 0.
I20260709 13:10:18.361078 31980 raft_consensus.cc:740] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.362115 31981 raft_consensus.cc:740] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.362082 31980 consensus_queue.cc:260] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.363049 31981 consensus_queue.cc:260] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.363903 31828 leader_election.cc:304] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:18.365262 32012 raft_consensus.cc:2804] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.365667 31981 ts_tablet_manager.cc:1434] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:10:18.365703 32012 raft_consensus.cc:493] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.366379 32012 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.366933 31981 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.363158 31682 catalog_manager.cc:5697] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.374089 31981 tablet_bootstrap.cc:654] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.375778 31980 ts_tablet_manager.cc:1434] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:10:18.376876 31980 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.378901 31979 tablet_bootstrap.cc:654] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.379899 32012 raft_consensus.cc:515] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.388057 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:18.389777 31980 tablet_bootstrap.cc:654] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:18.390280 31828 leader_election.cc:343] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.393338 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.394184 31936 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.405141 31936 raft_consensus.cc:2468] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:18.405848 32012 leader_election.cc:290] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.408285 31828 leader_election.cc:304] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:18.409770 32012 raft_consensus.cc:2804] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.410964 31980 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.411079 32012 raft_consensus.cc:697] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:18.411372 31979 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.410964 31981 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.411908 31980 ts_tablet_manager.cc:1403] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.005s
I20260709 13:10:18.412199 31979 ts_tablet_manager.cc:1403] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.002s
I20260709 13:10:18.412432 31981 ts_tablet_manager.cc:1403] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.026s
I20260709 13:10:18.412425 32012 consensus_queue.cc:237] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.415380 31979 raft_consensus.cc:359] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.415518 31981 raft_consensus.cc:359] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.416368 31981 raft_consensus.cc:385] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.416190 31979 raft_consensus.cc:385] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.416939 31981 raft_consensus.cc:740] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.416967 31979 raft_consensus.cc:740] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.416100 31980 raft_consensus.cc:359] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.417666 31980 raft_consensus.cc:385] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.418113 31980 raft_consensus.cc:740] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.417927 31979 consensus_queue.cc:260] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.417968 31981 consensus_queue.cc:260] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.419160 31980 consensus_queue.cc:260] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.420373 31979 ts_tablet_manager.cc:1434] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:18.421470 31979 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.421528 31980 ts_tablet_manager.cc:1434] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:10:18.422716 31980 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.427500 31981 ts_tablet_manager.cc:1434] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:10:18.427649 31683 catalog_manager.cc:5697] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.429549 31980 tablet_bootstrap.cc:654] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.431183 31979 tablet_bootstrap.cc:654] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.428733 31981 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.442930 31981 tablet_bootstrap.cc:654] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.452602 31979 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.453241 31979 ts_tablet_manager.cc:1403] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.014s
I20260709 13:10:18.454021 31980 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.454725 31980 ts_tablet_manager.cc:1403] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.016s
I20260709 13:10:18.456087 31979 raft_consensus.cc:359] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.456727 31979 raft_consensus.cc:385] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.456979 31979 raft_consensus.cc:740] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.457859 31980 raft_consensus.cc:359] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.457823 31979 consensus_queue.cc:260] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.458709 31980 raft_consensus.cc:385] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.459075 31980 raft_consensus.cc:740] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.459888 31981 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.459848 31980 consensus_queue.cc:260] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.460439 31981 ts_tablet_manager.cc:1403] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.006s
I20260709 13:10:18.460778 31979 ts_tablet_manager.cc:1434] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:10:18.461978 31979 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.462198 31980 ts_tablet_manager.cc:1434] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:18.463526 31980 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.465189 31987 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.465732 31987 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.467834 31981 raft_consensus.cc:359] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.468686 31981 raft_consensus.cc:385] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.468989 31981 raft_consensus.cc:740] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.469838 31981 consensus_queue.cc:260] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.470543 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.471236 31861 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.472271 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.472935 31936 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.468058 31987 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.474645 31751 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:18.475471 31987 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.475827 31987 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.476158 31987 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.478153 32012 raft_consensus.cc:493] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.478828 32012 raft_consensus.cc:515] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.479751 31986 raft_consensus.cc:493] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.480288 31986 raft_consensus.cc:515] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.482391 32012 leader_election.cc:290] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.482980 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.482411 31986 leader_election.cc:290] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.484074 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.483667 31936 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 0.
W20260709 13:10:18.485549 31751 leader_election.cc:343] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.486404 31828 leader_election.cc:304] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:18.486480 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.487196 31787 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 0.
I20260709 13:10:18.487190 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.489030 31979 tablet_bootstrap.cc:654] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.490015 31936 raft_consensus.cc:2468] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.490535 32012 raft_consensus.cc:2804] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.487138 31981 ts_tablet_manager.cc:1434] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:10:18.491067 32012 raft_consensus.cc:493] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.491070 31753 leader_election.cc:304] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7
I20260709 13:10:18.491906 32012 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.492321 31980 tablet_bootstrap.cc:654] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.492663 31986 raft_consensus.cc:2804] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.493146 31986 raft_consensus.cc:493] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.493542 31986 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.494282 31981 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: Bootstrap starting.
I20260709 13:10:18.490046 31987 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.497848 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:18.498450 31861 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.500507 31981 tablet_bootstrap.cc:654] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.500844 32012 raft_consensus.cc:515] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.504545 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:18.504908 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.505196 31787 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.505578 31936 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.506966 31861 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:18.508559 31751 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:18.513152 31936 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:18.514703 32012 leader_election.cc:290] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.514525 31935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.514614 31828 leader_election.cc:304] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:18.515705 31935 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.516580 31787 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:18.517221 31987 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Requested vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.518024 31987 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.518587 31987 raft_consensus.cc:697] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Becoming Leader. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Running, Role: LEADER
I20260709 13:10:18.519696 31987 consensus_queue.cc:237] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.521270 31990 raft_consensus.cc:2804] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.521929 31990 raft_consensus.cc:697] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:18.521713 31986 raft_consensus.cc:515] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.522933 31990 consensus_queue.cc:237] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.524685 31986 leader_election.cc:290] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Requested vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.525959 31986 raft_consensus.cc:493] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.526496 31986 raft_consensus.cc:515] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
W20260709 13:10:18.530222 31883 debug-util.cc:398] Leaking SignalData structure 0x7b08001d28c0 after lost signal to thread 20749
I20260709 13:10:18.530493 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.531077 31986 leader_election.cc:290] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.531415 31934 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.533532 31753 leader_election.cc:304] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:18.533664 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.534726 31862 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:18.528483 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:18.539366 31986 raft_consensus.cc:2804] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.539848 31986 raft_consensus.cc:493] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:18.540064 31751 leader_election.cc:343] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.540299 31986 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.544729 32012 raft_consensus.cc:493] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.545604 32012 raft_consensus.cc:515] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.547909 32027 consensus_queue.cc:1048] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:18.550203 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
W20260709 13:10:18.553185 31828 leader_election.cc:343] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.533658 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.598809 32012 leader_election.cc:290] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.600240 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.601996 31981 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: No bootstrap required, opened a new log
I20260709 13:10:18.602537 31981 ts_tablet_manager.cc:1403] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: Time spent bootstrapping tablet: real 0.108s	user 0.021s	sys 0.025s
I20260709 13:10:18.595875 31936 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.606057 31935 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:18.605512 31684 catalog_manager.cc:5697] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.609510 31980 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.609297 31981 raft_consensus.cc:359] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.609994 31980 ts_tablet_manager.cc:1403] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.147s	user 0.019s	sys 0.019s
I20260709 13:10:18.609998 31981 raft_consensus.cc:385] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.615626 31981 raft_consensus.cc:740] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.616882 31981 consensus_queue.cc:260] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
W20260709 13:10:18.625141 31828 leader_election.cc:343] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.626870 31981 ts_tablet_manager.cc:1434] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:10:18.628336 32027 raft_consensus.cc:493] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.625895 31828 leader_election.cc:304] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7; no voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:18.642179 32012 raft_consensus.cc:2749] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.643286 31987 consensus_queue.cc:1048] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:18.636040 32027 raft_consensus.cc:515] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.646792 31936 raft_consensus.cc:2468] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:18.654047 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.656024 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.658658 31934 raft_consensus.cc:2393] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35dabe2c8ebd4db5a12e2d57d589c650 in current term 1: Already voted for candidate 142ed2f6af504e9583ab5c467be2b4d7 in this term.
I20260709 13:10:18.660491 31753 leader_election.cc:304] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7
I20260709 13:10:18.663693 32032 raft_consensus.cc:2804] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.657313 32027 leader_election.cc:290] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.673238 31979 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.673851 31979 ts_tablet_manager.cc:1403] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.212s	user 0.019s	sys 0.006s
I20260709 13:10:18.677285 31979 raft_consensus.cc:359] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.677670 31751 leader_election.cc:304] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:18.678117 31979 raft_consensus.cc:385] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.678515 31979 raft_consensus.cc:740] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.679380 31979 consensus_queue.cc:260] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.668216 31980 raft_consensus.cc:359] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.680929 31980 raft_consensus.cc:385] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.681324 31980 raft_consensus.cc:740] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.673806 31986 raft_consensus.cc:515] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.683738 31990 raft_consensus.cc:493] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.684044 31979 ts_tablet_manager.cc:1434] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:18.685310 31979 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.684968 31980 consensus_queue.cc:260] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.688716 31984 raft_consensus.cc:493] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.689297 31984 raft_consensus.cc:515] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.690209 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:18.691035 31861 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.694923 31980 ts_tablet_manager.cc:1434] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 13:10:18.697469 31980 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:18.686373 32027 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.679762 31683 catalog_manager.cc:5697] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.684260 31990 raft_consensus.cc:515] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.703998 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:18.705276 31990 leader_election.cc:290] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.706007 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.706786 31787 raft_consensus.cc:2393] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c37cf16aa11340c79c452627e7d8c6f4 in current term 1: Already voted for candidate 142ed2f6af504e9583ab5c467be2b4d7 in this term.
I20260709 13:10:18.708117 31900 leader_election.cc:304] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:18.706765 31979 tablet_bootstrap.cc:654] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.696591 31984 leader_election.cc:290] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:18.721743 31984 raft_consensus.cc:3060] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.709857 32032 raft_consensus.cc:697] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Becoming Leader. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Running, Role: LEADER
I20260709 13:10:18.724350 32032 consensus_queue.cc:237] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.715754 31986 leader_election.cc:290] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Requested vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.717087 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.731814 31936 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.718679 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48121309c18d4fad8d28f098002b9d02" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:18.737532 31934 raft_consensus.cc:2468] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 0.
I20260709 13:10:18.720265 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48121309c18d4fad8d28f098002b9d02" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.739430 31787 raft_consensus.cc:2468] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 0.
I20260709 13:10:18.740767 31828 leader_election.cc:304] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:18.747227 31990 raft_consensus.cc:2804] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.747803 31990 raft_consensus.cc:493] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.748222 31990 raft_consensus.cc:3060] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.772601 31984 raft_consensus.cc:2749] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.775326 31861 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:18.775753 32027 raft_consensus.cc:2749] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.777215 31751 leader_election.cc:304] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:18.777714 32012 consensus_queue.cc:1048] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:18.778434 32027 raft_consensus.cc:2804] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.779332 32027 raft_consensus.cc:697] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Becoming Leader. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Running, Role: LEADER
I20260709 13:10:18.780516 32027 consensus_queue.cc:237] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.804515 31936 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
W20260709 13:10:18.806571 31900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.130:33599, user_credentials={real_user=slave}} blocked reactor thread for 98610us
I20260709 13:10:18.814528 31980 tablet_bootstrap.cc:654] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.809682 31990 raft_consensus.cc:515] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.822570 32039 raft_consensus.cc:493] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.823369 32039 raft_consensus.cc:515] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.822844 31683 catalog_manager.cc:5697] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.826339 32039 leader_election.cc:290] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:18.827603 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:18.828928 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
W20260709 13:10:18.830730 31900 leader_election.cc:343] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:18.831243 31902 leader_election.cc:343] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.828621 31990 leader_election.cc:290] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:18.831691 31902 leader_election.cc:304] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:18.833799 31979 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.834373 31979 ts_tablet_manager.cc:1403] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.149s	user 0.022s	sys 0.044s
I20260709 13:10:18.833976 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48121309c18d4fad8d28f098002b9d02" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:18.835181 31936 raft_consensus.cc:3060] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.838269 31979 raft_consensus.cc:359] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.839267 31979 raft_consensus.cc:385] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.839380 31787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48121309c18d4fad8d28f098002b9d02" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:18.839692 31979 raft_consensus.cc:740] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.840066 31787 raft_consensus.cc:3060] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.840687 31979 consensus_queue.cc:260] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.842903 31979 ts_tablet_manager.cc:1434] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:10:18.843844 32012 consensus_queue.cc:1048] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:18.847682 31979 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: Bootstrap starting.
I20260709 13:10:18.840911 32039 raft_consensus.cc:2749] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.856169 31787 raft_consensus.cc:2468] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:18.857623 31828 leader_election.cc:304] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:18.859164 31979 tablet_bootstrap.cc:654] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.859078 32012 raft_consensus.cc:2804] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.860004 32012 raft_consensus.cc:697] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:18.861233 32012 consensus_queue.cc:237] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.862772 31936 raft_consensus.cc:2468] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:18.866860 31980 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: No bootstrap required, opened a new log
I20260709 13:10:18.867499 31980 ts_tablet_manager.cc:1403] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.172s	user 0.021s	sys 0.004s
I20260709 13:10:18.870505 31980 raft_consensus.cc:359] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.871435 31980 raft_consensus.cc:385] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.871774 31980 raft_consensus.cc:740] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.872762 31980 consensus_queue.cc:260] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.875469 31980 ts_tablet_manager.cc:1434] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:18.875976 32016 consensus_queue.cc:1048] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:18.882292 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:18.906049 31683 catalog_manager.cc:5697] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.936923 31682 catalog_manager.cc:5697] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:18.948457 31828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:46117, user_credentials={real_user=slave}} blocked reactor thread for 62726.4us
I20260709 13:10:18.949157 31979 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: No bootstrap required, opened a new log
I20260709 13:10:18.949741 31979 ts_tablet_manager.cc:1403] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent bootstrapping tablet: real 0.102s	user 0.043s	sys 0.029s
I20260709 13:10:18.952011 31990 consensus_queue.cc:1048] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:18.952850 31979 raft_consensus.cc:359] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.953696 31979 raft_consensus.cc:385] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.954001 31979 raft_consensus.cc:740] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.954754 31979 consensus_queue.cc:260] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:18.956861 31979 ts_tablet_manager.cc:1434] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:18.996446 31990 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:18.997370 32051 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.055326 32039 raft_consensus.cc:493] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.055819 32039 raft_consensus.cc:515] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:19.057626 32039 leader_election.cc:290] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:19.058669 31786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" is_pre_election: true
I20260709 13:10:19.058835 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:19.059317 31786 raft_consensus.cc:2468] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 0.
I20260709 13:10:19.059643 31861 raft_consensus.cc:2468] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 0.
I20260709 13:10:19.060442 31902 leader_election.cc:304] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:19.061359 32039 raft_consensus.cc:2804] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.061731 32039 raft_consensus.cc:493] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.062014 32039 raft_consensus.cc:3060] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.068351 32039 raft_consensus.cc:515] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:19.070199 32039 leader_election.cc:290] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:19.071048 31786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:19.071733 31786 raft_consensus.cc:3060] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.071870 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:19.072563 31861 raft_consensus.cc:3060] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.076846 31786 raft_consensus.cc:2468] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 1.
I20260709 13:10:19.078030 31902 leader_election.cc:304] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:19.078760 32039 raft_consensus.cc:2804] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.079200 32039 raft_consensus.cc:697] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:19.080066 32039 consensus_queue.cc:237] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:19.080814 31861 raft_consensus.cc:2468] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 1.
I20260709 13:10:19.088755 32032 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.090114 31683 catalog_manager.cc:5697] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.104302 32027 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.155193 32038 raft_consensus.cc:493] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.155002 32027 consensus_queue.cc:1048] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.155747 32038 raft_consensus.cc:515] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:19.157799 32038 leader_election.cc:290] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:19.159272 31936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:19.160006 31936 raft_consensus.cc:2468] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:19.161337 31753 leader_election.cc:304] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:19.162158 32038 raft_consensus.cc:2804] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.159348 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:19.162683 32038 raft_consensus.cc:493] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.163045 32038 raft_consensus.cc:3060] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.162947 31861 raft_consensus.cc:2468] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 0.
I20260709 13:10:19.176025 32038 raft_consensus.cc:515] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:19.177927 31861 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.179009 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:19.179703 31862 raft_consensus.cc:3060] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.180454 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:19.181145 31933 raft_consensus.cc:3060] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.185227 32038 leader_election.cc:290] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Requested vote from peers 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:19.188680 31862 raft_consensus.cc:2468] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:19.188724 31933 raft_consensus.cc:2468] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dabe2c8ebd4db5a12e2d57d589c650 in term 1.
I20260709 13:10:19.190109 31751 leader_election.cc:304] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:19.191095 32038 raft_consensus.cc:2804] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.191588 32038 raft_consensus.cc:697] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Becoming Leader. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Running, Role: LEADER
I20260709 13:10:19.192533 32038 consensus_queue.cc:237] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:19.196990 32038 consensus_queue.cc:1048] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.203411 31682 catalog_manager.cc:5697] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.292752 32038 consensus_queue.cc:1048] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.306432 32027 consensus_queue.cc:1048] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.308324 31990 consensus_queue.cc:1048] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.309099 20746 ts_manager.cc:295] Set tserver state for 35dabe2c8ebd4db5a12e2d57d589c650 to MAINTENANCE_MODE
I20260709 13:10:19.310092 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
W20260709 13:10:19.314311 31828 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 209 similar messages]
W20260709 13:10:19.388207 31828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:44397, user_credentials={real_user=slave}} blocked reactor thread for 73561.5us
W20260709 13:10:19.408749 31828 consensus_peers.cc:597] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:19.413553 31828 consensus_peers.cc:597] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:19.418951 31828 consensus_peers.cc:597] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:19.437722 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:19.438639 20746 tablet_replica.cc:333] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.439574 20746 raft_consensus.cc:2243] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:19.440176 20746 raft_consensus.cc:2272] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.442898 20746 tablet_replica.cc:333] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.443789 20746 raft_consensus.cc:2243] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:19.444928 20746 raft_consensus.cc:2272] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.448210 20746 tablet_replica.cc:333] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.449092 20746 raft_consensus.cc:2243] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:19.449666 20746 raft_consensus.cc:2272] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.452838 20746 tablet_replica.cc:333] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.453696 20746 raft_consensus.cc:2243] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:19.454182 20746 raft_consensus.cc:2272] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.456513 20746 tablet_replica.cc:333] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.457273 20746 raft_consensus.cc:2243] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:19.458066 20746 pending_rounds.cc:70] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Trying to abort 1 pending ops.
I20260709 13:10:19.458369 20746 pending_rounds.cc:77] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636328223469568 op_type: NO_OP noop_request { }
I20260709 13:10:19.458856 20746 raft_consensus.cc:2889] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:19.459331 20746 raft_consensus.cc:2272] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.463007 20746 tablet_replica.cc:333] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.464082 20746 raft_consensus.cc:2243] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:19.464807 20746 raft_consensus.cc:2272] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.470099 20746 tablet_replica.cc:333] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.470685 20746 raft_consensus.cc:2243] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:19.471563 20746 raft_consensus.cc:2272] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.474009 20746 tablet_replica.cc:333] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.474560 20746 raft_consensus.cc:2243] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:19.475482 20746 raft_consensus.cc:2272] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.477903 20746 tablet_replica.cc:333] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.478525 20746 raft_consensus.cc:2243] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:19.478988 20746 raft_consensus.cc:2272] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.481446 20746 tablet_replica.cc:333] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: stopping tablet replica
I20260709 13:10:19.481951 20746 raft_consensus.cc:2243] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:19.482681 20746 raft_consensus.cc:2272] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:19.493592 32016 consensus_queue.cc:1048] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:19.522390 31828 consensus_peers.cc:597] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:19.522254 32039 consensus_queue.cc:1048] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.528095 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
W20260709 13:10:19.550841 31902 consensus_peers.cc:597] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:19.883533 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:20.568120 32051 raft_consensus.cc:493] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.569162 32051 raft_consensus.cc:515] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.571491 32051 leader_election.cc:290] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:20.574791 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
W20260709 13:10:20.579337 31828 leader_election.cc:336] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.579928 31828 leader_election.cc:304] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7; no voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:20.580843 32051 raft_consensus.cc:2749] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:20.608215 32067 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.609035 32067 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.611568 32051 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.612293 32051 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.613660 32067 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:20.615193 32051 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:20.615352 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:20.616649 31861 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 1.
I20260709 13:10:20.616766 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:20.617627 31933 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:20.618093 31900 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:20.618985 32067 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:20.619347 32067 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.619710 32067 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:20.620144 31902 leader_election.cc:336] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.622210 31828 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:20.624475 32051 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:20.624976 32051 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.625519 32051 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:20.630789 31828 leader_election.cc:336] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.632923 32067 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.634436 32051 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.635836 32067 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:20.636549 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:20.636896 32051 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:20.637409 31861 raft_consensus.cc:2393] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c37cf16aa11340c79c452627e7d8c6f4 in current term 2: Already voted for candidate 142ed2f6af504e9583ab5c467be2b4d7 in this term.
I20260709 13:10:20.639689 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:20.640609 31933 raft_consensus.cc:2393] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 142ed2f6af504e9583ab5c467be2b4d7 in current term 2: Already voted for candidate c37cf16aa11340c79c452627e7d8c6f4 in this term.
W20260709 13:10:20.642053 31902 leader_election.cc:336] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
W20260709 13:10:20.642561 31828 leader_election.cc:336] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.642700 31902 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.643107 31828 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7; no voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:20.644074 32051 raft_consensus.cc:2749] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:20.644126 32067 raft_consensus.cc:2749] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:20.669481 32067 raft_consensus.cc:493] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.670070 32067 raft_consensus.cc:515] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.672185 32067 leader_election.cc:290] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:20.673508 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
W20260709 13:10:20.676038 31902 leader_election.cc:336] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.676476 31902 leader_election.cc:304] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.677163 32067 raft_consensus.cc:2749] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:20.700903 32067 raft_consensus.cc:493] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.701479 32067 raft_consensus.cc:515] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.702366 32051 raft_consensus.cc:493] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.703471 32067 leader_election.cc:290] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:20.703001 32051 raft_consensus.cc:515] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.704713 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:20.705503 31861 raft_consensus.cc:2468] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 1.
I20260709 13:10:20.706036 32051 leader_election.cc:290] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
W20260709 13:10:20.707783 31902 leader_election.cc:336] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.708658 31900 leader_election.cc:304] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.709195 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:20.709764 32067 raft_consensus.cc:2804] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:20.709820 31933 raft_consensus.cc:2468] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:20.710315 32067 raft_consensus.cc:493] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.710690 32067 raft_consensus.cc:3060] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:20.711838 31828 leader_election.cc:304] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:20.712751 32051 raft_consensus.cc:2804] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:10:20.712816 31828 leader_election.cc:336] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.713236 32051 raft_consensus.cc:493] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.713658 32051 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:20.718010 32067 raft_consensus.cc:515] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.720906 32067 leader_election.cc:290] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:20.721135 32051 raft_consensus.cc:515] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.722565 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "832d464d4b3a43b0a1652b7c1e81e9aa" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:20.723251 31861 raft_consensus.cc:3060] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:20.724296 32051 leader_election.cc:290] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:20.725797 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:20.726390 31933 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:20.727918 31902 leader_election.cc:336] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.732689 31861 raft_consensus.cc:2468] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
W20260709 13:10:20.733759 31828 leader_election.cc:336] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.734157 31933 raft_consensus.cc:2468] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 2.
I20260709 13:10:20.734659 31900 leader_election.cc:304] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.735476 32067 raft_consensus.cc:2804] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:20.735357 31828 leader_election.cc:304] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.736362 32051 raft_consensus.cc:2804] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:20.737059 32051 raft_consensus.cc:697] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:20.736925 32067 raft_consensus.cc:697] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:20.739059 32067 consensus_queue.cc:237] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.739066 32051 consensus_queue.cc:237] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.749289 31682 catalog_manager.cc:5697] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 1 to 2, leader changed from 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:20.749832 31683 catalog_manager.cc:5697] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 1 to 2, leader changed from 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:20.810223 32067 raft_consensus.cc:493] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.810969 32067 raft_consensus.cc:515] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.813414 32067 leader_election.cc:290] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:20.814730 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
W20260709 13:10:20.817170 31902 leader_election.cc:336] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.817576 31902 leader_election.cc:304] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c37cf16aa11340c79c452627e7d8c6f4; no voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.818225 32067 raft_consensus.cc:2749] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:20.884691 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:20.892285 32051 raft_consensus.cc:493] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.892987 32051 raft_consensus.cc:515] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.895349 32051 leader_election.cc:290] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:20.896952 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:20.897667 31933 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
I20260709 13:10:20.898809 31828 leader_election.cc:304] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
W20260709 13:10:20.899456 31828 leader_election.cc:336] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.899471 32051 raft_consensus.cc:2804] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:20.900000 32051 raft_consensus.cc:493] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 35dabe2c8ebd4db5a12e2d57d589c650)
I20260709 13:10:20.900400 32051 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:20.908763 32051 raft_consensus.cc:515] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.911084 32051 leader_election.cc:290] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:20.913578 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:20.914449 31933 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:20.916146 31828 leader_election.cc:336] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:20.921890 31933 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 2.
I20260709 13:10:20.923238 31828 leader_election.cc:304] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:20.924072 32051 raft_consensus.cc:2804] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:20.924670 32051 raft_consensus.cc:697] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:20.925786 32051 consensus_queue.cc:237] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:20.936168 31683 catalog_manager.cc:5697] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 1 to 2, leader changed from 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:21.130339 31861 raft_consensus.cc:1275] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:21.132105 32067 consensus_queue.cc:1048] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:21.149434 31902 consensus_peers.cc:597] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:21.245343 31828 consensus_peers.cc:597] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:21.279517 31933 raft_consensus.cc:1275] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:21.281142 32051 consensus_queue.cc:1048] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:21.313568 31933 raft_consensus.cc:1275] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:21.316109 32086 consensus_queue.cc:1048] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:21.338042 32016 raft_consensus.cc:493] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.338714 32016 raft_consensus.cc:515] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:21.341266 32016 leader_election.cc:290] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:21.345254 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4" is_pre_election: true
I20260709 13:10:21.346307 31933 raft_consensus.cc:2468] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 1.
W20260709 13:10:21.345328 31828 consensus_peers.cc:597] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:21.348783 31828 leader_election.cc:304] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:21.349810 32016 raft_consensus.cc:2804] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:10:21.349880 31828 leader_election.cc:336] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:21.350438 32016 raft_consensus.cc:493] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.350852 32016 raft_consensus.cc:3060] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:21.358583 32016 raft_consensus.cc:515] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:21.360675 32016 leader_election.cc:290] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:21.362993 31933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4214d17086964585905299fae79f0bca" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:21.363719 31933 raft_consensus.cc:3060] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:21.366778 31828 leader_election.cc:336] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:21.370671 31933 raft_consensus.cc:2468] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 2.
I20260709 13:10:21.372027 31828 leader_election.cc:304] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:21.373131 32016 raft_consensus.cc:2804] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:21.373648 32016 raft_consensus.cc:697] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:21.374641 32016 consensus_queue.cc:237] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:21.385186 31683 catalog_manager.cc:5697] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 1 to 2, leader changed from 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:21.801469 31828 consensus_peers.cc:597] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:21.807410 31828 consensus_peers.cc:597] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:21.875934 31828 consensus_peers.cc:597] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:21.876472 31828 consensus_peers.cc:597] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:21.881372 31933 raft_consensus.cc:1275] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:21.883242 32051 consensus_queue.cc:1048] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:21.885922 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:21.951508 31828 consensus_peers.cc:597] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:22.211072 31902 consensus_peers.cc:597] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:22.573318 32067 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.574357 32067 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:22.576828 32067 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:22.578908 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7" is_pre_election: true
I20260709 13:10:22.579984 31861 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:22.582043 31900 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
W20260709 13:10:22.582825 31902 leader_election.cc:336] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:22.582976 32067 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:22.583436 32067 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.583719 32067 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:22.592178 32067 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:22.593916 32067 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:22.595125 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:22.595758 31861 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:22.597772 31902 leader_election.cc:336] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111)
I20260709 13:10:22.602835 31861 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 3.
I20260709 13:10:22.604015 31900 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 35dabe2c8ebd4db5a12e2d57d589c650
I20260709 13:10:22.604774 32067 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:22.605692 32067 raft_consensus.cc:697] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:22.606871 32067 consensus_queue.cc:237] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:22.623193 31683 catalog_manager.cc:5697] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 1 to 3, leader changed from 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:22.887372 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:23.025933 31861 raft_consensus.cc:1275] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:23.027710 32067 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:23.044349 31902 consensus_peers.cc:597] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:23.482124 31828 consensus_peers.cc:597] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:23.675745 31902 consensus_peers.cc:597] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:23.888550 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:23.934137 31828 consensus_peers.cc:597] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:24.111881 31828 consensus_peers.cc:597] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:24.158157 31828 consensus_peers.cc:597] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:24.356977 31902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111) [suppressed 96 similar messages]
W20260709 13:10:24.363637 31902 consensus_peers.cc:597] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:24.412492 31828 consensus_peers.cc:597] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:24.522964 31828 consensus_peers.cc:597] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:24.548643 31828 consensus_peers.cc:597] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:24.623595 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:24.634637 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:24.635295 32111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:24.637589 32113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:24.640004 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:24.644899 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:24.645335 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:24.645496 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602624645483 us; error 0 us; skew 500 ppm
I20260709 13:10:24.646078 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:24.649964 20746 webserver.cc:533] Webserver started at http://127.20.66.129:35263/ using document root <none> and password file <none>
I20260709 13:10:24.650702 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:24.650895 20746 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:10:24.656654 20746 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:24.662515 32118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:24.663568 20746 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:10:24.663921 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
format_stamp: "Formatted at 2026-07-09 13:10:17 on dist-test-slave-rgrd"
I20260709 13:10:24.664323 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:24.680958 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:24.682622 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:24.684870 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:24.690083 32125 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:10:24.763304 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:10:24.763679 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.075s	user 0.000s	sys 0.004s
I20260709 13:10:24.764046 20746 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:10:24.771689 32125 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.792610 32125 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:24.793859 32125 tablet_bootstrap.cc:492] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.794667 32125 ts_tablet_manager.cc:1403] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.008s
I20260709 13:10:24.798424 32125 raft_consensus.cc:359] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.799358 32125 raft_consensus.cc:740] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.800278 32125 consensus_queue.cc:260] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.804015 32125 ts_tablet_manager.cc:1434] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:10:24.805222 32125 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.822019 32125 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:24.824352 32125 tablet_bootstrap.cc:492] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.825376 32125 ts_tablet_manager.cc:1403] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 13:10:24.829294 32125 raft_consensus.cc:359] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.830078 32125 raft_consensus.cc:740] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.830956 32125 consensus_queue.cc:260] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.833779 32125 ts_tablet_manager.cc:1434] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:10:24.834786 32125 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.838950 20746 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:10:24.839870 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.076s	user 0.069s	sys 0.004s
I20260709 13:10:24.857813 32125 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:24.859395 32125 tablet_bootstrap.cc:492] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.860476 32125 ts_tablet_manager.cc:1403] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.008s
I20260709 13:10:24.864759 32125 raft_consensus.cc:359] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.865510 32125 raft_consensus.cc:740] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.866403 32125 consensus_queue.cc:260] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.868884 32125 ts_tablet_manager.cc:1434] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:10:24.869753 32125 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.889585 32125 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:10:24.890329 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:24.890741 32125 tablet_bootstrap.cc:492] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.891573 32125 ts_tablet_manager.cc:1403] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.005s
I20260709 13:10:24.894290 32125 raft_consensus.cc:359] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.894917 32125 raft_consensus.cc:740] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.895674 32125 consensus_queue.cc:260] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.897938 32125 ts_tablet_manager.cc:1434] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:24.899192 32125 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.911808 32125 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:24.912606 32125 tablet_bootstrap.cc:492] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.913380 32125 ts_tablet_manager.cc:1403] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:10:24.916146 32125 raft_consensus.cc:359] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.916777 32125 raft_consensus.cc:740] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.917565 32125 consensus_queue.cc:260] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.919652 32125 ts_tablet_manager.cc:1434] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:24.920452 32125 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.926465 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:44397
I20260709 13:10:24.926582 32187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:44397 every 8 connection(s)
I20260709 13:10:24.935199 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:24.944782 32125 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:24.947625 32125 tablet_bootstrap.cc:492] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.948836 32125 ts_tablet_manager.cc:1403] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.029s	user 0.020s	sys 0.000s
I20260709 13:10:24.953210 32125 raft_consensus.cc:359] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.954304 32125 raft_consensus.cc:740] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.955410 32125 consensus_queue.cc:260] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:24.958453 32125 ts_tablet_manager.cc:1434] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:10:24.959714 32125 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:24.977051 32162 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:24.990154 32161 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:10:24.991425 32125 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:24.993450 32188 heartbeater.cc:344] Connected to a master server at 127.20.66.190:41769
I20260709 13:10:24.994035 32188 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:24.994978 31861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:24.993494 32125 tablet_bootstrap.cc:492] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:24.995854 31861 raft_consensus.cc:606] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:24.996477 32125 ts_tablet_manager.cc:1403] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.008s
I20260709 13:10:24.995288 32188 heartbeater.cc:507] Master 127.20.66.190:41769 requested a full tablet report, sending...
I20260709 13:10:24.999429 32160 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:24.999764 32125 raft_consensus.cc:359] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.000634 32125 raft_consensus.cc:740] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.001909 32125 consensus_queue.cc:260] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.003263 31861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62978121cded40708c2fe686051eafa4"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:25.004025 31861 raft_consensus.cc:606] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:25.004462 32125 ts_tablet_manager.cc:1434] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:25.005548 32125 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:25.008275 31861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:25.016537 31861 raft_consensus.cc:606] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:25.018633 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:10:25.019440 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:25.020699 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:25.028790 32125 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:25.029961 32125 tablet_bootstrap.cc:492] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:25.030763 32125 ts_tablet_manager.cc:1403] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.007s
I20260709 13:10:25.033754 32125 raft_consensus.cc:359] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.034605 32125 raft_consensus.cc:740] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.035647 32125 consensus_queue.cc:260] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.039858 32125 ts_tablet_manager.cc:1434] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:10:25.040726 32125 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:25.042176 31682 ts_manager.cc:194] Re-registered known tserver with Master: 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397)
I20260709 13:10:25.053315 31682 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44990
I20260709 13:10:25.055608 32125 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:10:25.056739 32125 tablet_bootstrap.cc:492] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:25.056890 32188 heartbeater.cc:499] Master 127.20.66.190:41769 was elected leader, sending a full tablet report...
I20260709 13:10:25.057456 32125 ts_tablet_manager.cc:1403] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 13:10:25.059949 32125 raft_consensus.cc:359] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.060700 32125 raft_consensus.cc:740] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.061465 32125 consensus_queue.cc:260] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.063560 32125 ts_tablet_manager.cc:1434] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:25.064369 32125 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap starting.
I20260709 13:10:25.078415 32125 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:25.079182 32125 tablet_bootstrap.cc:492] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Bootstrap complete.
I20260709 13:10:25.079764 32125 ts_tablet_manager.cc:1403] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:10:25.082114 32125 raft_consensus.cc:359] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.082724 32125 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dabe2c8ebd4db5a12e2d57d589c650, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.083325 32125 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.084879 32125 ts_tablet_manager.cc:1434] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:10:25.107635 32160 raft_consensus.cc:3060] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.202947 32108 raft_consensus.cc:993] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:25.203483 32207 raft_consensus.cc:1081] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:25.205143 31933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:25.205616 31933 raft_consensus.cc:493] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:25.205857 31933 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.211824 31933 raft_consensus.cc:515] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.213521 31933 leader_election.cc:290] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:25.214488 32160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:25.214540 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:25.215142 32160 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.215283 31861 raft_consensus.cc:3055] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:25.215678 31861 raft_consensus.cc:740] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:25.216516 31861 consensus_queue.cc:260] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.217763 31861 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.222303 32108 raft_consensus.cc:993] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:25.222303 32160 raft_consensus.cc:2468] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:25.223091 32108 raft_consensus.cc:1081] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:25.223827 31902 leader_election.cc:304] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:25.224887 32109 raft_consensus.cc:2804] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:25.224797 31933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:25.225700 31933 raft_consensus.cc:493] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:25.226071 31933 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:25.226388 32109 raft_consensus.cc:697] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:25.227583 32109 consensus_queue.cc:237] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.231809 31861 raft_consensus.cc:2468] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:25.233091 31933 raft_consensus.cc:515] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.235917 31933 leader_election.cc:290] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:25.236675 32160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:25.237377 32160 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:25.236991 31683 catalog_manager.cc:5697] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 1 to 2, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.237941 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872ddbd361a64e06a386a4f1b6d8da8d" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:25.238595 31862 raft_consensus.cc:3055] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:25.238924 31862 raft_consensus.cc:740] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
W20260709 13:10:25.239418 31716 debug-util.cc:398] Leaking SignalData structure 0x7b08000322c0 after lost signal to thread 20749
I20260709 13:10:25.239818 31862 consensus_queue.cc:260] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.241304 31862 raft_consensus.cc:3060] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:25.242357 31716 debug-util.cc:398] Leaking SignalData structure 0x7b08003754a0 after lost signal to thread 31960
I20260709 13:10:25.245193 32160 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 3.
I20260709 13:10:25.246613 31902 leader_election.cc:304] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:25.247627 32109 raft_consensus.cc:2804] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:25.248168 32109 raft_consensus.cc:697] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:25.249384 32109 consensus_queue.cc:237] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.249818 31862 raft_consensus.cc:2468] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 3.
I20260709 13:10:25.270242 31683 catalog_manager.cc:5697] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 2 to 3, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.383562 32207 raft_consensus.cc:993] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:25.384219 32108 raft_consensus.cc:1081] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:25.386023 31933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62978121cded40708c2fe686051eafa4"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:25.386629 31933 raft_consensus.cc:493] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:25.386934 31933 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.392488 31933 raft_consensus.cc:515] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.396605 32160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:25.397362 32160 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.396149 31933 leader_election.cc:290] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:25.398015 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:25.398804 31862 raft_consensus.cc:3055] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:25.399202 31862 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:25.400288 31862 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.401983 31862 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.404971 32160 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:25.406435 31902 leader_election.cc:304] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:25.407455 32208 raft_consensus.cc:2804] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:25.407918 32208 raft_consensus.cc:697] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:25.408950 32208 consensus_queue.cc:237] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:25.411839 31862 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:25.416016 32160 raft_consensus.cc:3060] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:25.419440 31683 catalog_manager.cc:5697] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 1 to 2, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.421883 32160 pending_rounds.cc:85] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Aborting all ops after (but not including) 0
I20260709 13:10:25.422279 32160 pending_rounds.cc:107] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:10:25.422600 32160 raft_consensus.cc:2889] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:25.610666 31862 raft_consensus.cc:1275] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:25.611903 32208 consensus_queue.cc:1048] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:25.615039 32160 raft_consensus.cc:1275] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:25.616284 32208 consensus_queue.cc:1048] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:25.675621 31862 raft_consensus.cc:1275] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:25.677142 32215 consensus_queue.cc:1048] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:25.687484 32160 raft_consensus.cc:1275] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:25.689229 32109 consensus_queue.cc:1048] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:10:25.891748 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:25.995589 31862 raft_consensus.cc:1275] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:25.997215 32226 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:26.010175 32161 raft_consensus.cc:1275] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.011911 32208 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.034430 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:26.065487 31933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
mode: GRACEFUL
new_leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:46498
I20260709 13:10:26.066216 31933 raft_consensus.cc:606] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 LEADER]: Received request to transfer leadership to TS 142ed2f6af504e9583ab5c467be2b4d7
I20260709 13:10:26.069317 31934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62978121cded40708c2fe686051eafa4"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
mode: GRACEFUL
new_leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:46498
I20260709 13:10:26.069968 31934 raft_consensus.cc:606] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Received request to transfer leadership to TS 142ed2f6af504e9583ab5c467be2b4d7
I20260709 13:10:26.071259 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:26.071939 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:26.072721 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:26.247761 32208 raft_consensus.cc:993] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: : Instructing follower 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:26.248392 32215 raft_consensus.cc:1081] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 LEADER]: Signalling peer 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:26.250335 31862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:52712
I20260709 13:10:26.250871 31862 raft_consensus.cc:493] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.251201 31862 raft_consensus.cc:3060] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.256971 31862 raft_consensus.cc:515] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:26.259032 31862 leader_election.cc:290] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:26.260110 32161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:26.260453 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c0b90cccc2423f9160c57f32682bfc" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:26.260792 32161 raft_consensus.cc:3060] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.261076 31934 raft_consensus.cc:3055] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.261423 31934 raft_consensus.cc:740] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:26.262357 31934 consensus_queue.cc:260] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:26.263773 31934 raft_consensus.cc:3060] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.269060 32161 raft_consensus.cc:2468] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 2.
I20260709 13:10:26.270413 31828 leader_election.cc:304] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:26.271243 32108 raft_consensus.cc:2804] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.271709 32108 raft_consensus.cc:697] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:26.272811 31934 raft_consensus.cc:2468] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 2.
I20260709 13:10:26.272594 32108 consensus_queue.cc:237] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:26.285555 31684 catalog_manager.cc:5697] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 1 to 2, leader changed from c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.548959 32109 raft_consensus.cc:993] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: : Instructing follower 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:26.549484 32109 raft_consensus.cc:1081] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Signalling peer 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:26.551306 31862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62978121cded40708c2fe686051eafa4"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:52712
I20260709 13:10:26.551836 31862 raft_consensus.cc:493] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.552170 31862 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:26.557718 31862 raft_consensus.cc:515] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:26.559423 31862 leader_election.cc:290] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 3 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:26.560272 32161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:26.560345 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:26.561240 32161 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:26.561309 31934 raft_consensus.cc:3055] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:26.561846 31934 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:26.562665 31934 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:26.563660 31934 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:26.569810 31934 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 3.
I20260709 13:10:26.570822 31828 leader_election.cc:304] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:26.570633 32161 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 3.
I20260709 13:10:26.571631 32108 raft_consensus.cc:2804] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:26.572232 32108 raft_consensus.cc:697] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:26.573256 32108 consensus_queue.cc:237] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:26.582386 31684 catalog_manager.cc:5697] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 2 to 3, leader changed from c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.716173 32161 raft_consensus.cc:1275] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.717530 32108 consensus_queue.cc:1048] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.736342 31934 raft_consensus.cc:1275] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.738658 32108 consensus_queue.cc:1048] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:10:26.892858 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:27.083621 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:27.095371 31862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:27.095950 31862 raft_consensus.cc:606] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:27.098667 31862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ac731386824507bff9466dfed7f865"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:27.099208 31862 raft_consensus.cc:606] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:27.100262 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:27.100762 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:27.101389 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:27.160792 32160 raft_consensus.cc:1275] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:27.162323 32240 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:27.172983 31934 raft_consensus.cc:1275] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:27.175590 32108 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:27.514307 32240 raft_consensus.cc:993] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:27.514910 32108 raft_consensus.cc:1081] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:27.516798 31934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ac731386824507bff9466dfed7f865"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:27.517386 31934 raft_consensus.cc:493] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:27.517738 31934 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:27.524336 31934 raft_consensus.cc:515] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:27.526567 31934 leader_election.cc:290] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:27.527336 32160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:27.527661 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac731386824507bff9466dfed7f865" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:27.528126 32160 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:27.528317 31862 raft_consensus.cc:3055] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:27.528759 31862 raft_consensus.cc:740] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:27.529589 31862 consensus_queue.cc:260] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:27.530515 31862 raft_consensus.cc:3060] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:27.536460 31862 raft_consensus.cc:2468] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 3.
I20260709 13:10:27.536700 32160 raft_consensus.cc:2468] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 3.
I20260709 13:10:27.537806 31900 leader_election.cc:304] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:27.538754 32109 raft_consensus.cc:2804] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:27.540208 32109 raft_consensus.cc:697] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:27.541216 32109 consensus_queue.cc:237] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:27.550210 31682 catalog_manager.cc:5697] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 2 to 3, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.629266 32240 raft_consensus.cc:993] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:27.629740 32108 raft_consensus.cc:1081] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:27.631291 31934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:27.631786 31934 raft_consensus.cc:493] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:27.632092 31934 raft_consensus.cc:3060] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:27.637342 31934 raft_consensus.cc:515] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:27.639030 31934 leader_election.cc:290] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:27.639840 32160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:27.640151 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706b58c86e5a48ddac9daafa32041514" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:27.640547 32160 raft_consensus.cc:3060] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:27.640762 31862 raft_consensus.cc:3055] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:27.641230 31862 raft_consensus.cc:740] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:27.641963 31862 consensus_queue.cc:260] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:27.643174 31862 raft_consensus.cc:3060] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:27.648291 31862 raft_consensus.cc:2468] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:27.648558 32160 raft_consensus.cc:2468] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 2.
I20260709 13:10:27.649641 31900 leader_election.cc:304] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:27.650748 32109 raft_consensus.cc:2804] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:27.651247 32109 raft_consensus.cc:697] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:27.652231 32109 consensus_queue.cc:237] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:27.662637 31682 catalog_manager.cc:5697] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 1 to 2, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:27.893903 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:28.016111 31862 raft_consensus.cc:1275] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:28.017659 32109 consensus_queue.cc:1048] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:28.040580 32160 raft_consensus.cc:1275] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:28.042439 32245 consensus_queue.cc:1048] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:28.108381 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:28.121490 31934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
mode: GRACEFUL
new_leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:46498
I20260709 13:10:28.122246 31934 raft_consensus.cc:606] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Received request to transfer leadership to TS 142ed2f6af504e9583ab5c467be2b4d7
I20260709 13:10:28.125751 31934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
mode: GRACEFUL
new_leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:46498
I20260709 13:10:28.126408 31934 raft_consensus.cc:606] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Received request to transfer leadership to TS 142ed2f6af504e9583ab5c467be2b4d7
I20260709 13:10:28.127676 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:28.128358 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:28.129144 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:28.159575 32160 raft_consensus.cc:1275] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:28.161404 32109 consensus_queue.cc:1048] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:28.175247 31862 raft_consensus.cc:1275] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:28.180197 32109 consensus_queue.cc:1048] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:28.543905 32259 raft_consensus.cc:993] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: : Instructing follower 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:28.544428 32109 raft_consensus.cc:1081] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Signalling peer 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:28.546263 31861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:52712
I20260709 13:10:28.546818 31861 raft_consensus.cc:493] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:28.547142 31861 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:28.552479 31861 raft_consensus.cc:515] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:28.553978 31861 leader_election.cc:290] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 3 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:28.554912 32161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:28.555060 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b654b3d4ccb4ab9b4a5f4c58a14964a" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:28.555720 32161 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:28.555779 31934 raft_consensus.cc:3055] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:28.556216 31934 raft_consensus.cc:740] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:28.557106 31934 consensus_queue.cc:260] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:28.558386 31934 raft_consensus.cc:3060] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:28.563109 32161 raft_consensus.cc:2468] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 3.
I20260709 13:10:28.564450 31828 leader_election.cc:304] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:28.565200 32261 raft_consensus.cc:2804] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:28.565256 31934 raft_consensus.cc:2468] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 3.
I20260709 13:10:28.565801 32261 raft_consensus.cc:697] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:28.566762 32261 consensus_queue.cc:237] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:28.573154 31684 catalog_manager.cc:5697] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 2 to 3, leader changed from c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 3 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.686460 32259 raft_consensus.cc:993] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: : Instructing follower 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:28.686928 32109 raft_consensus.cc:1081] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Signalling peer 142ed2f6af504e9583ab5c467be2b4d7 to start an election
I20260709 13:10:28.688367 31861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
 from {username='slave'} at 127.0.0.1:52712
I20260709 13:10:28.688910 31861 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:28.689178 31861 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:28.695457 31861 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:28.696990 31861 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 4 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131:46117)
I20260709 13:10:28.697839 32161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:28.698545 32161 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:28.698253 31934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "142ed2f6af504e9583ab5c467be2b4d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
I20260709 13:10:28.699204 31934 raft_consensus.cc:3055] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:28.699486 31934 raft_consensus.cc:740] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:28.700206 31934 consensus_queue.cc:260] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:28.701313 31934 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:28.703996 32161 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 4.
I20260709 13:10:28.705122 31828 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, 35dabe2c8ebd4db5a12e2d57d589c650; no voters: 
I20260709 13:10:28.705927 32261 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:28.706382 32261 raft_consensus.cc:697] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 LEADER]: Becoming Leader. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:28.706687 31934 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142ed2f6af504e9583ab5c467be2b4d7 in term 4.
I20260709 13:10:28.707304 32261 consensus_queue.cc:237] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:28.714636 31684 catalog_manager.cc:5697] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 reported cstate change: term changed from 3 to 4, leader changed from c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131) to 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130). New cstate: current_term: 4 leader_uuid: "142ed2f6af504e9583ab5c467be2b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:28.894879 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:29.046375 31934 raft_consensus.cc:1275] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:29.047822 32108 consensus_queue.cc:1048] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:29.058655 32161 raft_consensus.cc:1275] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:29.060371 32108 consensus_queue.cc:1048] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:29.135654 32161 raft_consensus.cc:1275] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:29.137277 32108 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:29.140537 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:29.151146 31934 raft_consensus.cc:1275] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Refusing update from remote peer 142ed2f6af504e9583ab5c467be2b4d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:29.152817 32261 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:29.153592 31861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:29.154264 31861 raft_consensus.cc:606] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:29.157541 31861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62978121cded40708c2fe686051eafa4"
dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
mode: GRACEFUL
new_leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:57400
I20260709 13:10:29.158129 31861 raft_consensus.cc:606] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Received request to transfer leadership to TS c37cf16aa11340c79c452627e7d8c6f4
I20260709 13:10:29.159345 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:29.160171 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:29.161119 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:29.163053 32269 raft_consensus.cc:993] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:29.163688 32269 raft_consensus.cc:1081] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:29.165498 31934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:29.166290 31934 raft_consensus.cc:493] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:29.166726 31934 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:29.177248 31934 raft_consensus.cc:515] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:29.180168 31934 leader_election.cc:290] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 5 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:29.180867 31861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:29.180696 32161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa2541d650649f6969c9ac41ac82c83" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:29.181774 31861 raft_consensus.cc:3055] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:29.181941 32161 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:29.182143 31861 raft_consensus.cc:740] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:29.183173 31861 consensus_queue.cc:260] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:29.184698 31861 raft_consensus.cc:3060] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:29.190359 31861 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 5.
I20260709 13:10:29.190395 32161 raft_consensus.cc:2468] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 5.
I20260709 13:10:29.192026 31900 leader_election.cc:304] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142ed2f6af504e9583ab5c467be2b4d7, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:29.193148 32109 raft_consensus.cc:2804] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:29.194162 32109 raft_consensus.cc:697] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 5 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:29.195083 32109 consensus_queue.cc:237] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:29.202857 31682 catalog_manager.cc:5697] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 4 to 5, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 5 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.716969 32108 raft_consensus.cc:993] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: : Instructing follower c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:29.717610 32269 raft_consensus.cc:1081] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Signalling peer c37cf16aa11340c79c452627e7d8c6f4 to start an election
I20260709 13:10:29.719305 31934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62978121cded40708c2fe686051eafa4"
dest_uuid: "c37cf16aa11340c79c452627e7d8c6f4"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:10:29.719823 31934 raft_consensus.cc:493] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:29.720117 31934 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:29.726342 31934 raft_consensus.cc:515] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:29.728403 31934 leader_election.cc:290] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 4 election: Requested vote from peers 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397), 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599)
I20260709 13:10:29.729626 32161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "35dabe2c8ebd4db5a12e2d57d589c650"
I20260709 13:10:29.729535 31862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62978121cded40708c2fe686051eafa4" candidate_uuid: "c37cf16aa11340c79c452627e7d8c6f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "142ed2f6af504e9583ab5c467be2b4d7"
I20260709 13:10:29.730511 32161 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:29.730556 31862 raft_consensus.cc:3055] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:29.731173 31862 raft_consensus.cc:740] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 142ed2f6af504e9583ab5c467be2b4d7, State: Running, Role: LEADER
I20260709 13:10:29.731941 31862 consensus_queue.cc:260] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:29.733119 31862 raft_consensus.cc:3060] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:29.737074 32161 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 4.
I20260709 13:10:29.738531 31902 leader_election.cc:304] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dabe2c8ebd4db5a12e2d57d589c650, c37cf16aa11340c79c452627e7d8c6f4; no voters: 
I20260709 13:10:29.739255 31862 raft_consensus.cc:2468] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37cf16aa11340c79c452627e7d8c6f4 in term 4.
I20260709 13:10:29.739454 32276 raft_consensus.cc:2804] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:29.741432 32276 raft_consensus.cc:697] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 LEADER]: Becoming Leader. State: Replica: c37cf16aa11340c79c452627e7d8c6f4, State: Running, Role: LEADER
I20260709 13:10:29.742003 31862 raft_consensus.cc:1275] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 5 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:29.742429 32276 consensus_queue.cc:237] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } }
I20260709 13:10:29.744129 32277 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:29.754319 31682 catalog_manager.cc:5697] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 reported cstate change: term changed from 3 to 4, leader changed from 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130) to c37cf16aa11340c79c452627e7d8c6f4 (127.20.66.131). New cstate: current_term: 4 leader_uuid: "c37cf16aa11340c79c452627e7d8c6f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37cf16aa11340c79c452627e7d8c6f4" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 46117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.755697 32161 raft_consensus.cc:1275] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 5 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:29.758054 32109 consensus_queue.cc:1048] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:10:29.895996 31733 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:30.160823 32161 raft_consensus.cc:1275] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:30.162581 32276 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dabe2c8ebd4db5a12e2d57d589c650" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:30.172793 31862 raft_consensus.cc:1275] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Refusing update from remote peer c37cf16aa11340c79c452627e7d8c6f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:30.175331 32277 consensus_queue.cc:1048] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142ed2f6af504e9583ab5c467be2b4d7" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:30.176781 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:30.214773 20746 tablet_server.cc:179] TabletServer@127.20.66.129:44397 shutting down...
I20260709 13:10:30.236809 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:30.237805 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.238744 20746 raft_consensus.cc:2243] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.239564 20746 raft_consensus.cc:2272] T 62978121cded40708c2fe686051eafa4 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.243479 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.244174 20746 raft_consensus.cc:2243] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.244632 20746 raft_consensus.cc:2272] T 4214d17086964585905299fae79f0bca P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.246802 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.247416 20746 raft_consensus.cc:2243] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.247857 20746 raft_consensus.cc:2272] T 48121309c18d4fad8d28f098002b9d02 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.250412 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.250974 20746 raft_consensus.cc:2243] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.251381 20746 raft_consensus.cc:2272] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.253871 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.254449 20746 raft_consensus.cc:2243] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.254889 20746 raft_consensus.cc:2272] T 872ddbd361a64e06a386a4f1b6d8da8d P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.257234 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.257886 20746 raft_consensus.cc:2243] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.258392 20746 raft_consensus.cc:2272] T 706b58c86e5a48ddac9daafa32041514 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.260644 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.261348 20746 raft_consensus.cc:2243] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.261863 20746 raft_consensus.cc:2272] T 51c0b90cccc2423f9160c57f32682bfc P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.264312 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.265115 20746 raft_consensus.cc:2243] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.265883 20746 raft_consensus.cc:2272] T 832d464d4b3a43b0a1652b7c1e81e9aa P 35dabe2c8ebd4db5a12e2d57d589c650 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.268590 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.269326 20746 raft_consensus.cc:2243] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.269884 20746 raft_consensus.cc:2272] T 2aa2541d650649f6969c9ac41ac82c83 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.273384 20746 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:30.274030 20746 raft_consensus.cc:2243] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.274544 20746 raft_consensus.cc:2272] T 93ac731386824507bff9466dfed7f865 P 35dabe2c8ebd4db5a12e2d57d589c650 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.310994 20746 tablet_server.cc:196] TabletServer@127.20.66.129:44397 shutdown complete.
W20260709 13:10:30.332832 31902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260709 13:10:30.337316 31902 consensus_peers.cc:597] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.339735 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
I20260709 13:10:30.382380 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:30.383723 20746 tablet_replica.cc:333] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.385288 20746 raft_consensus.cc:2243] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.386224 20746 raft_consensus.cc:2272] T 93ac731386824507bff9466dfed7f865 P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.389823 20746 tablet_replica.cc:333] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.390735 20746 raft_consensus.cc:2243] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.391835 20746 raft_consensus.cc:2272] T 51c0b90cccc2423f9160c57f32682bfc P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.394665 20746 tablet_replica.cc:333] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.395423 20746 raft_consensus.cc:2243] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.395999 20746 raft_consensus.cc:2272] T 872ddbd361a64e06a386a4f1b6d8da8d P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.399019 20746 tablet_replica.cc:333] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.399739 20746 raft_consensus.cc:2243] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.400956 20746 raft_consensus.cc:2272] T 2aa2541d650649f6969c9ac41ac82c83 P 142ed2f6af504e9583ab5c467be2b4d7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.405280 20746 tablet_replica.cc:333] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.406035 20746 raft_consensus.cc:2243] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.407796 20746 raft_consensus.cc:2272] T 4214d17086964585905299fae79f0bca P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.410744 20746 tablet_replica.cc:333] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.411654 20746 raft_consensus.cc:2243] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.412206 20746 raft_consensus.cc:2272] T 62978121cded40708c2fe686051eafa4 P 142ed2f6af504e9583ab5c467be2b4d7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.416843 20746 tablet_replica.cc:333] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.417738 20746 raft_consensus.cc:2243] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.418884 20746 raft_consensus.cc:2272] T 48121309c18d4fad8d28f098002b9d02 P 142ed2f6af504e9583ab5c467be2b4d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.422904 20746 tablet_replica.cc:333] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.423724 20746 raft_consensus.cc:2243] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.425264 20746 raft_consensus.cc:2272] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P 142ed2f6af504e9583ab5c467be2b4d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.428018 20746 tablet_replica.cc:333] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.428743 20746 raft_consensus.cc:2243] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.429359 20746 raft_consensus.cc:2272] T 706b58c86e5a48ddac9daafa32041514 P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.431988 20746 tablet_replica.cc:333] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7: stopping tablet replica
I20260709 13:10:30.432631 20746 raft_consensus.cc:2243] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.433180 20746 raft_consensus.cc:2272] T 832d464d4b3a43b0a1652b7c1e81e9aa P 142ed2f6af504e9583ab5c467be2b4d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.491925 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
W20260709 13:10:30.512101 31900 consensus_peers.cc:597] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Couldn't send request to peer 142ed2f6af504e9583ab5c467be2b4d7. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:33599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.529691 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:10:30.539740 31900 consensus_peers.cc:597] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 142ed2f6af504e9583ab5c467be2b4d7 (127.20.66.130:33599): Couldn't send request to peer 142ed2f6af504e9583ab5c467be2b4d7. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:33599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:30.567258 31902 consensus_peers.cc:597] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.573552 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:30.574541 20746 tablet_replica.cc:333] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.575611 20746 raft_consensus.cc:2243] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.576292 20746 raft_consensus.cc:2272] T 51c0b90cccc2423f9160c57f32682bfc P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.579533 20746 tablet_replica.cc:333] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.580448 20746 raft_consensus.cc:2243] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.581565 20746 raft_consensus.cc:2272] T 706b58c86e5a48ddac9daafa32041514 P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.585033 20746 tablet_replica.cc:333] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.586002 20746 raft_consensus.cc:2243] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.586994 20746 raft_consensus.cc:2272] T 93ac731386824507bff9466dfed7f865 P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:30.588724 31902 consensus_peers.cc:597] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 -> Peer 35dabe2c8ebd4db5a12e2d57d589c650 (127.20.66.129:44397): Couldn't send request to peer 35dabe2c8ebd4db5a12e2d57d589c650. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:44397: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.590296 20746 tablet_replica.cc:333] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.591166 20746 raft_consensus.cc:2243] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.591792 20746 raft_consensus.cc:2272] T 48121309c18d4fad8d28f098002b9d02 P c37cf16aa11340c79c452627e7d8c6f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.594674 20746 tablet_replica.cc:333] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.595439 20746 raft_consensus.cc:2243] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.595952 20746 raft_consensus.cc:2272] T 5b654b3d4ccb4ab9b4a5f4c58a14964a P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.598624 20746 tablet_replica.cc:333] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.599273 20746 raft_consensus.cc:2243] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.600324 20746 raft_consensus.cc:2272] T 872ddbd361a64e06a386a4f1b6d8da8d P c37cf16aa11340c79c452627e7d8c6f4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.603020 20746 tablet_replica.cc:333] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.603690 20746 raft_consensus.cc:2243] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:30.604269 20746 raft_consensus.cc:2272] T 4214d17086964585905299fae79f0bca P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.607234 20746 tablet_replica.cc:333] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.607897 20746 raft_consensus.cc:2243] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.608891 20746 raft_consensus.cc:2272] T 62978121cded40708c2fe686051eafa4 P c37cf16aa11340c79c452627e7d8c6f4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.612017 20746 tablet_replica.cc:333] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.612673 20746 raft_consensus.cc:2243] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.614398 20746 raft_consensus.cc:2272] T 2aa2541d650649f6969c9ac41ac82c83 P c37cf16aa11340c79c452627e7d8c6f4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.618320 20746 tablet_replica.cc:333] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4: stopping tablet replica
I20260709 13:10:30.618974 20746 raft_consensus.cc:2243] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.620016 20746 raft_consensus.cc:2272] T 832d464d4b3a43b0a1652b7c1e81e9aa P c37cf16aa11340c79c452627e7d8c6f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.662107 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:10:30.698594 20746 master.cc:562] Master@127.20.66.190:41769 shutting down...
I20260709 13:10:30.731879 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:30.732671 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:30.733114 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P b7256215a205464eab3332b0c8659cfc: stopping tablet replica
I20260709 13:10:30.777351 20746 master.cc:584] Master@127.20.66.190:41769 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14094 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:10:30.827762 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:43777
I20260709 13:10:30.829234 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:30.835042 32292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:30.836611 32293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:30.836959 32295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:30.837735 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:30.838896 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:30.839082 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:30.839283 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602630839248 us; error 0 us; skew 500 ppm
I20260709 13:10:30.839980 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:30.842916 20746 webserver.cc:533] Webserver started at http://127.20.66.190:46779/ using document root <none> and password file <none>
I20260709 13:10:30.843608 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:30.843843 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:30.844113 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:30.845490 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "58557cc7c6864f528509d53d90ac5089"
format_stamp: "Formatted at 2026-07-09 13:10:30 on dist-test-slave-rgrd"
I20260709 13:10:30.850538 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:10:30.855147 32300 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:30.856074 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:10:30.856469 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "58557cc7c6864f528509d53d90ac5089"
format_stamp: "Formatted at 2026-07-09 13:10:30 on dist-test-slave-rgrd"
I20260709 13:10:30.856822 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:30.878129 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:30.879596 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:30.930447 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:43777
I20260709 13:10:30.930595 32351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:43777 every 8 connection(s)
I20260709 13:10:30.935806 32352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:30.954900 32352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089: Bootstrap starting.
I20260709 13:10:30.961959 32352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.968155 32352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089: No bootstrap required, opened a new log
I20260709 13:10:30.971191 32352 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58557cc7c6864f528509d53d90ac5089" member_type: VOTER }
I20260709 13:10:30.971724 32352 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.971932 32352 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58557cc7c6864f528509d53d90ac5089, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.972565 32352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58557cc7c6864f528509d53d90ac5089" member_type: VOTER }
I20260709 13:10:30.973176 32352 raft_consensus.cc:399] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:30.973507 32352 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:30.973771 32352 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.981312 32352 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58557cc7c6864f528509d53d90ac5089" member_type: VOTER }
I20260709 13:10:30.982111 32352 leader_election.cc:304] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58557cc7c6864f528509d53d90ac5089; no voters: 
I20260709 13:10:30.983788 32352 leader_election.cc:290] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:30.984256 32355 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:30.986014 32355 raft_consensus.cc:697] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 1 LEADER]: Becoming Leader. State: Replica: 58557cc7c6864f528509d53d90ac5089, State: Running, Role: LEADER
I20260709 13:10:30.987486 32352 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:30.987267 32355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58557cc7c6864f528509d53d90ac5089" member_type: VOTER }
I20260709 13:10:30.993789 32357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58557cc7c6864f528509d53d90ac5089. Latest consensus state: current_term: 1 leader_uuid: "58557cc7c6864f528509d53d90ac5089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58557cc7c6864f528509d53d90ac5089" member_type: VOTER } }
I20260709 13:10:30.994685 32357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:30.993732 32356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58557cc7c6864f528509d53d90ac5089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58557cc7c6864f528509d53d90ac5089" member_type: VOTER } }
I20260709 13:10:30.995397 32356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:30.999171 32364 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:31.005725 32364 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:31.008006 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:31.017046 32364 catalog_manager.cc:1383] Generated new cluster ID: 93155f7fd36649888b4c4c06d45d4e61
I20260709 13:10:31.017452 32364 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:31.034813 32364 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:31.036419 32364 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:31.054103 32364 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089: Generated new TSK 0
I20260709 13:10:31.054893 32364 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:31.076272 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:31.084250 32373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.085466 32374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.087347 32376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:31.087880 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:31.089288 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:31.089603 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:31.089808 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602631089788 us; error 0 us; skew 500 ppm
I20260709 13:10:31.090500 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:31.093532 20746 webserver.cc:533] Webserver started at http://127.20.66.129:39131/ using document root <none> and password file <none>
I20260709 13:10:31.094432 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:31.094672 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:31.095119 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:31.096758 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "8569613fa71949f9bb8a9e69fd074167"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-rgrd"
I20260709 13:10:31.102419 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:10:31.108305 32381 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.109881 20746 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:10:31.110422 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "8569613fa71949f9bb8a9e69fd074167"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-rgrd"
I20260709 13:10:31.110894 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:31.138465 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:31.140584 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:31.142555 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:31.145387 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:31.145670 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.145954 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:31.146136 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.190084 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:36429
I20260709 13:10:31.190233 32443 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:36429 every 8 connection(s)
I20260709 13:10:31.195232 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:31.204077 32448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.205907 32449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.206813 32451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:31.207274 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:31.208462 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:31.208653 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:31.208833 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602631208818 us; error 0 us; skew 500 ppm
I20260709 13:10:31.209460 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:31.211880 20746 webserver.cc:533] Webserver started at http://127.20.66.130:40855/ using document root <none> and password file <none>
I20260709 13:10:31.212458 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:31.212692 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:31.212972 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:31.214282 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "38612b8c96924add8ee726dd4976c7b4"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-rgrd"
I20260709 13:10:31.215993 32444 heartbeater.cc:344] Connected to a master server at 127.20.66.190:43777
I20260709 13:10:31.216440 32444 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:31.217265 32444 heartbeater.cc:507] Master 127.20.66.190:43777 requested a full tablet report, sending...
I20260709 13:10:31.220156 32317 ts_manager.cc:194] Registered new tserver with Master: 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429)
I20260709 13:10:31.220435 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:10:31.222733 32317 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59704
I20260709 13:10:31.225626 32456 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.226737 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:10:31.227006 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "38612b8c96924add8ee726dd4976c7b4"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-rgrd"
I20260709 13:10:31.227253 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:31.237980 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:31.239163 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:31.240684 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:31.243063 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:31.243240 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.243547 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:31.243718 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.286417 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:33683
I20260709 13:10:31.286604 32518 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:33683 every 8 connection(s)
I20260709 13:10:31.291684 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:31.299681 32523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.300277 32522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.303156 32525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:31.303476 32519 heartbeater.cc:344] Connected to a master server at 127.20.66.190:43777
I20260709 13:10:31.303891 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:31.304005 32519 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:31.305130 32519 heartbeater.cc:507] Master 127.20.66.190:43777 requested a full tablet report, sending...
I20260709 13:10:31.305135 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:31.305590 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:31.305810 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602631305792 us; error 0 us; skew 500 ppm
I20260709 13:10:31.306571 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:31.307778 32317 ts_manager.cc:194] Registered new tserver with Master: 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683)
I20260709 13:10:31.309387 20746 webserver.cc:533] Webserver started at http://127.20.66.131:34453/ using document root <none> and password file <none>
I20260709 13:10:31.309540 32317 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59710
I20260709 13:10:31.309993 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:31.310263 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:31.310603 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:31.312106 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-rgrd"
I20260709 13:10:31.316620 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:10:31.319998 32530 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.320815 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:31.321194 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-rgrd"
I20260709 13:10:31.321671 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:31.338291 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:31.339569 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:31.341169 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:31.343470 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:31.343708 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.343953 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:31.344146 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.388983 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:38003
I20260709 13:10:31.389181 32592 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:38003 every 8 connection(s)
I20260709 13:10:31.404604 32593 heartbeater.cc:344] Connected to a master server at 127.20.66.190:43777
I20260709 13:10:31.405052 32593 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:31.405864 32593 heartbeater.cc:507] Master 127.20.66.190:43777 requested a full tablet report, sending...
I20260709 13:10:31.408156 32317 ts_manager.cc:194] Registered new tserver with Master: 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:31.408834 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016112779s
I20260709 13:10:31.409744 32317 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59714
I20260709 13:10:31.436751 32317 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59720:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:31.439141 32317 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:31.484760 32484 tablet_service.cc:1511] Processing CreateTablet for tablet df8defd3184a45f8899cfa1fb6d53293 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=159a70eb2e5c413dade4d87d3298bc2c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:10:31.486294 32484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df8defd3184a45f8899cfa1fb6d53293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.490674 32483 tablet_service.cc:1511] Processing CreateTablet for tablet a2997cffb02e424da1a4fbe4b6d0d858 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=159a70eb2e5c413dade4d87d3298bc2c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:10:31.492131 32483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2997cffb02e424da1a4fbe4b6d0d858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.496379 32409 tablet_service.cc:1511] Processing CreateTablet for tablet df8defd3184a45f8899cfa1fb6d53293 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=159a70eb2e5c413dade4d87d3298bc2c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:10:31.497857 32409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df8defd3184a45f8899cfa1fb6d53293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.502889 32408 tablet_service.cc:1511] Processing CreateTablet for tablet a2997cffb02e424da1a4fbe4b6d0d858 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=159a70eb2e5c413dade4d87d3298bc2c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:10:31.504379 32408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2997cffb02e424da1a4fbe4b6d0d858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.510267 32558 tablet_service.cc:1511] Processing CreateTablet for tablet df8defd3184a45f8899cfa1fb6d53293 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=159a70eb2e5c413dade4d87d3298bc2c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:10:31.511688 32558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df8defd3184a45f8899cfa1fb6d53293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.514175 32557 tablet_service.cc:1511] Processing CreateTablet for tablet a2997cffb02e424da1a4fbe4b6d0d858 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=159a70eb2e5c413dade4d87d3298bc2c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:10:31.515583 32557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2997cffb02e424da1a4fbe4b6d0d858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.530484 32608 tablet_bootstrap.cc:492] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4: Bootstrap starting.
I20260709 13:10:31.538132 32608 tablet_bootstrap.cc:654] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.549631 32609 tablet_bootstrap.cc:492] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167: Bootstrap starting.
I20260709 13:10:31.551626 32608 tablet_bootstrap.cc:492] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4: No bootstrap required, opened a new log
I20260709 13:10:31.552367 32608 ts_tablet_manager.cc:1403] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.005s
I20260709 13:10:31.555882 32611 tablet_bootstrap.cc:492] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: Bootstrap starting.
I20260709 13:10:31.560020 32609 tablet_bootstrap.cc:654] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.559809 32608 raft_consensus.cc:359] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.560621 32608 raft_consensus.cc:385] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.560953 32608 raft_consensus.cc:740] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38612b8c96924add8ee726dd4976c7b4, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.561790 32608 consensus_queue.cc:260] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.564435 32611 tablet_bootstrap.cc:654] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.566195 32609 tablet_bootstrap.cc:492] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167: No bootstrap required, opened a new log
I20260709 13:10:31.566949 32609 ts_tablet_manager.cc:1403] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:10:31.569872 32611 tablet_bootstrap.cc:492] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: No bootstrap required, opened a new log
I20260709 13:10:31.570457 32611 ts_tablet_manager.cc:1403] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:10:31.570729 32608 ts_tablet_manager.cc:1434] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4: Time spent starting tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 13:10:31.571213 32519 heartbeater.cc:499] Master 127.20.66.190:43777 was elected leader, sending a full tablet report...
I20260709 13:10:31.572134 32608 tablet_bootstrap.cc:492] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4: Bootstrap starting.
I20260709 13:10:31.570706 32609 raft_consensus.cc:359] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.573868 32611 raft_consensus.cc:359] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.574864 32609 raft_consensus.cc:385] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.574872 32611 raft_consensus.cc:385] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.575556 32609 raft_consensus.cc:740] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.575556 32611 raft_consensus.cc:740] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.576845 32611 consensus_queue.cc:260] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.576812 32609 consensus_queue.cc:260] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.579319 32608 tablet_bootstrap.cc:654] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.588809 32611 ts_tablet_manager.cc:1434] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent starting tablet: real 0.018s	user 0.014s	sys 0.004s
I20260709 13:10:31.588860 32593 heartbeater.cc:499] Master 127.20.66.190:43777 was elected leader, sending a full tablet report...
I20260709 13:10:31.589926 32609 ts_tablet_manager.cc:1434] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 13:10:31.589154 32444 heartbeater.cc:499] Master 127.20.66.190:43777 was elected leader, sending a full tablet report...
I20260709 13:10:31.590442 32611 tablet_bootstrap.cc:492] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf: Bootstrap starting.
I20260709 13:10:31.592204 32609 tablet_bootstrap.cc:492] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167: Bootstrap starting.
I20260709 13:10:31.594444 32608 tablet_bootstrap.cc:492] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4: No bootstrap required, opened a new log
I20260709 13:10:31.595146 32608 ts_tablet_manager.cc:1403] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 13:10:31.598460 32608 raft_consensus.cc:359] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.599086 32611 tablet_bootstrap.cc:654] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.599328 32608 raft_consensus.cc:385] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.599888 32608 raft_consensus.cc:740] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38612b8c96924add8ee726dd4976c7b4, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.600725 32608 consensus_queue.cc:260] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.601765 32609 tablet_bootstrap.cc:654] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.603626 32608 ts_tablet_manager.cc:1434] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:10:31.612030 32611 tablet_bootstrap.cc:492] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf: No bootstrap required, opened a new log
I20260709 13:10:31.612794 32611 ts_tablet_manager.cc:1403] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.009s
I20260709 13:10:31.616386 32611 raft_consensus.cc:359] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.616981 32611 raft_consensus.cc:385] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.617339 32611 raft_consensus.cc:740] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.617863 32609 tablet_bootstrap.cc:492] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167: No bootstrap required, opened a new log
I20260709 13:10:31.618464 32609 ts_tablet_manager.cc:1403] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.007s
I20260709 13:10:31.617986 32611 consensus_queue.cc:260] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.619973 32611 ts_tablet_manager.cc:1434] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:10:31.621610 32609 raft_consensus.cc:359] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.622346 32609 raft_consensus.cc:385] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.622651 32609 raft_consensus.cc:740] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.623347 32609 consensus_queue.cc:260] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.625608 32609 ts_tablet_manager.cc:1434] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 13:10:31.703250 32615 raft_consensus.cc:493] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.703871 32615 raft_consensus.cc:515] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.706022 32615 leader_election.cc:290] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683), 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429)
I20260709 13:10:31.717783 32494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38612b8c96924add8ee726dd4976c7b4" is_pre_election: true
I20260709 13:10:31.718538 32494 raft_consensus.cc:2468] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 0.
I20260709 13:10:31.719866 32533 leader_election.cc:304] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 4e72baae1c784e2faf50b4bdbba1bebf; no voters: 
I20260709 13:10:31.719890 32419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569613fa71949f9bb8a9e69fd074167" is_pre_election: true
I20260709 13:10:31.721238 32419 raft_consensus.cc:2468] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 0.
I20260709 13:10:31.721268 32615 raft_consensus.cc:2804] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.721886 32615 raft_consensus.cc:493] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.722329 32615 raft_consensus.cc:3060] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.728883 32615 raft_consensus.cc:515] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.730836 32615 leader_election.cc:290] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 election: Requested vote from peers 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683), 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429)
I20260709 13:10:31.731824 32494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38612b8c96924add8ee726dd4976c7b4"
I20260709 13:10:31.732074 32419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
I20260709 13:10:31.732744 32494 raft_consensus.cc:3060] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.732755 32419 raft_consensus.cc:3060] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.741905 32494 raft_consensus.cc:2468] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 1.
I20260709 13:10:31.741905 32419 raft_consensus.cc:2468] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 1.
I20260709 13:10:31.743530 32533 leader_election.cc:304] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 4e72baae1c784e2faf50b4bdbba1bebf; no voters: 
I20260709 13:10:31.744451 32615 raft_consensus.cc:2804] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.745796 32615 raft_consensus.cc:697] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Becoming Leader. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Running, Role: LEADER
I20260709 13:10:31.746614 32615 consensus_queue.cc:237] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:31.754390 32317 catalog_manager.cc:5697] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131). New cstate: current_term: 1 leader_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.822860 32613 raft_consensus.cc:493] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.823524 32613 raft_consensus.cc:515] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.825856 32613 leader_election.cc:290] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429), 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:31.835733 32419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2997cffb02e424da1a4fbe4b6d0d858" candidate_uuid: "38612b8c96924add8ee726dd4976c7b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569613fa71949f9bb8a9e69fd074167" is_pre_election: true
I20260709 13:10:31.836076 32568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2997cffb02e424da1a4fbe4b6d0d858" candidate_uuid: "38612b8c96924add8ee726dd4976c7b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" is_pre_election: true
I20260709 13:10:31.836567 32419 raft_consensus.cc:2468] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38612b8c96924add8ee726dd4976c7b4 in term 0.
I20260709 13:10:31.836692 32568 raft_consensus.cc:2468] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38612b8c96924add8ee726dd4976c7b4 in term 0.
I20260709 13:10:31.837805 32459 leader_election.cc:304] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 8569613fa71949f9bb8a9e69fd074167; no voters: 
I20260709 13:10:31.838608 32613 raft_consensus.cc:2804] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.838973 32613 raft_consensus.cc:493] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.839246 32613 raft_consensus.cc:3060] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.844714 32613 raft_consensus.cc:515] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.846386 32613 leader_election.cc:290] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429), 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:31.847458 32419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2997cffb02e424da1a4fbe4b6d0d858" candidate_uuid: "38612b8c96924add8ee726dd4976c7b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
I20260709 13:10:31.847458 32568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2997cffb02e424da1a4fbe4b6d0d858" candidate_uuid: "38612b8c96924add8ee726dd4976c7b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
I20260709 13:10:31.848410 32419 raft_consensus.cc:3060] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.848438 32568 raft_consensus.cc:3060] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.857085 32419 raft_consensus.cc:2468] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38612b8c96924add8ee726dd4976c7b4 in term 1.
I20260709 13:10:31.857090 32568 raft_consensus.cc:2468] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38612b8c96924add8ee726dd4976c7b4 in term 1.
I20260709 13:10:31.858728 32457 leader_election.cc:304] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 4e72baae1c784e2faf50b4bdbba1bebf; no voters: 
I20260709 13:10:31.859527 32613 raft_consensus.cc:2804] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.860662 32613 raft_consensus.cc:697] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 1 LEADER]: Becoming Leader. State: Replica: 38612b8c96924add8ee726dd4976c7b4, State: Running, Role: LEADER
I20260709 13:10:31.861423 32613 consensus_queue.cc:237] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.868561 32317 catalog_manager.cc:5697] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "38612b8c96924add8ee726dd4976c7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.913811 32317 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38722:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:31.915835 32317 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:31.938905 32409 tablet_service.cc:1511] Processing CreateTablet for tablet edf3c98a157744068c6c8000be21f6f7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a438b7792962470bb456726836a99238]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:10:31.940248 32409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf3c98a157744068c6c8000be21f6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.940044 32483 tablet_service.cc:1511] Processing CreateTablet for tablet edf3c98a157744068c6c8000be21f6f7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a438b7792962470bb456726836a99238]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:10:31.941298 32483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf3c98a157744068c6c8000be21f6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.941443 32558 tablet_service.cc:1511] Processing CreateTablet for tablet edf3c98a157744068c6c8000be21f6f7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a438b7792962470bb456726836a99238]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:10:31.942663 32558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf3c98a157744068c6c8000be21f6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.946398 32484 tablet_service.cc:1511] Processing CreateTablet for tablet 45912923c9554d3680bf20605627e0e9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a438b7792962470bb456726836a99238]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:10:31.947243 32557 tablet_service.cc:1511] Processing CreateTablet for tablet 45912923c9554d3680bf20605627e0e9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a438b7792962470bb456726836a99238]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:10:31.947696 32484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45912923c9554d3680bf20605627e0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.948501 32557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45912923c9554d3680bf20605627e0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.950963 32408 tablet_service.cc:1511] Processing CreateTablet for tablet 45912923c9554d3680bf20605627e0e9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a438b7792962470bb456726836a99238]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:10:31.952232 32408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45912923c9554d3680bf20605627e0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.959097 32609 tablet_bootstrap.cc:492] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: Bootstrap starting.
I20260709 13:10:31.964068 32611 tablet_bootstrap.cc:492] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: Bootstrap starting.
I20260709 13:10:31.965693 32609 tablet_bootstrap.cc:654] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.971099 32611 tablet_bootstrap.cc:654] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.978579 32609 tablet_bootstrap.cc:492] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: No bootstrap required, opened a new log
I20260709 13:10:31.979135 32609 ts_tablet_manager.cc:1403] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 13:10:31.982228 32609 raft_consensus.cc:359] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.982954 32609 raft_consensus.cc:385] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.983227 32609 raft_consensus.cc:740] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.983872 32609 consensus_queue.cc:260] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.985126 32608 tablet_bootstrap.cc:492] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4: Bootstrap starting.
I20260709 13:10:31.987825 32611 tablet_bootstrap.cc:492] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: No bootstrap required, opened a new log
I20260709 13:10:31.988406 32611 ts_tablet_manager.cc:1403] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 13:10:31.988879 32609 ts_tablet_manager.cc:1434] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:31.990072 32609 tablet_bootstrap.cc:492] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167: Bootstrap starting.
I20260709 13:10:31.992327 32608 tablet_bootstrap.cc:654] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.991595 32611 raft_consensus.cc:359] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.993428 32611 raft_consensus.cc:385] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.993764 32611 raft_consensus.cc:740] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.994580 32611 consensus_queue.cc:260] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:31.998126 32611 ts_tablet_manager.cc:1434] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:10:31.999625 32611 tablet_bootstrap.cc:492] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf: Bootstrap starting.
W20260709 13:10:32.001576 32365 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:32.002494 32609 tablet_bootstrap.cc:654] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:32.007814 32611 tablet_bootstrap.cc:654] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:32.017278 32609 tablet_bootstrap.cc:492] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167: No bootstrap required, opened a new log
I20260709 13:10:32.017782 32609 ts_tablet_manager.cc:1403] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.001s
I20260709 13:10:32.018635 32608 tablet_bootstrap.cc:492] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4: No bootstrap required, opened a new log
I20260709 13:10:32.019229 32608 ts_tablet_manager.cc:1403] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.014s
I20260709 13:10:32.020193 32611 tablet_bootstrap.cc:492] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf: No bootstrap required, opened a new log
I20260709 13:10:32.020951 32611 ts_tablet_manager.cc:1403] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.009s
I20260709 13:10:32.020581 32609 raft_consensus.cc:359] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.021353 32609 raft_consensus.cc:385] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:32.021631 32609 raft_consensus.cc:740] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Initialized, Role: FOLLOWER
I20260709 13:10:32.022221 32608 raft_consensus.cc:359] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.022380 32609 consensus_queue.cc:260] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.023030 32608 raft_consensus.cc:385] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:32.023386 32608 raft_consensus.cc:740] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38612b8c96924add8ee726dd4976c7b4, State: Initialized, Role: FOLLOWER
I20260709 13:10:32.023962 32611 raft_consensus.cc:359] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.024256 32608 consensus_queue.cc:260] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.024698 32611 raft_consensus.cc:385] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:32.025197 32611 raft_consensus.cc:740] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Initialized, Role: FOLLOWER
I20260709 13:10:32.026573 32608 ts_tablet_manager.cc:1434] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:10:32.026167 32611 consensus_queue.cc:260] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.027678 32608 tablet_bootstrap.cc:492] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4: Bootstrap starting.
I20260709 13:10:32.028618 32611 ts_tablet_manager.cc:1434] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:10:32.024916 32609 ts_tablet_manager.cc:1434] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:32.034022 32608 tablet_bootstrap.cc:654] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:32.041453 32608 tablet_bootstrap.cc:492] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4: No bootstrap required, opened a new log
I20260709 13:10:32.041920 32608 ts_tablet_manager.cc:1403] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.005s
I20260709 13:10:32.044580 32608 raft_consensus.cc:359] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.045145 32608 raft_consensus.cc:385] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:32.045408 32608 raft_consensus.cc:740] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38612b8c96924add8ee726dd4976c7b4, State: Initialized, Role: FOLLOWER
I20260709 13:10:32.046015 32608 consensus_queue.cc:260] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.047555 32608 ts_tablet_manager.cc:1434] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:32.150238 32616 raft_consensus.cc:493] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.150817 32616 raft_consensus.cc:515] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.153024 32616 leader_election.cc:290] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683), 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:32.157486 32615 raft_consensus.cc:493] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.158100 32615 raft_consensus.cc:515] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.160566 32615 leader_election.cc:290] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429), 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683)
I20260709 13:10:32.161863 32419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569613fa71949f9bb8a9e69fd074167" is_pre_election: true
I20260709 13:10:32.161872 32494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38612b8c96924add8ee726dd4976c7b4" is_pre_election: true
I20260709 13:10:32.162922 32419 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 0.
I20260709 13:10:32.162987 32494 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 0.
I20260709 13:10:32.164261 32533 leader_election.cc:304] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 4e72baae1c784e2faf50b4bdbba1bebf; no voters: 
I20260709 13:10:32.165457 32615 raft_consensus.cc:2804] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:32.165840 32615 raft_consensus.cc:493] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.166231 32615 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.166389 32494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45912923c9554d3680bf20605627e0e9" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38612b8c96924add8ee726dd4976c7b4" is_pre_election: true
I20260709 13:10:32.166505 32568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45912923c9554d3680bf20605627e0e9" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" is_pre_election: true
I20260709 13:10:32.167272 32494 raft_consensus.cc:2468] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 0.
I20260709 13:10:32.167344 32568 raft_consensus.cc:2468] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 0.
I20260709 13:10:32.168304 32384 leader_election.cc:304] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 8569613fa71949f9bb8a9e69fd074167; no voters: 
I20260709 13:10:32.169044 32616 raft_consensus.cc:2804] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:32.169413 32616 raft_consensus.cc:493] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.169750 32616 raft_consensus.cc:3060] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.173431 32615 raft_consensus.cc:515] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.175149 32615 leader_election.cc:290] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 election: Requested vote from peers 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429), 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683)
I20260709 13:10:32.176020 32419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
I20260709 13:10:32.175637 32616 raft_consensus.cc:515] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.176409 32494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38612b8c96924add8ee726dd4976c7b4"
I20260709 13:10:32.176891 32419 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.177083 32494 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.179275 32568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45912923c9554d3680bf20605627e0e9" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
I20260709 13:10:32.179251 32493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45912923c9554d3680bf20605627e0e9" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38612b8c96924add8ee726dd4976c7b4"
I20260709 13:10:32.180084 32568 raft_consensus.cc:3060] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.180140 32493 raft_consensus.cc:3060] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.184114 32419 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 1.
I20260709 13:10:32.184114 32494 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 1.
I20260709 13:10:32.184949 32616 leader_election.cc:290] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 1 election: Requested vote from peers 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683), 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:32.185837 32533 leader_election.cc:304] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e72baae1c784e2faf50b4bdbba1bebf, 8569613fa71949f9bb8a9e69fd074167; no voters: 
I20260709 13:10:32.186683 32615 raft_consensus.cc:2804] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:32.187287 32615 raft_consensus.cc:697] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Becoming Leader. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Running, Role: LEADER
I20260709 13:10:32.188011 32568 raft_consensus.cc:2468] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 1.
I20260709 13:10:32.188016 32493 raft_consensus.cc:2468] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 1.
I20260709 13:10:32.188190 32615 consensus_queue.cc:237] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.189610 32384 leader_election.cc:304] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 8569613fa71949f9bb8a9e69fd074167; no voters: 
I20260709 13:10:32.190490 32616 raft_consensus.cc:2804] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:32.191540 32616 raft_consensus.cc:697] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 1 LEADER]: Becoming Leader. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Running, Role: LEADER
I20260709 13:10:32.194115 32616 consensus_queue.cc:237] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.196437 32316 catalog_manager.cc:5697] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131). New cstate: current_term: 1 leader_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:32.202839 32315 catalog_manager.cc:5697] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "8569613fa71949f9bb8a9e69fd074167" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:32.232810 32568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293"
dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
mode: GRACEFUL
new_leader_uuid: "8569613fa71949f9bb8a9e69fd074167"
 from {username='slave'} at 127.0.0.1:39228
I20260709 13:10:32.233482 32568 raft_consensus.cc:606] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Received request to transfer leadership to TS 8569613fa71949f9bb8a9e69fd074167
I20260709 13:10:32.238557 32615 consensus_queue.cc:1048] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.243788 32568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7"
dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
mode: GRACEFUL
new_leader_uuid: "8569613fa71949f9bb8a9e69fd074167"
 from {username='slave'} at 127.0.0.1:39228
I20260709 13:10:32.244406 32568 raft_consensus.cc:606] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Received request to transfer leadership to TS 8569613fa71949f9bb8a9e69fd074167
I20260709 13:10:32.246675 32621 raft_consensus.cc:993] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: : Instructing follower 8569613fa71949f9bb8a9e69fd074167 to start an election
I20260709 13:10:32.251278 32645 raft_consensus.cc:1081] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Signalling peer 8569613fa71949f9bb8a9e69fd074167 to start an election
I20260709 13:10:32.253933 32418 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293"
dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
 from {username='slave'} at 127.0.0.1:55284
I20260709 13:10:32.254619 32418 raft_consensus.cc:493] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:32.255012 32418 raft_consensus.cc:3060] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:32.259840 32645 consensus_queue.cc:1048] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.263896 32418 raft_consensus.cc:515] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:32.265273 32419 raft_consensus.cc:1240] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Rejecting Update request from peer 4e72baae1c784e2faf50b4bdbba1bebf for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:32.267418 32494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38612b8c96924add8ee726dd4976c7b4"
I20260709 13:10:32.270251 32418 leader_election.cc:290] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 2 election: Requested vote from peers 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683), 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:32.270675 32568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8defd3184a45f8899cfa1fb6d53293" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
I20260709 13:10:32.271239 32645 consensus_queue.cc:1059] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:32.274902 32568 raft_consensus.cc:3055] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:32.275282 32568 raft_consensus.cc:740] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Running, Role: LEADER
I20260709 13:10:32.276109 32568 consensus_queue.cc:260] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:32.277736 32568 raft_consensus.cc:3060] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:32.284466 32568 raft_consensus.cc:2468] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 2.
W20260709 13:10:32.284991 32621 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:10:32.285838 32382 leader_election.cc:304] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e72baae1c784e2faf50b4bdbba1bebf, 8569613fa71949f9bb8a9e69fd074167; no voters: 38612b8c96924add8ee726dd4976c7b4
I20260709 13:10:32.286936 32616 raft_consensus.cc:2804] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:32.287554 32616 raft_consensus.cc:697] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Becoming Leader. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Running, Role: LEADER
I20260709 13:10:32.288368 32616 consensus_queue.cc:237] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } }
I20260709 13:10:32.296977 32315 catalog_manager.cc:5697] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 reported cstate change: term changed from 1 to 2, leader changed from 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131) to 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "8569613fa71949f9bb8a9e69fd074167" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:32.436170 32613 consensus_queue.cc:1048] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.449386 32613 consensus_queue.cc:1048] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.592289 32616 consensus_queue.cc:1048] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:32.618029 32616 consensus_queue.cc:1048] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.699867 32621 consensus_queue.cc:1048] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.714797 32621 consensus_queue.cc:1048] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.724046 32615 raft_consensus.cc:993] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: : Instructing follower 8569613fa71949f9bb8a9e69fd074167 to start an election
I20260709 13:10:32.724560 32621 raft_consensus.cc:1081] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Signalling peer 8569613fa71949f9bb8a9e69fd074167 to start an election
I20260709 13:10:32.726465 32418 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7"
dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
 from {username='slave'} at 127.0.0.1:55284
I20260709 13:10:32.727156 32418 raft_consensus.cc:493] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:32.727488 32418 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:32.736094 32418 raft_consensus.cc:515] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.738245 32418 leader_election.cc:290] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 2 election: Requested vote from peers 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683), 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131:38003)
I20260709 13:10:32.739372 32493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38612b8c96924add8ee726dd4976c7b4"
I20260709 13:10:32.739508 32568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "8569613fa71949f9bb8a9e69fd074167" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
I20260709 13:10:32.740264 32493 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:32.740315 32568 raft_consensus.cc:3055] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:32.740780 32568 raft_consensus.cc:740] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Running, Role: LEADER
I20260709 13:10:32.741961 32568 consensus_queue.cc:260] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.743525 32568 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:32.748309 32493 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 2.
I20260709 13:10:32.750053 32384 leader_election.cc:304] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 8569613fa71949f9bb8a9e69fd074167; no voters: 
I20260709 13:10:32.750957 32616 raft_consensus.cc:2804] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:32.751266 32568 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8569613fa71949f9bb8a9e69fd074167 in term 2.
I20260709 13:10:32.751503 32616 raft_consensus.cc:697] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Becoming Leader. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Running, Role: LEADER
I20260709 13:10:32.753113 32493 raft_consensus.cc:3060] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:32.752841 32616 consensus_queue.cc:237] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:32.761340 32493 raft_consensus.cc:1275] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 2 FOLLOWER]: Refusing update from remote peer 8569613fa71949f9bb8a9e69fd074167: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:32.761386 32315 catalog_manager.cc:5697] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 reported cstate change: term changed from 1 to 2, leader changed from 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131) to 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "8569613fa71949f9bb8a9e69fd074167" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:32.763180 32616 consensus_queue.cc:1048] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:32.780551 32568 raft_consensus.cc:1275] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Refusing update from remote peer 8569613fa71949f9bb8a9e69fd074167: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:32.782248 32640 consensus_queue.cc:1048] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:32.787673 20746 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:10:32.790374 20746 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:10:32.791819 20746 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:10:32.812095 32418 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7"
dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
mode: GRACEFUL
new_leader_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
 from {username='slave'} at 127.0.0.1:55304
I20260709 13:10:32.812870 32418 raft_consensus.cc:606] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Received request to transfer leadership to TS 4e72baae1c784e2faf50b4bdbba1bebf
I20260709 13:10:32.814249 20746 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:10:33.176517 32493 raft_consensus.cc:1275] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 2 FOLLOWER]: Refusing update from remote peer 8569613fa71949f9bb8a9e69fd074167: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:33.177914 32616 consensus_queue.cc:1048] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:33.185870 32568 raft_consensus.cc:1275] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Refusing update from remote peer 8569613fa71949f9bb8a9e69fd074167: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:33.188405 32659 consensus_queue.cc:1048] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.196043 32640 raft_consensus.cc:993] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: : Instructing follower 4e72baae1c784e2faf50b4bdbba1bebf to start an election
I20260709 13:10:33.196575 32659 raft_consensus.cc:1081] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Signalling peer 4e72baae1c784e2faf50b4bdbba1bebf to start an election
I20260709 13:10:33.198400 32568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7"
dest_uuid: "4e72baae1c784e2faf50b4bdbba1bebf"
 from {username='slave'} at 127.0.0.1:39220
I20260709 13:10:33.199061 32568 raft_consensus.cc:493] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:33.199501 32568 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.205598 32568 raft_consensus.cc:515] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:33.207716 32568 leader_election.cc:290] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 3 election: Requested vote from peers 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129:36429), 38612b8c96924add8ee726dd4976c7b4 (127.20.66.130:33683)
I20260709 13:10:33.208794 32418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8569613fa71949f9bb8a9e69fd074167"
I20260709 13:10:33.209118 32493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf3c98a157744068c6c8000be21f6f7" candidate_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "38612b8c96924add8ee726dd4976c7b4"
I20260709 13:10:33.209605 32418 raft_consensus.cc:3055] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:33.209738 32493 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.210007 32418 raft_consensus.cc:740] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8569613fa71949f9bb8a9e69fd074167, State: Running, Role: LEADER
I20260709 13:10:33.211025 32418 consensus_queue.cc:260] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:33.212580 32418 raft_consensus.cc:3060] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.216353 32493 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 3.
I20260709 13:10:33.217839 32533 leader_election.cc:304] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38612b8c96924add8ee726dd4976c7b4, 4e72baae1c784e2faf50b4bdbba1bebf; no voters: 
I20260709 13:10:33.218828 32621 raft_consensus.cc:2804] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:33.219256 32621 raft_consensus.cc:697] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 3 LEADER]: Becoming Leader. State: Replica: 4e72baae1c784e2faf50b4bdbba1bebf, State: Running, Role: LEADER
I20260709 13:10:33.219615 32418 raft_consensus.cc:2468] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e72baae1c784e2faf50b4bdbba1bebf in term 3.
I20260709 13:10:33.220268 32621 consensus_queue.cc:237] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } }
I20260709 13:10:33.227890 32316 catalog_manager.cc:5697] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf reported cstate change: term changed from 2 to 3, leader changed from 8569613fa71949f9bb8a9e69fd074167 (127.20.66.129) to 4e72baae1c784e2faf50b4bdbba1bebf (127.20.66.131). New cstate: current_term: 3 leader_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569613fa71949f9bb8a9e69fd074167" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38612b8c96924add8ee726dd4976c7b4" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e72baae1c784e2faf50b4bdbba1bebf" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 38003 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:33.432196 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
I20260709 13:10:33.467293 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:33.468114 20746 tablet_replica.cc:333] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167: stopping tablet replica
I20260709 13:10:33.468937 20746 raft_consensus.cc:2243] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.469493 20746 raft_consensus.cc:2272] T edf3c98a157744068c6c8000be21f6f7 P 8569613fa71949f9bb8a9e69fd074167 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.472594 20746 tablet_replica.cc:333] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167: stopping tablet replica
I20260709 13:10:33.473232 20746 raft_consensus.cc:2243] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.473737 20746 raft_consensus.cc:2272] T a2997cffb02e424da1a4fbe4b6d0d858 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.476295 20746 tablet_replica.cc:333] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167: stopping tablet replica
I20260709 13:10:33.476845 20746 raft_consensus.cc:2243] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.477769 20746 raft_consensus.cc:2272] T 45912923c9554d3680bf20605627e0e9 P 8569613fa71949f9bb8a9e69fd074167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.480509 20746 tablet_replica.cc:333] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167: stopping tablet replica
I20260709 13:10:33.481132 20746 raft_consensus.cc:2243] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.481935 20746 raft_consensus.cc:2272] T df8defd3184a45f8899cfa1fb6d53293 P 8569613fa71949f9bb8a9e69fd074167 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.510061 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
I20260709 13:10:33.528658 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
I20260709 13:10:33.557572 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:33.558283 20746 tablet_replica.cc:333] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4: stopping tablet replica
I20260709 13:10:33.558986 20746 raft_consensus.cc:2243] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.559435 20746 raft_consensus.cc:2272] T edf3c98a157744068c6c8000be21f6f7 P 38612b8c96924add8ee726dd4976c7b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.562229 20746 tablet_replica.cc:333] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4: stopping tablet replica
I20260709 13:10:33.562760 20746 raft_consensus.cc:2243] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.563159 20746 raft_consensus.cc:2272] T 45912923c9554d3680bf20605627e0e9 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.565810 20746 tablet_replica.cc:333] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4: stopping tablet replica
I20260709 13:10:33.566322 20746 raft_consensus.cc:2243] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.566802 20746 raft_consensus.cc:2272] T df8defd3184a45f8899cfa1fb6d53293 P 38612b8c96924add8ee726dd4976c7b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.569393 20746 tablet_replica.cc:333] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4: stopping tablet replica
I20260709 13:10:33.569877 20746 raft_consensus.cc:2243] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.570698 20746 raft_consensus.cc:2272] T a2997cffb02e424da1a4fbe4b6d0d858 P 38612b8c96924add8ee726dd4976c7b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.585206 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
I20260709 13:10:33.603714 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
I20260709 13:10:33.627161 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:33.627866 20746 tablet_replica.cc:333] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: stopping tablet replica
I20260709 13:10:33.628504 20746 raft_consensus.cc:2243] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.629236 20746 pending_rounds.cc:70] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: Trying to abort 1 pending ops.
I20260709 13:10:33.629401 20746 pending_rounds.cc:77] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636385679622144 op_type: NO_OP noop_request { }
I20260709 13:10:33.629647 20746 raft_consensus.cc:2889] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:33.629889 20746 raft_consensus.cc:2272] T edf3c98a157744068c6c8000be21f6f7 P 4e72baae1c784e2faf50b4bdbba1bebf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.632475 20746 tablet_replica.cc:333] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf: stopping tablet replica
I20260709 13:10:33.633229 20746 raft_consensus.cc:2243] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.633731 20746 raft_consensus.cc:2272] T a2997cffb02e424da1a4fbe4b6d0d858 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.635898 20746 tablet_replica.cc:333] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf: stopping tablet replica
I20260709 13:10:33.636426 20746 raft_consensus.cc:2243] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.636806 20746 raft_consensus.cc:2272] T 45912923c9554d3680bf20605627e0e9 P 4e72baae1c784e2faf50b4bdbba1bebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.638907 20746 tablet_replica.cc:333] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf: stopping tablet replica
I20260709 13:10:33.639329 20746 raft_consensus.cc:2243] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.639719 20746 raft_consensus.cc:2272] T df8defd3184a45f8899cfa1fb6d53293 P 4e72baae1c784e2faf50b4bdbba1bebf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.665228 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:10:33.684491 20746 master.cc:562] Master@127.20.66.190:43777 shutting down...
I20260709 13:10:33.708545 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.709307 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.709735 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P 58557cc7c6864f528509d53d90ac5089: stopping tablet replica
I20260709 13:10:33.730203 20746 master.cc:584] Master@127.20.66.190:43777 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2943 ms)
[----------] 6 tests from LeaderRebalancerTest (375111 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:10:33.771327 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:40127
I20260709 13:10:33.772511 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:33.778292 32662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:33.778653 32663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:33.779943 32665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:33.780350 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:33.781508 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:33.781721 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:33.781837 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602633781825 us; error 0 us; skew 500 ppm
I20260709 13:10:33.782316 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:33.784819 20746 webserver.cc:533] Webserver started at http://127.20.66.190:45383/ using document root <none> and password file <none>
I20260709 13:10:33.785496 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:33.785671 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:33.785888 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:33.787071 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "14e6a3ba75c4405e80e3a8840d54dd52"
format_stamp: "Formatted at 2026-07-09 13:10:33 on dist-test-slave-rgrd"
I20260709 13:10:33.791616 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:10:33.795385 32670 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:33.796240 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:33.796564 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "14e6a3ba75c4405e80e3a8840d54dd52"
format_stamp: "Formatted at 2026-07-09 13:10:33 on dist-test-slave-rgrd"
I20260709 13:10:33.796866 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:33.811021 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:33.812282 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:33.851974 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:40127
I20260709 13:10:33.852082 32721 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:40127 every 8 connection(s)
I20260709 13:10:33.855942 32722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:33.867906 32722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52: Bootstrap starting.
I20260709 13:10:33.872918 32722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.877513 32722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52: No bootstrap required, opened a new log
I20260709 13:10:33.879715 32722 raft_consensus.cc:359] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" member_type: VOTER }
I20260709 13:10:33.880105 32722 raft_consensus.cc:385] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.880374 32722 raft_consensus.cc:740] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14e6a3ba75c4405e80e3a8840d54dd52, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.880952 32722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" member_type: VOTER }
I20260709 13:10:33.881398 32722 raft_consensus.cc:399] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:33.881601 32722 raft_consensus.cc:493] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:33.881806 32722 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.886580 32722 raft_consensus.cc:515] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" member_type: VOTER }
I20260709 13:10:33.887038 32722 leader_election.cc:304] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14e6a3ba75c4405e80e3a8840d54dd52; no voters: 
I20260709 13:10:33.888159 32722 leader_election.cc:290] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:33.888645 32725 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.890317 32725 raft_consensus.cc:697] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 1 LEADER]: Becoming Leader. State: Replica: 14e6a3ba75c4405e80e3a8840d54dd52, State: Running, Role: LEADER
I20260709 13:10:33.891057 32722 sys_catalog.cc:565] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:33.891047 32725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" member_type: VOTER }
I20260709 13:10:33.894937 32727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e6a3ba75c4405e80e3a8840d54dd52. Latest consensus state: current_term: 1 leader_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" member_type: VOTER } }
I20260709 13:10:33.895704 32727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:33.895364 32726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e6a3ba75c4405e80e3a8840d54dd52" member_type: VOTER } }
I20260709 13:10:33.896436 32726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:33.900498 32732 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:33.906742 32732 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:33.908094 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:33.916102 32732 catalog_manager.cc:1383] Generated new cluster ID: 2766ef5400484f59bcaf36e4f580899a
I20260709 13:10:33.916417 32732 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:33.927418 32732 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:33.928784 32732 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:33.949203 32732 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52: Generated new TSK 0
I20260709 13:10:33.949887 32732 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:33.975251 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:33.981478 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:33.982466 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:33.985512 20746 server_base.cc:1061] running on GCE node
W20260709 13:10:33.985466 32746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:33.986737 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:33.986917 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:33.987040 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602633987029 us; error 0 us; skew 500 ppm
I20260709 13:10:33.987509 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:33.989925 20746 webserver.cc:533] Webserver started at http://127.20.66.129:43575/ using document root <none> and password file <none>
I20260709 13:10:33.990365 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:33.990526 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:33.990773 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:33.991852 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "5592f0c3492e4e11b54a6f21387bef95"
format_stamp: "Formatted at 2026-07-09 13:10:33 on dist-test-slave-rgrd"
I20260709 13:10:33.996603 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:34.000270 32751 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.001163 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:34.001504 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "5592f0c3492e4e11b54a6f21387bef95"
format_stamp: "Formatted at 2026-07-09 13:10:33 on dist-test-slave-rgrd"
I20260709 13:10:34.001804 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:34.026283 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:34.027631 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:34.029166 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:34.031486 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:34.031687 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.031955 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:34.032109 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.073377 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:40573
I20260709 13:10:34.073493   345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:40573 every 8 connection(s)
I20260709 13:10:34.078507 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:34.087342   350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:34.088249   351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:34.091113   353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:34.094262 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:34.094672   346 heartbeater.cc:344] Connected to a master server at 127.20.66.190:40127
I20260709 13:10:34.095108   346 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:34.095908   346 heartbeater.cc:507] Master 127.20.66.190:40127 requested a full tablet report, sending...
I20260709 13:10:34.096136 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:34.096441 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:34.096627 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602634096607 us; error 0 us; skew 500 ppm
I20260709 13:10:34.097308 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:34.098177 32687 ts_manager.cc:194] Registered new tserver with Master: 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573)
I20260709 13:10:34.100169 20746 webserver.cc:533] Webserver started at http://127.20.66.130:45965/ using document root <none> and password file <none>
I20260709 13:10:34.100615 32687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48234
I20260709 13:10:34.100781 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:34.101127 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:34.101423 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:34.102809 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "5f9b867a960045008f271fe1fc3a447d"
format_stamp: "Formatted at 2026-07-09 13:10:34 on dist-test-slave-rgrd"
I20260709 13:10:34.107506 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:10:34.110975   358 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.111795 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:34.112098 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "5f9b867a960045008f271fe1fc3a447d"
format_stamp: "Formatted at 2026-07-09 13:10:34 on dist-test-slave-rgrd"
I20260709 13:10:34.112394 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:34.137499 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:34.138796 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:34.140280 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:34.142551 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:34.142786 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.143028 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:34.143237 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.183240 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:41289
I20260709 13:10:34.183352   420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:41289 every 8 connection(s)
I20260709 13:10:34.188141 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:34.195719   424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:34.195938   425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:34.199545   421 heartbeater.cc:344] Connected to a master server at 127.20.66.190:40127
I20260709 13:10:34.200085   421 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:34.201190   421 heartbeater.cc:507] Master 127.20.66.190:40127 requested a full tablet report, sending...
I20260709 13:10:34.201478 20746 server_base.cc:1061] running on GCE node
W20260709 13:10:34.201444   427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:34.202939 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:34.203291 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:34.203500 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602634203461 us; error 0 us; skew 500 ppm
I20260709 13:10:34.203646 32687 ts_manager.cc:194] Registered new tserver with Master: 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:34.204159 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:34.205291 32687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48238
I20260709 13:10:34.207659 20746 webserver.cc:533] Webserver started at http://127.20.66.131:37547/ using document root <none> and password file <none>
I20260709 13:10:34.208307 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:34.208535 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:34.208832 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:34.210095 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "d0e9408b380e414dbe5daf286b6a0a41"
format_stamp: "Formatted at 2026-07-09 13:10:34 on dist-test-slave-rgrd"
I20260709 13:10:34.215235 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:10:34.218912   432 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.219795 20746 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:10:34.220145 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "d0e9408b380e414dbe5daf286b6a0a41"
format_stamp: "Formatted at 2026-07-09 13:10:34 on dist-test-slave-rgrd"
I20260709 13:10:34.220420 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:34.255131 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:34.256340 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:34.257874 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:34.260226 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:34.260406 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.260703 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:34.260875 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.302763 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:42073
I20260709 13:10:34.302944   495 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:42073 every 8 connection(s)
I20260709 13:10:34.317806   496 heartbeater.cc:344] Connected to a master server at 127.20.66.190:40127
I20260709 13:10:34.318267   496 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:34.318984   496 heartbeater.cc:507] Master 127.20.66.190:40127 requested a full tablet report, sending...
I20260709 13:10:34.320895 32687 ts_manager.cc:194] Registered new tserver with Master: d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:34.321385 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014889936s
I20260709 13:10:34.322572 32687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48244
I20260709 13:10:34.348027 32687 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48260:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:34.350927 32687 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:34.417547   311 tablet_service.cc:1511] Processing CreateTablet for tablet f514b957834b4abab3de4af238532008 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:34.419138   311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f514b957834b4abab3de4af238532008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.421949   310 tablet_service.cc:1511] Processing CreateTablet for tablet da123722312340b088054d80b503c16f (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:34.423759   310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da123722312340b088054d80b503c16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.430423   309 tablet_service.cc:1511] Processing CreateTablet for tablet 4e124b2bdf5a4807a77b0b9a4df0c3a0 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:34.432202   309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e124b2bdf5a4807a77b0b9a4df0c3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.439072   308 tablet_service.cc:1511] Processing CreateTablet for tablet f28674dd0a884252b8d11396c9d8c3e1 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:34.440742   308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28674dd0a884252b8d11396c9d8c3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.443316   461 tablet_service.cc:1511] Processing CreateTablet for tablet f514b957834b4abab3de4af238532008 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:34.445247   307 tablet_service.cc:1511] Processing CreateTablet for tablet 9da00b8b1f394254bb3e4070096361c7 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:34.446327   460 tablet_service.cc:1511] Processing CreateTablet for tablet da123722312340b088054d80b503c16f (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:34.446964   307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da00b8b1f394254bb3e4070096361c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.448894   459 tablet_service.cc:1511] Processing CreateTablet for tablet 4e124b2bdf5a4807a77b0b9a4df0c3a0 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:34.452785   458 tablet_service.cc:1511] Processing CreateTablet for tablet f28674dd0a884252b8d11396c9d8c3e1 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:34.456130   457 tablet_service.cc:1511] Processing CreateTablet for tablet 9da00b8b1f394254bb3e4070096361c7 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:34.444921   461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f514b957834b4abab3de4af238532008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.458905   457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da00b8b1f394254bb3e4070096361c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.461844   460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da123722312340b088054d80b503c16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.463023   306 tablet_service.cc:1511] Processing CreateTablet for tablet 9b8bb5f5852b4bd89fa8abeb60d6ad8e (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:34.464413   306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8bb5f5852b4bd89fa8abeb60d6ad8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.465382   458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28674dd0a884252b8d11396c9d8c3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.470343   459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e124b2bdf5a4807a77b0b9a4df0c3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.470788   456 tablet_service.cc:1511] Processing CreateTablet for tablet e242b10da2df46778fbbd4910a97808f (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:34.471110   455 tablet_service.cc:1511] Processing CreateTablet for tablet 9b8bb5f5852b4bd89fa8abeb60d6ad8e (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:34.471822   305 tablet_service.cc:1511] Processing CreateTablet for tablet 70f28fd9f6fa4c4799b20e357dc87813 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:34.473243   305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70f28fd9f6fa4c4799b20e357dc87813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.472260   456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e242b10da2df46778fbbd4910a97808f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.477701   454 tablet_service.cc:1511] Processing CreateTablet for tablet 70f28fd9f6fa4c4799b20e357dc87813 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:34.479111   454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70f28fd9f6fa4c4799b20e357dc87813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.480531   455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8bb5f5852b4bd89fa8abeb60d6ad8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.487102   304 tablet_service.cc:1511] Processing CreateTablet for tablet 49cf29e468ff44aba1ab2ab8981ad02a (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:34.488615   304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cf29e468ff44aba1ab2ab8981ad02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.498131   453 tablet_service.cc:1511] Processing CreateTablet for tablet 49cf29e468ff44aba1ab2ab8981ad02a (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:34.499784   453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cf29e468ff44aba1ab2ab8981ad02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.516228   310 tablet_service.cc:1511] Processing CreateTablet for tablet e242b10da2df46778fbbd4910a97808f (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:34.537972   310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e242b10da2df46778fbbd4910a97808f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.539973   386 tablet_service.cc:1511] Processing CreateTablet for tablet f514b957834b4abab3de4af238532008 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:34.542253   385 tablet_service.cc:1511] Processing CreateTablet for tablet da123722312340b088054d80b503c16f (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:34.543702   385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da123722312340b088054d80b503c16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.547048   381 tablet_service.cc:1511] Processing CreateTablet for tablet e242b10da2df46778fbbd4910a97808f (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:34.548519   381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e242b10da2df46778fbbd4910a97808f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.550011   379 tablet_service.cc:1511] Processing CreateTablet for tablet 70f28fd9f6fa4c4799b20e357dc87813 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:34.551354   379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70f28fd9f6fa4c4799b20e357dc87813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.554121   380 tablet_service.cc:1511] Processing CreateTablet for tablet 9b8bb5f5852b4bd89fa8abeb60d6ad8e (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:34.556833   384 tablet_service.cc:1511] Processing CreateTablet for tablet 4e124b2bdf5a4807a77b0b9a4df0c3a0 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:34.558809   378 tablet_service.cc:1511] Processing CreateTablet for tablet 49cf29e468ff44aba1ab2ab8981ad02a (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:34.560613   383 tablet_service.cc:1511] Processing CreateTablet for tablet f28674dd0a884252b8d11396c9d8c3e1 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:34.564111   382 tablet_service.cc:1511] Processing CreateTablet for tablet 9da00b8b1f394254bb3e4070096361c7 (DEFAULT_TABLE table=test-workload [id=2750056265994e48a3c811eb09ed184a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:34.567104   386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f514b957834b4abab3de4af238532008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.569777   380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8bb5f5852b4bd89fa8abeb60d6ad8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.576376   384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e124b2bdf5a4807a77b0b9a4df0c3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.578601   383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28674dd0a884252b8d11396c9d8c3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.581393   382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da00b8b1f394254bb3e4070096361c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.584012   378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cf29e468ff44aba1ab2ab8981ad02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.606475   511 tablet_bootstrap.cc:492] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:34.610175   512 tablet_bootstrap.cc:492] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.648313   511 tablet_bootstrap.cc:654] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.650183   513 tablet_bootstrap.cc:492] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:34.652454   512 tablet_bootstrap.cc:654] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.657912   512 tablet_bootstrap.cc:492] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:34.658411   512 ts_tablet_manager.cc:1403] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.049s	user 0.001s	sys 0.011s
I20260709 13:10:34.661739   512 raft_consensus.cc:359] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.662762   512 raft_consensus.cc:385] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.663146   512 raft_consensus.cc:740] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.663725   513 tablet_bootstrap.cc:654] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.664371   512 consensus_queue.cc:260] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.673205   511 tablet_bootstrap.cc:492] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:34.674042   511 ts_tablet_manager.cc:1403] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.012s
I20260709 13:10:34.676613   513 tablet_bootstrap.cc:492] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:34.677182   512 ts_tablet_manager.cc:1434] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.009s
I20260709 13:10:34.677224   513 ts_tablet_manager.cc:1403] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.006s
I20260709 13:10:34.677413   496 heartbeater.cc:499] Master 127.20.66.190:40127 was elected leader, sending a full tablet report...
I20260709 13:10:34.677878   511 raft_consensus.cc:359] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.678519   512 tablet_bootstrap.cc:492] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.678637   511 raft_consensus.cc:385] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.679100   511 raft_consensus.cc:740] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.680037   511 consensus_queue.cc:260] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.680769   513 raft_consensus.cc:359] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.681577   513 raft_consensus.cc:385] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.681856   513 raft_consensus.cc:740] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.682718   513 consensus_queue.cc:260] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.686980   512 tablet_bootstrap.cc:654] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.691311   421 heartbeater.cc:499] Master 127.20.66.190:40127 was elected leader, sending a full tablet report...
I20260709 13:10:34.692314   513 ts_tablet_manager.cc:1434] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.003s
I20260709 13:10:34.693511   513 tablet_bootstrap.cc:492] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:34.700501   513 tablet_bootstrap.cc:654] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.709121   346 heartbeater.cc:499] Master 127.20.66.190:40127 was elected leader, sending a full tablet report...
I20260709 13:10:34.712361   513 tablet_bootstrap.cc:492] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:34.713100   513 ts_tablet_manager.cc:1403] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 13:10:34.716243   513 raft_consensus.cc:359] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.714609   511 ts_tablet_manager.cc:1434] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.009s
I20260709 13:10:34.717058   513 raft_consensus.cc:385] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.717536   513 raft_consensus.cc:740] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.718490   513 consensus_queue.cc:260] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.718405   511 tablet_bootstrap.cc:492] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:34.722478   512 tablet_bootstrap.cc:492] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:34.722596   513 ts_tablet_manager.cc:1434] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:34.723136   512 ts_tablet_manager.cc:1403] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.023s
I20260709 13:10:34.723961   513 tablet_bootstrap.cc:492] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:34.726572   512 raft_consensus.cc:359] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.726960   511 tablet_bootstrap.cc:654] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.727303   512 raft_consensus.cc:385] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.727694   512 raft_consensus.cc:740] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.728633   512 consensus_queue.cc:260] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.732012   513 tablet_bootstrap.cc:654] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.733475   512 ts_tablet_manager.cc:1434] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:34.734592   512 tablet_bootstrap.cc:492] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.741271   512 tablet_bootstrap.cc:654] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.745924   513 tablet_bootstrap.cc:492] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:34.746838   513 ts_tablet_manager.cc:1403] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.008s
I20260709 13:10:34.751360   513 raft_consensus.cc:359] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.752595   513 raft_consensus.cc:385] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.753291   512 tablet_bootstrap.cc:492] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:34.753106   513 raft_consensus.cc:740] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.753957   512 ts_tablet_manager.cc:1403] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.009s
I20260709 13:10:34.754635   513 consensus_queue.cc:260] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.757243   512 raft_consensus.cc:359] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.757889   513 ts_tablet_manager.cc:1434] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 13:10:34.758018   512 raft_consensus.cc:385] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.758613   512 raft_consensus.cc:740] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.759346   513 tablet_bootstrap.cc:492] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:34.759415   512 consensus_queue.cc:260] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.762295   512 ts_tablet_manager.cc:1434] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:10:34.763279   512 tablet_bootstrap.cc:492] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.769886   512 tablet_bootstrap.cc:654] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.770394   513 tablet_bootstrap.cc:654] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.781793   512 tablet_bootstrap.cc:492] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:34.782437   512 ts_tablet_manager.cc:1403] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.003s
I20260709 13:10:34.785483   512 raft_consensus.cc:359] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.786188   512 raft_consensus.cc:385] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.786448   512 raft_consensus.cc:740] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.786657   513 tablet_bootstrap.cc:492] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:34.787130   512 consensus_queue.cc:260] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.787487   513 ts_tablet_manager.cc:1403] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.008s
I20260709 13:10:34.788460   518 raft_consensus.cc:493] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.788957   518 raft_consensus.cc:515] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.790424   512 ts_tablet_manager.cc:1434] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:34.790817   511 tablet_bootstrap.cc:492] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:34.791481   512 tablet_bootstrap.cc:492] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.791484   511 ts_tablet_manager.cc:1403] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.073s	user 0.033s	sys 0.033s
I20260709 13:10:34.795315   511 raft_consensus.cc:359] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.796222   511 raft_consensus.cc:385] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.796648   511 raft_consensus.cc:740] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.798700   511 consensus_queue.cc:260] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.799773   512 tablet_bootstrap.cc:654] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.801741   518 leader_election.cc:290] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:34.811411   512 tablet_bootstrap.cc:492] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:34.811949   512 ts_tablet_manager.cc:1403] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 13:10:34.814226   512 raft_consensus.cc:359] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.814896   512 raft_consensus.cc:385] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.815230   512 raft_consensus.cc:740] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.815430   511 ts_tablet_manager.cc:1434] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.002s
I20260709 13:10:34.815924   512 consensus_queue.cc:260] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.816834   511 tablet_bootstrap.cc:492] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:34.819823   513 raft_consensus.cc:359] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.821455   512 ts_tablet_manager.cc:1434] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:10:34.823004   512 tablet_bootstrap.cc:492] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.824326   513 raft_consensus.cc:385] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.824882   513 raft_consensus.cc:740] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.825596   396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514b957834b4abab3de4af238532008" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:34.826396   396 raft_consensus.cc:2468] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:34.828002 32754 leader_election.cc:304] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:34.829607   529 raft_consensus.cc:2804] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:34.829981   512 tablet_bootstrap.cc:654] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.830117   529 raft_consensus.cc:493] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:34.830682   529 raft_consensus.cc:3060] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:34.826740   513 consensus_queue.cc:260] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.837546   513 ts_tablet_manager.cc:1434] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.006s
I20260709 13:10:34.840361   511 tablet_bootstrap.cc:654] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.841682   513 tablet_bootstrap.cc:492] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:34.844146   529 raft_consensus.cc:515] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.848459   396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514b957834b4abab3de4af238532008" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:34.849383   396 raft_consensus.cc:3060] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:34.856840   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514b957834b4abab3de4af238532008" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
W20260709 13:10:34.860013 32752 leader_election.cc:343] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.866194   396 raft_consensus.cc:2468] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:34.867383   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514b957834b4abab3de4af238532008" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:34.868794   512 tablet_bootstrap.cc:492] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
W20260709 13:10:34.869067 32752 leader_election.cc:343] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.869135   513 tablet_bootstrap.cc:654] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.869549   529 leader_election.cc:290] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:34.869366   512 ts_tablet_manager.cc:1403] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.047s	user 0.024s	sys 0.016s
I20260709 13:10:34.870321 32754 leader_election.cc:304] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:34.873178   529 raft_consensus.cc:2804] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:34.874433   529 raft_consensus.cc:697] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:34.875258   529 consensus_queue.cc:237] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.877157   512 raft_consensus.cc:359] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.878008   512 raft_consensus.cc:385] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.879707   512 raft_consensus.cc:740] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.881472   511 tablet_bootstrap.cc:492] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:34.880905   512 consensus_queue.cc:260] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.881855   513 tablet_bootstrap.cc:492] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:34.882210   511 ts_tablet_manager.cc:1403] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.066s	user 0.033s	sys 0.015s
I20260709 13:10:34.882563   513 ts_tablet_manager.cc:1403] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.011s
I20260709 13:10:34.883731   512 ts_tablet_manager.cc:1434] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:10:34.884850   512 tablet_bootstrap.cc:492] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.885674   511 raft_consensus.cc:359] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.885999   513 raft_consensus.cc:359] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.886596   511 raft_consensus.cc:385] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.886879   513 raft_consensus.cc:385] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.886982   511 raft_consensus.cc:740] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.887389   513 raft_consensus.cc:740] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.888113   511 consensus_queue.cc:260] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.890349   516 raft_consensus.cc:493] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.890980   511 ts_tablet_manager.cc:1434] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:34.888394   513 consensus_queue.cc:260] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.890981   516 raft_consensus.cc:515] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.892236   511 tablet_bootstrap.cc:492] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:34.893863   512 tablet_bootstrap.cc:654] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.896940   513 ts_tablet_manager.cc:1434] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:10:34.899418   513 tablet_bootstrap.cc:492] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:34.900511   519 raft_consensus.cc:493] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.901290   511 tablet_bootstrap.cc:654] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.901115   519 raft_consensus.cc:515] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
W20260709 13:10:34.908515 32735 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:34.910173   516 leader_election.cc:290] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:34.911023   516 raft_consensus.cc:493] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.912775   519 leader_election.cc:290] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:34.929901   519 raft_consensus.cc:493] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.930853   519 raft_consensus.cc:515] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.934223   519 leader_election.cc:290] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:34.938265   512 tablet_bootstrap.cc:492] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:34.940109   512 ts_tablet_manager.cc:1403] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.015s
I20260709 13:10:34.913420   516 raft_consensus.cc:515] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.943303   512 raft_consensus.cc:359] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.943969   535 raft_consensus.cc:493] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.944202   512 raft_consensus.cc:385] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.944650   535 raft_consensus.cc:515] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.945058   512 raft_consensus.cc:740] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.946113   512 consensus_queue.cc:260] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.944135   516 leader_election.cc:290] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:34.952258   512 ts_tablet_manager.cc:1434] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:10:34.953537   512 tablet_bootstrap.cc:492] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:34.960402   512 tablet_bootstrap.cc:654] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.977094 32687 catalog_manager.cc:5697] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.983327   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:34.984540   321 raft_consensus.cc:2468] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:34.986020   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:34.988890   543 raft_consensus.cc:493] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.989034   516 raft_consensus.cc:493] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.989554   516 raft_consensus.cc:515] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.989542   543 raft_consensus.cc:515] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:34.991845   516 leader_election.cc:290] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:34.991657   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:34.992470   436 leader_election.cc:304] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:34.992637   320 raft_consensus.cc:2468] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 0.
I20260709 13:10:34.993212   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
W20260709 13:10:34.993610   436 leader_election.cc:343] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.994098   321 raft_consensus.cc:2468] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 0.
I20260709 13:10:34.991845   543 leader_election.cc:290] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:34.996637   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514b957834b4abab3de4af238532008" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:34.997287   543 raft_consensus.cc:2804] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:34.997725   543 raft_consensus.cc:493] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:34.998257   543 raft_consensus.cc:3060] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:34.996186   362 leader_election.cc:304] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:34.999514   519 raft_consensus.cc:2804] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.000073   519 raft_consensus.cc:493] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.000535   519 raft_consensus.cc:3060] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.003952   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e124b2bdf5a4807a77b0b9a4df0c3a0" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
W20260709 13:10:35.006847   436 leader_election.cc:343] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.008649   516 raft_consensus.cc:493] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.008846   362 leader_election.cc:304] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:35.009639   516 raft_consensus.cc:515] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.012655   516 leader_election.cc:290] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.014854   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:35.015933   321 raft_consensus.cc:2468] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:35.017318   513 tablet_bootstrap.cc:654] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.017897   436 leader_election.cc:304] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:35.019306   516 raft_consensus.cc:2804] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.019892   516 raft_consensus.cc:493] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.020376   516 raft_consensus.cc:3060] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.024277   511 tablet_bootstrap.cc:492] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:35.024922   511 ts_tablet_manager.cc:1403] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.133s	user 0.036s	sys 0.063s
I20260709 13:10:35.026007   547 raft_consensus.cc:2804] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.026516   547 raft_consensus.cc:493] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.026880   547 raft_consensus.cc:3060] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.027158   535 leader_election.cc:290] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.028918   511 raft_consensus.cc:359] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.054433   511 raft_consensus.cc:385] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.054787   511 raft_consensus.cc:740] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.055614   511 consensus_queue.cc:260] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.057999   511 ts_tablet_manager.cc:1434] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 13:10:35.059063   511 tablet_bootstrap.cc:492] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:35.064783   512 tablet_bootstrap.cc:492] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:35.065455   512 ts_tablet_manager.cc:1403] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.112s	user 0.044s	sys 0.054s
I20260709 13:10:35.064719   516 raft_consensus.cc:515] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.069283   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:35.069543   511 tablet_bootstrap.cc:654] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.069940   320 raft_consensus.cc:3060] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.071933   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.073047   516 leader_election.cc:290] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.068446   512 raft_consensus.cc:359] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.045964   396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.046931   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.074071   512 raft_consensus.cc:385] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.047845   394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514b957834b4abab3de4af238532008" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.074494   512 raft_consensus.cc:740] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.074963   394 raft_consensus.cc:2393] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d0e9408b380e414dbe5daf286b6a0a41 in current term 1: Already voted for candidate 5592f0c3492e4e11b54a6f21387bef95 in this term.
I20260709 13:10:35.048738   393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e124b2bdf5a4807a77b0b9a4df0c3a0" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.075443   512 consensus_queue.cc:260] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.049665   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.049530   391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cf29e468ff44aba1ab2ab8981ad02a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.077708   547 raft_consensus.cc:515] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.079094   435 leader_election.cc:304] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.079468   396 raft_consensus.cc:2393] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d0e9408b380e414dbe5daf286b6a0a41 in current term 1: Already voted for candidate 5f9b867a960045008f271fe1fc3a447d in this term.
W20260709 13:10:35.080466   435 leader_election.cc:343] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.080718   516 raft_consensus.cc:3060] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.059144   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.080981   435 leader_election.cc:304] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.081667   471 raft_consensus.cc:2393] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.082010   535 raft_consensus.cc:2749] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:35.083178   435 leader_election.cc:343] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.089118   320 raft_consensus.cc:2468] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:35.090615   512 ts_tablet_manager.cc:1434] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.002s
I20260709 13:10:35.024741   519 raft_consensus.cc:515] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.061357   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.034924   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cf29e468ff44aba1ab2ab8981ad02a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:35.091862   512 tablet_bootstrap.cc:492] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:35.091795   321 raft_consensus.cc:2468] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:35.041706   549 raft_consensus.cc:493] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.050817   543 raft_consensus.cc:515] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.093187   519 leader_election.cc:290] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.092890   549 raft_consensus.cc:515] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.094018   390 raft_consensus.cc:2393] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d0e9408b380e414dbe5daf286b6a0a41 in current term 1: Already voted for candidate 5f9b867a960045008f271fe1fc3a447d in this term.
I20260709 13:10:35.094026   395 raft_consensus.cc:2468] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:35.094408   543 leader_election.cc:290] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.095499   470 raft_consensus.cc:2393] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.096210   549 leader_election.cc:290] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.091104   436 leader_election.cc:304] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:35.097258   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:35.098134   321 raft_consensus.cc:2393] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.098493   436 leader_election.cc:304] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.099068   516 raft_consensus.cc:2749] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.099928   535 raft_consensus.cc:2804] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.100180   516 raft_consensus.cc:2804] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.100689   516 raft_consensus.cc:493] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.100741   535 raft_consensus.cc:697] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:35.101214   516 raft_consensus.cc:3060] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.101889   535 consensus_queue.cc:237] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.104386   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:35.105213   321 raft_consensus.cc:3060] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.105829   435 leader_election.cc:304] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.109381   543 raft_consensus.cc:2804] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.109840   543 raft_consensus.cc:493] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.110234   543 raft_consensus.cc:3060] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.110039   516 raft_consensus.cc:515] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.115379   321 raft_consensus.cc:2468] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:35.107053   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:35.112763   516 leader_election.cc:290] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.116719   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
W20260709 13:10:35.118376   362 leader_election.cc:343] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.098852   512 tablet_bootstrap.cc:654] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.106091   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.121232   436 leader_election.cc:304] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:35.122426   516 raft_consensus.cc:2804] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.123600   516 raft_consensus.cc:697] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:35.125283   516 consensus_queue.cc:237] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.134887   547 leader_election.cc:290] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.123485   359 leader_election.cc:304] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:35.126889   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cf29e468ff44aba1ab2ab8981ad02a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:35.129583   513 tablet_bootstrap.cc:492] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:35.133311   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:35.137734   321 raft_consensus.cc:2393] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.137535   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.138720   395 raft_consensus.cc:2393] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d0e9408b380e414dbe5daf286b6a0a41 in current term 1: Already voted for candidate 5f9b867a960045008f271fe1fc3a447d in this term.
I20260709 13:10:35.138795   511 tablet_bootstrap.cc:492] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:35.139523   511 ts_tablet_manager.cc:1403] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.081s	user 0.033s	sys 0.023s
I20260709 13:10:35.139676   513 ts_tablet_manager.cc:1403] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.241s	user 0.017s	sys 0.002s
I20260709 13:10:35.140455   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cf29e468ff44aba1ab2ab8981ad02a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.142534   320 raft_consensus.cc:3060] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.142990   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:35.143309   513 raft_consensus.cc:359] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.144166   513 raft_consensus.cc:385] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.144495   513 raft_consensus.cc:740] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.145341   513 consensus_queue.cc:260] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.145869   359 leader_election.cc:304] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
W20260709 13:10:35.147877   435 leader_election.cc:343] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.149672   511 raft_consensus.cc:359] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.150554   511 raft_consensus.cc:385] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.150887   511 raft_consensus.cc:740] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.151794   549 raft_consensus.cc:2749] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.152216   543 raft_consensus.cc:515] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.152842   549 raft_consensus.cc:2749] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.153584   471 raft_consensus.cc:2393] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.155879   359 leader_election.cc:304] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:35.156927   549 raft_consensus.cc:3060] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.157279   320 raft_consensus.cc:2468] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:35.161412   436 leader_election.cc:304] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.162636   543 leader_election.cc:290] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.163443   543 raft_consensus.cc:2804] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.166163   543 raft_consensus.cc:697] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:35.166939   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.167042   543 consensus_queue.cc:237] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.167402   511 consensus_queue.cc:260] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.165406   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
W20260709 13:10:35.169960   436 leader_election.cc:343] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.170135   511 ts_tablet_manager.cc:1434] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 13:10:35.170881   549 raft_consensus.cc:2749] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.171463   511 tablet_bootstrap.cc:492] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:35.172212   529 raft_consensus.cc:493] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.172744   529 raft_consensus.cc:515] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.175544   512 tablet_bootstrap.cc:492] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:35.176199   512 ts_tablet_manager.cc:1403] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.085s	user 0.034s	sys 0.017s
I20260709 13:10:35.167057   513 ts_tablet_manager.cc:1434] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:10:35.177517   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.178224   471 raft_consensus.cc:2468] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:35.179201   512 raft_consensus.cc:359] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.179807 32752 leader_election.cc:304] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:35.179939   395 raft_consensus.cc:2468] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:35.180891   513 tablet_bootstrap.cc:492] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:35.180958   518 raft_consensus.cc:2804] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.181538   518 raft_consensus.cc:493] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.181892   518 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.179960   512 raft_consensus.cc:385] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.182597   512 raft_consensus.cc:740] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.183547   512 consensus_queue.cc:260] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.186368   512 ts_tablet_manager.cc:1434] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:10:35.187752   516 raft_consensus.cc:493] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.188402   516 raft_consensus.cc:515] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.191166   516 leader_election.cc:290] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.192153   529 leader_election.cc:290] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.192438   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:35.194077   513 tablet_bootstrap.cc:654] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.193042   435 leader_election.cc:304] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.194602   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
W20260709 13:10:35.194841   436 leader_election.cc:343] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.195556   516 raft_consensus.cc:2804] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.194133   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.196270   516 raft_consensus.cc:697] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:35.196457   395 raft_consensus.cc:2468] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:35.167057 32687 catalog_manager.cc:5697] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:35.197763 32754 leader_election.cc:343] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.197278   516 consensus_queue.cc:237] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.198964   518 raft_consensus.cc:515] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.199420   435 leader_election.cc:304] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.200533 32687 catalog_manager.cc:5697] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:35.202806   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:35.203428   471 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.193635   511 tablet_bootstrap.cc:654] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.204288   543 raft_consensus.cc:2804] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.206118   543 raft_consensus.cc:493] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.210793   543 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.212308   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.213846   471 raft_consensus.cc:2468] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
W20260709 13:10:35.214740 32754 leader_election.cc:343] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.215502 32752 leader_election.cc:304] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.216698   529 raft_consensus.cc:2804] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.220335   518 leader_election.cc:290] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.223392   513 tablet_bootstrap.cc:492] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:35.224035   513 ts_tablet_manager.cc:1403] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.000s
I20260709 13:10:35.225445   529 raft_consensus.cc:697] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:35.226377   543 raft_consensus.cc:515] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.226905   529 consensus_queue.cc:237] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.231611   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
W20260709 13:10:35.233517   436 leader_election.cc:343] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.233845   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.234500   395 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.237253   513 raft_consensus.cc:359] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.238104   513 raft_consensus.cc:385] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.238415   513 raft_consensus.cc:740] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.240589   513 consensus_queue.cc:260] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.242429   511 tablet_bootstrap.cc:492] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:35.243204   511 ts_tablet_manager.cc:1403] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.072s	user 0.029s	sys 0.016s
I20260709 13:10:35.246148   513 ts_tablet_manager.cc:1434] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:10:35.247364   513 tablet_bootstrap.cc:492] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:35.242894 32684 catalog_manager.cc:5697] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:35.247514   511 raft_consensus.cc:359] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.248600   511 raft_consensus.cc:385] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.249025   511 raft_consensus.cc:740] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.249545 32684 catalog_manager.cc:5697] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:35.253201   543 leader_election.cc:290] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:35.250033   511 consensus_queue.cc:260] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.257182   395 raft_consensus.cc:2468] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:35.258486   511 ts_tablet_manager.cc:1434] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:10:35.258496   435 leader_election.cc:304] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.260604   543 raft_consensus.cc:2804] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.261312   543 raft_consensus.cc:697] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:35.262452   543 consensus_queue.cc:237] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.255478   513 tablet_bootstrap.cc:654] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.270090   511 tablet_bootstrap.cc:492] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:35.259917 32686 catalog_manager.cc:5697] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:35.277745   511 tablet_bootstrap.cc:654] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.280049   549 raft_consensus.cc:493] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.280777   549 raft_consensus.cc:515] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.283051   549 leader_election.cc:290] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.284297   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f28fd9f6fa4c4799b20e357dc87813" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.285017   471 raft_consensus.cc:2468] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 0.
I20260709 13:10:35.285386   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f28fd9f6fa4c4799b20e357dc87813" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:35.286371   359 leader_election.cc:304] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:35.287295   549 raft_consensus.cc:2804] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:35.287289   362 leader_election.cc:343] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.288498   513 tablet_bootstrap.cc:492] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:35.289090   513 ts_tablet_manager.cc:1403] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.010s
I20260709 13:10:35.287866   549 raft_consensus.cc:493] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.292032   549 raft_consensus.cc:3060] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.297919   513 raft_consensus.cc:359] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.298718   513 raft_consensus.cc:385] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.298241   529 consensus_queue.cc:1048] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:35.301208 32686 catalog_manager.cc:5697] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:35.301777   549 raft_consensus.cc:515] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.298964   513 raft_consensus.cc:740] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.305095   549 leader_election.cc:290] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.305369   513 consensus_queue.cc:260] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.305933   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f28fd9f6fa4c4799b20e357dc87813" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:35.305934   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f28fd9f6fa4c4799b20e357dc87813" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:35.306795   470 raft_consensus.cc:3060] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:35.307421   362 leader_election.cc:343] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:35.307667   513 ts_tablet_manager.cc:1434] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:10:35.308640   513 tablet_bootstrap.cc:492] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:35.315208   511 tablet_bootstrap.cc:492] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:35.315865   511 ts_tablet_manager.cc:1403] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.017s
I20260709 13:10:35.315073   470 raft_consensus.cc:2468] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 1.
I20260709 13:10:35.317780   359 leader_election.cc:304] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.318995   549 raft_consensus.cc:2804] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.319579   513 tablet_bootstrap.cc:654] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.318981   511 raft_consensus.cc:359] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.319931   549 raft_consensus.cc:697] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:35.320210   511 raft_consensus.cc:385] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.320603   511 raft_consensus.cc:740] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.321477   511 consensus_queue.cc:260] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.321111   549 consensus_queue.cc:237] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.326372   511 ts_tablet_manager.cc:1434] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 13:10:35.327831   511 tablet_bootstrap.cc:492] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:35.341540   511 tablet_bootstrap.cc:654] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.352504 32686 catalog_manager.cc:5697] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 1 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:35.354610   511 tablet_bootstrap.cc:492] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:35.355399   511 ts_tablet_manager.cc:1403] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.001s
I20260709 13:10:35.367818   511 raft_consensus.cc:359] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.368744   511 raft_consensus.cc:385] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.369088   511 raft_consensus.cc:740] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.369956   511 consensus_queue.cc:260] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.371388   513 tablet_bootstrap.cc:492] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:35.371958   513 ts_tablet_manager.cc:1403] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.032s
I20260709 13:10:35.372310   511 ts_tablet_manager.cc:1434] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.010s
I20260709 13:10:35.373392   511 tablet_bootstrap.cc:492] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:35.374898   513 raft_consensus.cc:359] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.375830   513 raft_consensus.cc:385] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.376171   513 raft_consensus.cc:740] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.377233   513 consensus_queue.cc:260] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.380175   511 tablet_bootstrap.cc:654] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.382808   513 ts_tablet_manager.cc:1434] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:35.391908   559 consensus_queue.cc:1048] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:35.401525   511 tablet_bootstrap.cc:492] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:35.402122   511 ts_tablet_manager.cc:1403] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.015s
I20260709 13:10:35.405310   511 raft_consensus.cc:359] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.406015   511 raft_consensus.cc:385] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.406284   511 raft_consensus.cc:740] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.406950   511 consensus_queue.cc:260] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.409103   511 ts_tablet_manager.cc:1434] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:35.413753   549 raft_consensus.cc:493] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.414335   549 raft_consensus.cc:515] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.416546   549 leader_election.cc:290] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.417310   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cf29e468ff44aba1ab2ab8981ad02a" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:35.418160   320 raft_consensus.cc:2393] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.418222   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cf29e468ff44aba1ab2ab8981ad02a" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.420006   359 leader_election.cc:304] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:35.420713   549 raft_consensus.cc:3060] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.427289   549 raft_consensus.cc:2749] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.437363   529 raft_consensus.cc:493] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.437866   529 raft_consensus.cc:515] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.439607   529 leader_election.cc:290] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.440376   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f28fd9f6fa4c4799b20e357dc87813" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.440874   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f28fd9f6fa4c4799b20e357dc87813" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.441654   470 raft_consensus.cc:2393] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5592f0c3492e4e11b54a6f21387bef95 in current term 1: Already voted for candidate 5f9b867a960045008f271fe1fc3a447d in this term.
I20260709 13:10:35.442687 32752 leader_election.cc:304] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95; no voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:35.443521   529 raft_consensus.cc:3060] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.448594   529 raft_consensus.cc:2749] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.530303   549 raft_consensus.cc:493] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.530898   549 raft_consensus.cc:515] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.532488   549 leader_election.cc:290] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.533705   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.533655   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:35.534865   470 raft_consensus.cc:2393] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate 5592f0c3492e4e11b54a6f21387bef95 in this term.
I20260709 13:10:35.535976   359 leader_election.cc:304] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:35.536583   549 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.541594   549 raft_consensus.cc:2749] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.567430   543 consensus_queue.cc:1048] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.570117   543 consensus_queue.cc:1048] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.578859   535 consensus_queue.cc:1048] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.588317   535 consensus_queue.cc:1048] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:35.603435   529 raft_consensus.cc:493] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.603948   529 raft_consensus.cc:515] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.606886   529 leader_election.cc:290] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.607062   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.607206   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.608040   390 raft_consensus.cc:2393] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5592f0c3492e4e11b54a6f21387bef95 in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:35.609099 32754 leader_election.cc:304] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95; no voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:35.610086   529 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.615053   529 raft_consensus.cc:2749] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:35.647388   529 consensus_queue.cc:1048] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.658841   559 consensus_queue.cc:1048] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.679179   575 consensus_queue.cc:1048] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.685683   321 raft_consensus.cc:3060] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.694151   575 consensus_queue.cc:1048] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:35.697618   516 consensus_queue.cc:1048] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.707914   575 consensus_queue.cc:1048] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.720322   559 raft_consensus.cc:493] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.721101   559 raft_consensus.cc:515] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.723198   559 leader_election.cc:290] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.723958   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e124b2bdf5a4807a77b0b9a4df0c3a0" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:35.724433   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e124b2bdf5a4807a77b0b9a4df0c3a0" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:35.724727   390 raft_consensus.cc:2468] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:35.725178   470 raft_consensus.cc:2468] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:35.725869 32754 leader_election.cc:304] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:35.726689   559 raft_consensus.cc:2804] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:35.727032   559 raft_consensus.cc:493] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.727254   559 raft_consensus.cc:3060] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.732228   559 raft_consensus.cc:515] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.733961   559 leader_election.cc:290] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:35.734656   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e124b2bdf5a4807a77b0b9a4df0c3a0" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:35.734941   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e124b2bdf5a4807a77b0b9a4df0c3a0" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:35.735404   390 raft_consensus.cc:3060] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.735466   470 raft_consensus.cc:3060] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:35.742966   470 raft_consensus.cc:2468] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:35.742970   390 raft_consensus.cc:2468] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:35.744349 32754 leader_election.cc:304] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:35.745116   559 raft_consensus.cc:2804] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:35.745644   559 raft_consensus.cc:697] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:35.745878   549 consensus_queue.cc:1048] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.746470   559 consensus_queue.cc:237] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:35.753465   575 consensus_queue.cc:1048] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.756796 32687 catalog_manager.cc:5697] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:35.761879   587 consensus_queue.cc:1048] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:35.774542   516 consensus_queue.cc:1048] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.838064   470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da123722312340b088054d80b503c16f"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:35.838596   470 raft_consensus.cc:606] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.846467   470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:35.846972   470 raft_consensus.cc:606] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:35.860739   320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:35376
I20260709 13:10:35.861325   320 raft_consensus.cc:606] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.866291   470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:35.866758   470 raft_consensus.cc:606] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:35.871765   470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:35.872277   470 raft_consensus.cc:606] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:36.042187   543 raft_consensus.cc:993] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:36.042743   516 raft_consensus.cc:1081] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:36.044399   390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41194
I20260709 13:10:36.045130   390 raft_consensus.cc:493] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.045470   390 raft_consensus.cc:3060] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.050963   390 raft_consensus.cc:515] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.053150   390 leader_election.cc:290] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:36.053917   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:36.054365   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8bb5f5852b4bd89fa8abeb60d6ad8e" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:36.054605   320 raft_consensus.cc:3060] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.055014   470 raft_consensus.cc:3055] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.055327   470 raft_consensus.cc:740] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:36.056130   470 consensus_queue.cc:260] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.057284   470 raft_consensus.cc:3060] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.060000   320 raft_consensus.cc:2468] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:36.061393   362 leader_election.cc:304] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:36.062309   587 raft_consensus.cc:2804] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.062805   587 raft_consensus.cc:697] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:36.063616   470 raft_consensus.cc:2468] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:36.063599   587 consensus_queue.cc:237] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.070659 32687 catalog_manager.cc:5697] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.101523   543 raft_consensus.cc:993] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:36.101964   516 raft_consensus.cc:1081] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:36.103468   320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:36.103974   320 raft_consensus.cc:493] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.104298   320 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.109450   320 raft_consensus.cc:515] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.111131   320 leader_election.cc:290] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:36.112185   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:36.112205   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:36.113114   390 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.113212   470 raft_consensus.cc:3055] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.113648   470 raft_consensus.cc:740] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:36.114459   470 consensus_queue.cc:260] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.115512   470 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.118644   390 raft_consensus.cc:2468] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:36.120002 32754 leader_election.cc:304] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:36.120790   559 raft_consensus.cc:2804] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.120944   470 raft_consensus.cc:2468] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:36.121346   559 raft_consensus.cc:697] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:36.122215   559 consensus_queue.cc:237] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.129110 32687 catalog_manager.cc:5697] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.192399   535 raft_consensus.cc:993] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:36.193045   516 raft_consensus.cc:1081] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:36.194707   320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da123722312340b088054d80b503c16f"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:36.195255   320 raft_consensus.cc:493] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.195642   320 raft_consensus.cc:3060] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.202111   518 raft_consensus.cc:993] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:36.202806   518 raft_consensus.cc:1081] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:36.203437   320 raft_consensus.cc:515] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.206503   390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41184
I20260709 13:10:36.206869   320 leader_election.cc:290] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:36.207150   395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:36.207789   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da123722312340b088054d80b503c16f" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:36.207846   395 raft_consensus.cc:3060] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.208400   470 raft_consensus.cc:3055] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.207089   390 raft_consensus.cc:493] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.208827   470 raft_consensus.cc:740] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:36.209097   390 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.209820   470 consensus_queue.cc:260] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.210898   470 raft_consensus.cc:3060] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.214658   535 raft_consensus.cc:993] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:36.215246   543 raft_consensus.cc:1081] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:36.216639   395 raft_consensus.cc:2468] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:36.217182   392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41194
I20260709 13:10:36.217525   396 raft_consensus.cc:1240] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Rejecting Update request from peer d0e9408b380e414dbe5daf286b6a0a41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:36.217814   392 raft_consensus.cc:493] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.218140 32754 leader_election.cc:304] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:36.218169   392 raft_consensus.cc:3060] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.219193   559 raft_consensus.cc:2804] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.219753   559 raft_consensus.cc:697] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:36.220517   559 consensus_queue.cc:237] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.218354   390 raft_consensus.cc:515] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.225203   390 leader_election.cc:290] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:36.225912   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:36.226747   320 raft_consensus.cc:3055] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.226517   392 raft_consensus.cc:515] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.226945   471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:36.227308   320 raft_consensus.cc:740] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:36.227614   471 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.229059   392 leader_election.cc:290] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:36.229825   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:36.230485   321 raft_consensus.cc:3060] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.229075 32687 catalog_manager.cc:5697] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.232656   470 raft_consensus.cc:2468] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:36.234323   320 consensus_queue.cc:260] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.238166   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e242b10da2df46778fbbd4910a97808f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:36.238860   470 raft_consensus.cc:3055] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.240434   470 raft_consensus.cc:740] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:36.240479   471 raft_consensus.cc:2468] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:36.241065   518 consensus_queue.cc:1048] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.241513   470 consensus_queue.cc:260] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.241937   359 leader_election.cc:304] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:36.242923   587 raft_consensus.cc:2804] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.243098   470 raft_consensus.cc:3060] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.243498   587 raft_consensus.cc:697] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:36.239928   320 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.244480   587 consensus_queue.cc:237] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.244450   321 raft_consensus.cc:2468] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:36.248133   362 leader_election.cc:304] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:36.249037   519 raft_consensus.cc:2804] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.249590   519 raft_consensus.cc:697] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:36.250341   519 consensus_queue.cc:237] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:36.255168 32687 catalog_manager.cc:5697] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.256623   320 raft_consensus.cc:2468] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:36.262959   470 raft_consensus.cc:2468] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:36.263430   559 consensus_queue.cc:1048] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.276067 32687 catalog_manager.cc:5697] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.558188   471 raft_consensus.cc:1275] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.559650   519 consensus_queue.cc:1048] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.568606   320 raft_consensus.cc:1275] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.570385   587 consensus_queue.cc:1048] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.662953   392 raft_consensus.cc:1275] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.664335   559 consensus_queue.cc:1048] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.672590   471 raft_consensus.cc:1275] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.674089   518 consensus_queue.cc:1048] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.685918   320 raft_consensus.cc:1275] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.687335   597 consensus_queue.cc:1048] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.696558   392 raft_consensus.cc:1275] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.696633   471 raft_consensus.cc:1275] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.698304   529 consensus_queue.cc:1048] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.700728   598 consensus_queue.cc:1048] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.711184   470 raft_consensus.cc:1275] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.713207   559 consensus_queue.cc:1048] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.715080   320 raft_consensus.cc:1275] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.716816   597 consensus_queue.cc:1048] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.732344   471 raft_consensus.cc:1275] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:36.733986   519 consensus_queue.cc:1048] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:38.914696 32687 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48262:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:38.917173 32687 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:10:38.951640 32735 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:38.972692   308 tablet_service.cc:1511] Processing CreateTablet for tablet 6709838ded034f6599c94a98e115fbeb (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:38.974409   308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6709838ded034f6599c94a98e115fbeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:38.974125   384 tablet_service.cc:1511] Processing CreateTablet for tablet 6709838ded034f6599c94a98e115fbeb (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:38.975595   384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6709838ded034f6599c94a98e115fbeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:38.975955   458 tablet_service.cc:1511] Processing CreateTablet for tablet 6709838ded034f6599c94a98e115fbeb (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:38.977427   458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6709838ded034f6599c94a98e115fbeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:38.979671   311 tablet_service.cc:1511] Processing CreateTablet for tablet fa2129ab623d475783bc6781676dbc7a (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:38.981204   311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa2129ab623d475783bc6781676dbc7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:38.999964   382 tablet_service.cc:1511] Processing CreateTablet for tablet fa2129ab623d475783bc6781676dbc7a (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:39.002744   382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa2129ab623d475783bc6781676dbc7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.003865   454 tablet_service.cc:1511] Processing CreateTablet for tablet fa2129ab623d475783bc6781676dbc7a (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:39.005470   454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa2129ab623d475783bc6781676dbc7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.020682   309 tablet_service.cc:1511] Processing CreateTablet for tablet 610388a9a21e472f910feae8f7b149b9 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:39.022593   309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610388a9a21e472f910feae8f7b149b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.025689   378 tablet_service.cc:1511] Processing CreateTablet for tablet 610388a9a21e472f910feae8f7b149b9 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:39.027395   378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610388a9a21e472f910feae8f7b149b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.031273   461 tablet_service.cc:1511] Processing CreateTablet for tablet 610388a9a21e472f910feae8f7b149b9 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:39.035254   461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610388a9a21e472f910feae8f7b149b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.031301   304 tablet_service.cc:1511] Processing CreateTablet for tablet dfb5083bb35f41eb906f8c0cc6821424 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:39.046056   616 tablet_bootstrap.cc:492] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.048519   383 tablet_service.cc:1511] Processing CreateTablet for tablet dfb5083bb35f41eb906f8c0cc6821424 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:39.050292   383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb5083bb35f41eb906f8c0cc6821424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.054203   616 tablet_bootstrap.cc:654] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.059118   304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb5083bb35f41eb906f8c0cc6821424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.063052   460 tablet_service.cc:1511] Processing CreateTablet for tablet dfb5083bb35f41eb906f8c0cc6821424 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:39.066356   460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb5083bb35f41eb906f8c0cc6821424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.066886   308 tablet_service.cc:1511] Processing CreateTablet for tablet e1fb913ccb2043d08d0dcd002bc8b70b (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:39.068285   308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1fb913ccb2043d08d0dcd002bc8b70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.071261   455 tablet_service.cc:1511] Processing CreateTablet for tablet e1fb913ccb2043d08d0dcd002bc8b70b (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:39.075558   617 tablet_bootstrap.cc:492] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.075380   310 tablet_service.cc:1511] Processing CreateTablet for tablet 5b913f3515b545c88b5298fb10c97a6c (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:39.077356   378 tablet_service.cc:1511] Processing CreateTablet for tablet e1fb913ccb2043d08d0dcd002bc8b70b (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:39.078590   310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b913f3515b545c88b5298fb10c97a6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.078838   378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1fb913ccb2043d08d0dcd002bc8b70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.072876   455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1fb913ccb2043d08d0dcd002bc8b70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.087611   384 tablet_service.cc:1511] Processing CreateTablet for tablet 5b913f3515b545c88b5298fb10c97a6c (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:39.089336   384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b913f3515b545c88b5298fb10c97a6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.089605   458 tablet_service.cc:1511] Processing CreateTablet for tablet 5b913f3515b545c88b5298fb10c97a6c (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:39.094702   618 tablet_bootstrap.cc:492] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.097790   458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b913f3515b545c88b5298fb10c97a6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.099712   311 tablet_service.cc:1511] Processing CreateTablet for tablet 5f2face53ea443e6b104f8a4062cdd3f (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:39.102054   311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2face53ea443e6b104f8a4062cdd3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.102837   381 tablet_service.cc:1511] Processing CreateTablet for tablet 5f2face53ea443e6b104f8a4062cdd3f (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:39.104458   381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2face53ea443e6b104f8a4062cdd3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.106353   617 tablet_bootstrap.cc:654] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.109825   453 tablet_service.cc:1511] Processing CreateTablet for tablet 5f2face53ea443e6b104f8a4062cdd3f (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:39.175493   453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2face53ea443e6b104f8a4062cdd3f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:39.151589   362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.129:40573, user_credentials={real_user=slave}} blocked reactor thread for 89001.1us
I20260709 13:10:39.166743   309 tablet_service.cc:1511] Processing CreateTablet for tablet c7dd0d5bcb96486d96ad9152c34bbcfc (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:39.177973   309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7dd0d5bcb96486d96ad9152c34bbcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.159116   618 tablet_bootstrap.cc:654] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.170328   382 tablet_service.cc:1511] Processing CreateTablet for tablet c7dd0d5bcb96486d96ad9152c34bbcfc (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:39.173045   454 tablet_service.cc:1511] Processing CreateTablet for tablet c7dd0d5bcb96486d96ad9152c34bbcfc (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:39.182565   382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7dd0d5bcb96486d96ad9152c34bbcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.182977   454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7dd0d5bcb96486d96ad9152c34bbcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.187068   616 tablet_bootstrap.cc:492] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.188128   308 tablet_service.cc:1511] Processing CreateTablet for tablet 48f09b3a8da74c6d914203d040e3b786 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:39.189706   308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48f09b3a8da74c6d914203d040e3b786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.195638   380 tablet_service.cc:1511] Processing CreateTablet for tablet 48f09b3a8da74c6d914203d040e3b786 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:39.187800   616 ts_tablet_manager.cc:1403] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.036s
I20260709 13:10:39.205715   380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48f09b3a8da74c6d914203d040e3b786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.207649   616 raft_consensus.cc:359] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.208536   616 raft_consensus.cc:385] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.208822   616 raft_consensus.cc:740] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.209932   616 consensus_queue.cc:260] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.211752   460 tablet_service.cc:1511] Processing CreateTablet for tablet 48f09b3a8da74c6d914203d040e3b786 (DEFAULT_TABLE table=soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:39.213532   460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48f09b3a8da74c6d914203d040e3b786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:39.226727   616 ts_tablet_manager.cc:1434] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 13:10:39.259236   616 tablet_bootstrap.cc:492] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.272688   617 tablet_bootstrap.cc:492] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.273607   617 ts_tablet_manager.cc:1403] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.198s	user 0.022s	sys 0.016s
I20260709 13:10:39.275766   616 tablet_bootstrap.cc:654] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.277292   617 raft_consensus.cc:359] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.280048   618 tablet_bootstrap.cc:492] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.280844   618 ts_tablet_manager.cc:1403] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.187s	user 0.017s	sys 0.000s
I20260709 13:10:39.281965   617 raft_consensus.cc:385] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.282358   617 raft_consensus.cc:740] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.283253   617 consensus_queue.cc:260] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.283799   616 tablet_bootstrap.cc:492] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.284536   616 ts_tablet_manager.cc:1403] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.003s
I20260709 13:10:39.284387   618 raft_consensus.cc:359] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.285249   618 raft_consensus.cc:385] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.285554   618 raft_consensus.cc:740] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.286381   618 consensus_queue.cc:260] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.287910   616 raft_consensus.cc:359] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.288630   616 raft_consensus.cc:385] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.288904   616 raft_consensus.cc:740] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.288934   618 ts_tablet_manager.cc:1434] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:10:39.289829   616 consensus_queue.cc:260] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.290375   618 tablet_bootstrap.cc:492] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.296357   616 ts_tablet_manager.cc:1434] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:10:39.298204   616 tablet_bootstrap.cc:492] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.299422   618 tablet_bootstrap.cc:654] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.305889   616 tablet_bootstrap.cc:654] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.312212   618 tablet_bootstrap.cc:492] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.312873   618 ts_tablet_manager.cc:1403] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:10:39.316475   618 raft_consensus.cc:359] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.317448   618 raft_consensus.cc:385] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.317849   618 raft_consensus.cc:740] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.318833   618 consensus_queue.cc:260] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.320307   529 raft_consensus.cc:493] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.320856   529 raft_consensus.cc:515] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.323217   529 leader_election.cc:290] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.323369   616 tablet_bootstrap.cc:492] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.324086   616 ts_tablet_manager.cc:1403] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.004s
I20260709 13:10:39.325301   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6709838ded034f6599c94a98e115fbeb" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.325840   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6709838ded034f6599c94a98e115fbeb" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.326121   470 raft_consensus.cc:2468] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:39.326565   392 raft_consensus.cc:2468] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:39.327741 32752 leader_election.cc:304] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.327572   616 raft_consensus.cc:359] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.328804   616 raft_consensus.cc:385] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.329128   616 raft_consensus.cc:740] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.329489   529 raft_consensus.cc:2804] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.319234   617 ts_tablet_manager.cc:1434] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.045s	user 0.002s	sys 0.005s
I20260709 13:10:39.329907   616 consensus_queue.cc:260] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.330924   617 tablet_bootstrap.cc:492] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.332494   616 ts_tablet_manager.cc:1434] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:10:39.329943   529 raft_consensus.cc:493] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.333524   529 raft_consensus.cc:3060] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.333709   616 tablet_bootstrap.cc:492] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.342124   616 tablet_bootstrap.cc:654] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.343430   529 raft_consensus.cc:515] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.347862   529 leader_election.cc:290] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.349514   617 tablet_bootstrap.cc:654] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.352010   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6709838ded034f6599c94a98e115fbeb" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:39.348757   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6709838ded034f6599c94a98e115fbeb" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.353039   470 raft_consensus.cc:3060] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.353161   392 raft_consensus.cc:3060] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.372174   616 tablet_bootstrap.cc:492] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.372387   470 raft_consensus.cc:2468] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:39.372879   616 ts_tablet_manager.cc:1403] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.016s
I20260709 13:10:39.375917   616 raft_consensus.cc:359] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.376659   616 raft_consensus.cc:385] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.376955   616 raft_consensus.cc:740] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.377853   616 consensus_queue.cc:260] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.379011   618 ts_tablet_manager.cc:1434] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.066s	user 0.010s	sys 0.000s
I20260709 13:10:39.379658   616 ts_tablet_manager.cc:1434] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:39.380844 32752 leader_election.cc:304] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.382216   616 tablet_bootstrap.cc:492] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.382154   626 raft_consensus.cc:2804] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.387487   618 tablet_bootstrap.cc:492] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.392133   616 tablet_bootstrap.cc:654] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.394171   626 raft_consensus.cc:697] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:39.395284   626 consensus_queue.cc:237] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.397362   617 tablet_bootstrap.cc:492] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.398053   617 ts_tablet_manager.cc:1403] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.067s	user 0.009s	sys 0.012s
I20260709 13:10:39.402642   617 raft_consensus.cc:359] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.403450   617 raft_consensus.cc:385] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.403754   617 raft_consensus.cc:740] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.383378   392 raft_consensus.cc:2468] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:39.404557   617 consensus_queue.cc:260] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.410245   616 tablet_bootstrap.cc:492] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.410904   616 ts_tablet_manager.cc:1403] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.005s
I20260709 13:10:39.412482   618 tablet_bootstrap.cc:654] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.413801   616 raft_consensus.cc:359] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.414636   616 raft_consensus.cc:385] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.417374   616 raft_consensus.cc:740] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.417397 32683 catalog_manager.cc:5697] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.418381   616 consensus_queue.cc:260] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.420931   616 ts_tablet_manager.cc:1434] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:10:39.422325   616 tablet_bootstrap.cc:492] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.423066   617 ts_tablet_manager.cc:1434] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:10:39.424070   587 raft_consensus.cc:493] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.424340   617 tablet_bootstrap.cc:492] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.431586   616 tablet_bootstrap.cc:654] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.424665   587 raft_consensus.cc:515] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.431906   617 tablet_bootstrap.cc:654] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.441881   616 tablet_bootstrap.cc:492] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.442770   616 ts_tablet_manager.cc:1403] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:10:39.444546   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.445389   468 raft_consensus.cc:2468] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 0.
I20260709 13:10:39.445436   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.446861   616 raft_consensus.cc:359] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.449081   587 leader_election.cc:290] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.449265   616 raft_consensus.cc:385] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.447247   359 leader_election.cc:304] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.450341   616 raft_consensus.cc:740] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
W20260709 13:10:39.450894   362 leader_election.cc:343] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.451576   617 tablet_bootstrap.cc:492] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.451385   616 consensus_queue.cc:260] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.452239   617 ts_tablet_manager.cc:1403] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.014s
I20260709 13:10:39.453198   587 raft_consensus.cc:2804] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.453781   618 tablet_bootstrap.cc:492] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.454170   616 ts_tablet_manager.cc:1434] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:10:39.454475   618 ts_tablet_manager.cc:1403] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.000s
I20260709 13:10:39.455325   616 tablet_bootstrap.cc:492] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.455561   617 raft_consensus.cc:359] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.456377   617 raft_consensus.cc:385] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.456729   617 raft_consensus.cc:740] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.457732   617 consensus_queue.cc:260] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.458127   618 raft_consensus.cc:359] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.459123   618 raft_consensus.cc:385] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.459535   618 raft_consensus.cc:740] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.460676   617 ts_tablet_manager.cc:1434] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:10:39.460412   618 consensus_queue.cc:260] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.453716   587 raft_consensus.cc:493] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.461584   587 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.462785   616 tablet_bootstrap.cc:654] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.463138   618 ts_tablet_manager.cc:1434] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:39.464313   618 tablet_bootstrap.cc:492] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.469324   617 tablet_bootstrap.cc:492] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.471333   609 raft_consensus.cc:493] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.471884   609 raft_consensus.cc:515] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.474495   609 leader_election.cc:290] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.477169   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.478346   392 raft_consensus.cc:2468] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:39.478870   617 tablet_bootstrap.cc:654] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.479882   435 leader_election.cc:304] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.477236   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.481419   321 raft_consensus.cc:2468] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:39.480923   609 raft_consensus.cc:2804] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.483525   609 raft_consensus.cc:493] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.483996   609 raft_consensus.cc:3060] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.493014   617 tablet_bootstrap.cc:492] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.493953   617 ts_tablet_manager.cc:1403] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.011s
I20260709 13:10:39.496034   609 raft_consensus.cc:515] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.497627   617 raft_consensus.cc:359] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.499187   617 raft_consensus.cc:385] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.499672   617 raft_consensus.cc:740] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.501362   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:39.502077   321 raft_consensus.cc:3060] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.503033   617 consensus_queue.cc:260] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.506490   617 ts_tablet_manager.cc:1434] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:10:39.507949   617 tablet_bootstrap.cc:492] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.520313   619 raft_consensus.cc:493] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.521132   619 raft_consensus.cc:515] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.476680   587 raft_consensus.cc:515] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.525310   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.525874   320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.526914   319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:39.528263   619 leader_election.cc:290] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.525480   587 leader_election.cc:290] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
W20260709 13:10:39.529136   362 leader_election.cc:343] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:39.529846   362 leader_election.cc:343] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.530336   616 tablet_bootstrap.cc:492] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.530953   616 ts_tablet_manager.cc:1403] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.076s	user 0.048s	sys 0.024s
I20260709 13:10:39.532682   617 tablet_bootstrap.cc:654] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.534313   616 raft_consensus.cc:359] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
W20260709 13:10:39.535125   359 leader_election.cc:343] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.535462   616 raft_consensus.cc:385] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.535737   359 leader_election.cc:304] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:39.535939   616 raft_consensus.cc:740] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.536695   587 raft_consensus.cc:2749] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:39.536849   616 consensus_queue.cc:260] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.534261   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:39.537973   468 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.539985   617 tablet_bootstrap.cc:492] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.540264   616 ts_tablet_manager.cc:1434] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:10:39.541347   616 tablet_bootstrap.cc:492] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.540529   617 ts_tablet_manager.cc:1403] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.000s
I20260709 13:10:39.546176   468 raft_consensus.cc:2468] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 1.
I20260709 13:10:39.548022   616 tablet_bootstrap.cc:654] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.549032   617 raft_consensus.cc:359] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.549566   359 leader_election.cc:304] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.549907   617 raft_consensus.cc:385] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.550493   619 raft_consensus.cc:2804] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.551030   619 raft_consensus.cc:697] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:39.552147   619 consensus_queue.cc:237] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.556337   321 raft_consensus.cc:2468] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:39.556689   616 tablet_bootstrap.cc:492] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.557451   616 ts_tablet_manager.cc:1403] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.008s
I20260709 13:10:39.557754   436 leader_election.cc:304] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.558073   618 tablet_bootstrap.cc:654] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.557979   617 raft_consensus.cc:740] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.558838   638 raft_consensus.cc:2804] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.560066   617 consensus_queue.cc:260] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.560858   616 raft_consensus.cc:359] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.562254   616 raft_consensus.cc:385] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.562579   616 raft_consensus.cc:740] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.563375   609 leader_election.cc:290] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.563326   616 consensus_queue.cc:260] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.563964   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.564579   392 raft_consensus.cc:3060] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.565157   609 raft_consensus.cc:493] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.565743   616 ts_tablet_manager.cc:1434] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:39.565889   609 raft_consensus.cc:515] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.566761   616 tablet_bootstrap.cc:492] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: Bootstrap starting.
I20260709 13:10:39.569533   609 leader_election.cc:290] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.570103   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.570794   390 raft_consensus.cc:2468] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:39.572305   435 leader_election.cc:304] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.572963   392 raft_consensus.cc:2468] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:39.573788   609 raft_consensus.cc:2804] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.574221   609 raft_consensus.cc:493] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.574609   609 raft_consensus.cc:3060] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.578815   617 ts_tablet_manager.cc:1434] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:10:39.580026   618 tablet_bootstrap.cc:492] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.580112   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.580914   618 ts_tablet_manager.cc:1403] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.117s	user 0.018s	sys 0.004s
I20260709 13:10:39.581830   616 tablet_bootstrap.cc:654] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:39.581998   436 leader_election.cc:343] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.583380   638 raft_consensus.cc:697] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:39.584479   618 raft_consensus.cc:359] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.584465   638 consensus_queue.cc:237] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.585568   618 raft_consensus.cc:385] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.586001   618 raft_consensus.cc:740] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.586757   618 consensus_queue.cc:260] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.591816 32684 catalog_manager.cc:5697] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 1 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.593632   618 ts_tablet_manager.cc:1434] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:39.594712   618 tablet_bootstrap.cc:492] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.591886   609 raft_consensus.cc:515] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.598256   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:39.598641   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.598953   617 tablet_bootstrap.cc:492] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.599325   392 raft_consensus.cc:3060] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:39.600029   436 leader_election.cc:343] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.606554   616 tablet_bootstrap.cc:492] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: No bootstrap required, opened a new log
I20260709 13:10:39.606997   618 tablet_bootstrap.cc:654] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.607190   616 ts_tablet_manager.cc:1403] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.002s
I20260709 13:10:39.608544   617 tablet_bootstrap.cc:654] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.610761   616 raft_consensus.cc:359] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.611635   616 raft_consensus.cc:385] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.612012   616 raft_consensus.cc:740] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.612946   616 consensus_queue.cc:260] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.615801   616 ts_tablet_manager.cc:1434] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:39.618902   618 tablet_bootstrap.cc:492] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.619444   618 ts_tablet_manager.cc:1403] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260709 13:10:39.621999   618 raft_consensus.cc:359] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.622704   618 raft_consensus.cc:385] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.621580 32684 catalog_manager.cc:5697] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:39.622965   618 raft_consensus.cc:740] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.623775   618 consensus_queue.cc:260] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.627120   618 ts_tablet_manager.cc:1434] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:10:39.627902   617 tablet_bootstrap.cc:492] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.628171   618 tablet_bootstrap.cc:492] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.628542   617 ts_tablet_manager.cc:1403] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.008s
I20260709 13:10:39.629201   626 raft_consensus.cc:493] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.629709   626 raft_consensus.cc:515] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.631482   638 raft_consensus.cc:493] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.632019   638 raft_consensus.cc:515] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.633718   609 leader_election.cc:290] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.634001   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.634359   638 leader_election.cc:290] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.634716   468 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:39.634810   392 raft_consensus.cc:2468] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:39.635536   618 tablet_bootstrap.cc:654] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.636822   617 raft_consensus.cc:359] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.637247   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.637763   617 raft_consensus.cc:385] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.637955   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
W20260709 13:10:39.638797   436 leader_election.cc:343] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.638129   617 raft_consensus.cc:740] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.639909   617 consensus_queue.cc:260] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.640738   390 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:39.641884 32752 leader_election.cc:304] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.642450   617 ts_tablet_manager.cc:1434] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:10:39.643353   629 raft_consensus.cc:2804] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.643887   629 raft_consensus.cc:493] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.643904   617 tablet_bootstrap.cc:492] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.644316   629 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.647357   618 tablet_bootstrap.cc:492] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.647869   618 ts_tablet_manager.cc:1403] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:10:39.644757   435 leader_election.cc:304] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.649307   638 raft_consensus.cc:2804] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.649920   638 raft_consensus.cc:697] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:39.650898   638 consensus_queue.cc:237] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.658758   617 tablet_bootstrap.cc:654] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.662091   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.662966   392 raft_consensus.cc:2468] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:39.656206   626 leader_election.cc:290] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.651106   618 raft_consensus.cc:359] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.663748   529 raft_consensus.cc:493] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.666400   618 raft_consensus.cc:385] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.666734   618 raft_consensus.cc:740] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.666762   529 raft_consensus.cc:515] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.667654   618 consensus_queue.cc:260] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.670173   618 ts_tablet_manager.cc:1434] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:10:39.669976   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.671195   618 tablet_bootstrap.cc:492] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
W20260709 13:10:39.671751 32754 leader_election.cc:343] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.664121   629 raft_consensus.cc:515] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.674470   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.674726   629 leader_election.cc:290] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.675153   468 raft_consensus.cc:2468] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 0.
I20260709 13:10:39.676864 32752 leader_election.cc:304] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:39.678759   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:39.679356   468 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.681874   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.682523   629 raft_consensus.cc:2804] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.683121   529 leader_election.cc:290] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.682618   390 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.684455   617 tablet_bootstrap.cc:492] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.684985   617 ts_tablet_manager.cc:1403] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.016s
I20260709 13:10:39.685699 32684 catalog_manager.cc:5697] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:39.687208   618 tablet_bootstrap.cc:654] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.688074   617 raft_consensus.cc:359] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.688807   617 raft_consensus.cc:385] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.689107   617 raft_consensus.cc:740] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.689890   617 consensus_queue.cc:260] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.691113   619 raft_consensus.cc:493] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.691601   619 raft_consensus.cc:515] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.692286   617 ts_tablet_manager.cc:1434] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:10:39.693354   617 tablet_bootstrap.cc:492] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.699668   619 leader_election.cc:290] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.700675   617 tablet_bootstrap.cc:654] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.700747   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.703248   470 raft_consensus.cc:2468] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 0.
I20260709 13:10:39.703914   390 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:39.704792   468 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:39.705315 32754 leader_election.cc:304] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:39.706223   529 raft_consensus.cc:2804] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.706693   359 leader_election.cc:304] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.701880   435 leader_election.cc:304] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.707646   638 raft_consensus.cc:2804] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.702065   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.706705   529 raft_consensus.cc:697] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:39.708000   638 raft_consensus.cc:493] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.708463   638 raft_consensus.cc:3060] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.709254   619 raft_consensus.cc:2804] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.711858   619 raft_consensus.cc:493] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.712460   619 raft_consensus.cc:3060] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.709264   529 consensus_queue.cc:237] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.715463   638 raft_consensus.cc:515] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.683171   629 raft_consensus.cc:493] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.720752   629 raft_consensus.cc:3060] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.725327   638 leader_election.cc:290] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.726620   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.726455   319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
W20260709 13:10:39.728430   436 leader_election.cc:343] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.729209   390 raft_consensus.cc:3060] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.745083   390 raft_consensus.cc:2468] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:39.746876   435 leader_election.cc:304] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.747889   638 raft_consensus.cc:2804] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.748466   638 raft_consensus.cc:697] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:39.749413   638 consensus_queue.cc:237] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.750797   617 tablet_bootstrap.cc:492] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.755309   617 ts_tablet_manager.cc:1403] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.002s
I20260709 13:10:39.757042   618 tablet_bootstrap.cc:492] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.757654   618 ts_tablet_manager.cc:1403] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.087s	user 0.031s	sys 0.037s
I20260709 13:10:39.758248   617 raft_consensus.cc:359] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.760088   619 raft_consensus.cc:515] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.732439   629 raft_consensus.cc:515] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.761893   617 raft_consensus.cc:385] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.762383   617 raft_consensus.cc:740] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.763123   617 consensus_queue.cc:260] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.765070   321 raft_consensus.cc:2393] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate 5592f0c3492e4e11b54a6f21387bef95 in this term.
I20260709 13:10:39.765537   617 ts_tablet_manager.cc:1434] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:10:39.765278 32684 catalog_manager.cc:5697] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.766522   617 tablet_bootstrap.cc:492] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: Bootstrap starting.
I20260709 13:10:39.767417   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:39.768043   468 raft_consensus.cc:3060] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.760978   618 raft_consensus.cc:359] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.770841   618 raft_consensus.cc:385] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.771148   618 raft_consensus.cc:740] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.771863   618 consensus_queue.cc:260] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.773327   617 tablet_bootstrap.cc:654] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.773649   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.781500   390 raft_consensus.cc:2393] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5592f0c3492e4e11b54a6f21387bef95 in current term 1: Already voted for candidate 5f9b867a960045008f271fe1fc3a447d in this term.
I20260709 13:10:39.782263   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:39.783053   321 raft_consensus.cc:2393] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f9b867a960045008f271fe1fc3a447d in current term 1: Already voted for candidate 5592f0c3492e4e11b54a6f21387bef95 in this term.
I20260709 13:10:39.774142   618 ts_tablet_manager.cc:1434] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:10:39.786571   618 tablet_bootstrap.cc:492] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.775800   470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:39.787654 32686 catalog_manager.cc:5697] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:39.789317   359 leader_election.cc:304] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d; no voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:39.790205   617 tablet_bootstrap.cc:492] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: No bootstrap required, opened a new log
I20260709 13:10:39.790387   640 raft_consensus.cc:2749] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:39.793169   618 tablet_bootstrap.cc:654] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.797310   617 ts_tablet_manager.cc:1403] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.000s
I20260709 13:10:39.798264   529 raft_consensus.cc:493] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.799026   529 raft_consensus.cc:515] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.800556   617 raft_consensus.cc:359] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.777426   629 leader_election.cc:290] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.777885   468 raft_consensus.cc:2468] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 1.
I20260709 13:10:39.774401   619 leader_election.cc:290] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.801366   617 raft_consensus.cc:385] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.801900   617 raft_consensus.cc:740] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.802578   626 raft_consensus.cc:493] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.802613   617 consensus_queue.cc:260] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.803166   626 raft_consensus.cc:515] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.805171   626 leader_election.cc:290] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.805794   617 ts_tablet_manager.cc:1434] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:39.806305 32752 leader_election.cc:304] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:39.806906   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.808444   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.809329   468 raft_consensus.cc:2393] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5592f0c3492e4e11b54a6f21387bef95 in current term 1: Already voted for candidate 5f9b867a960045008f271fe1fc3a447d in this term.
I20260709 13:10:39.810752   626 raft_consensus.cc:2804] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.811887   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.813706 32754 leader_election.cc:304] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95; no voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:39.815088   619 raft_consensus.cc:493] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.807722   390 raft_consensus.cc:2393] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5592f0c3492e4e11b54a6f21387bef95 in current term 1: Already voted for candidate d0e9408b380e414dbe5daf286b6a0a41 in this term.
I20260709 13:10:39.815652   619 raft_consensus.cc:515] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.817361   629 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.817962   619 leader_election.cc:290] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.818734   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.819574   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.820374   468 raft_consensus.cc:2468] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 0.
W20260709 13:10:39.820833   362 leader_election.cc:343] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:39.822094   359 leader_election.cc:304] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.823020   619 raft_consensus.cc:2804] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.823390   619 raft_consensus.cc:493] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.823277   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41" is_pre_election: true
I20260709 13:10:39.823827   619 raft_consensus.cc:3060] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.824936 32752 leader_election.cc:304] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95; no voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:39.826757   655 raft_consensus.cc:3060] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.820659   618 tablet_bootstrap.cc:492] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.829628   618 ts_tablet_manager.cc:1403] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.012s
I20260709 13:10:39.831246   529 leader_election.cc:290] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.833899   619 raft_consensus.cc:515] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.834770   638 raft_consensus.cc:493] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.835664   626 raft_consensus.cc:697] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:39.835568   638 raft_consensus.cc:515] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.836701   626 consensus_queue.cc:237] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.838371   655 raft_consensus.cc:2749] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:39.838675   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95" is_pre_election: true
I20260709 13:10:39.840548   321 raft_consensus.cc:2468] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:39.841032   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d" is_pre_election: true
I20260709 13:10:39.841696   390 raft_consensus.cc:2468] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 0.
I20260709 13:10:39.841888   436 leader_election.cc:304] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.842279   629 raft_consensus.cc:2749] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:39.842796   609 raft_consensus.cc:2804] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:39.834257   618 raft_consensus.cc:359] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.843228   609 raft_consensus.cc:493] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.843451   618 raft_consensus.cc:385] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.843750   609 raft_consensus.cc:3060] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.844265   618 raft_consensus.cc:740] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.845214   618 consensus_queue.cc:260] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.846482   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:39.847091   321 raft_consensus.cc:3060] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.847957   618 ts_tablet_manager.cc:1434] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:10:39.849267   618 tablet_bootstrap.cc:492] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: Bootstrap starting.
I20260709 13:10:39.851468   609 raft_consensus.cc:515] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.854424   321 raft_consensus.cc:2468] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 1.
I20260709 13:10:39.856519   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:39.857344   468 raft_consensus.cc:3060] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.858405   319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:39.859243   319 raft_consensus.cc:3060] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.859472   618 tablet_bootstrap.cc:654] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:39.861439   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:39.862066   390 raft_consensus.cc:3060] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:39.868772   468 raft_consensus.cc:2468] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 1.
I20260709 13:10:39.843494   638 leader_election.cc:290] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.870628   619 leader_election.cc:290] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:39.871590   362 leader_election.cc:304] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:39.872586   618 tablet_bootstrap.cc:492] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: No bootstrap required, opened a new log
I20260709 13:10:39.872678   619 raft_consensus.cc:2804] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.873251   618 ts_tablet_manager.cc:1403] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.008s
I20260709 13:10:39.873379   619 raft_consensus.cc:697] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:39.874307   619 consensus_queue.cc:237] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.874748   626 consensus_queue.cc:1048] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:39.875965   618 raft_consensus.cc:359] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.876642   618 raft_consensus.cc:385] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:39.876953   618 raft_consensus.cc:740] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.877674   618 consensus_queue.cc:260] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.878760   319 raft_consensus.cc:2468] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:39.879730   618 ts_tablet_manager.cc:1434] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:39.880085   436 leader_election.cc:304] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:39.880070 32686 catalog_manager.cc:5697] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.881086   390 raft_consensus.cc:2468] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 1.
I20260709 13:10:39.881203   638 raft_consensus.cc:2804] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:39.883661   638 raft_consensus.cc:697] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:39.884788   638 consensus_queue.cc:237] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:39.858417   609 leader_election.cc:290] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 1 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:39.902172 32684 catalog_manager.cc:5697] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 1 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.906847 32683 catalog_manager.cc:5697] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:39.907606   626 consensus_queue.cc:1048] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:39.938603   468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:39.939236   468 raft_consensus.cc:606] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.955142   392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
mode: GRACEFUL
new_leader_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:41200
I20260709 13:10:39.955724   392 raft_consensus.cc:606] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Received request to transfer leadership to TS 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.961503   468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:39.962160   468 raft_consensus.cc:606] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5592f0c3492e4e11b54a6f21387bef95
I20260709 13:10:39.967829   319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:35376
I20260709 13:10:39.968370   319 raft_consensus.cc:606] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:39.973793   468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:39.974471   468 raft_consensus.cc:606] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:39.980669   319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:35376
I20260709 13:10:39.981236   319 raft_consensus.cc:606] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:39.986248   468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
mode: GRACEFUL
new_leader_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:50404
I20260709 13:10:39.986721   468 raft_consensus.cc:606] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Received request to transfer leadership to TS 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:39.991693   392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
mode: GRACEFUL
new_leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:41200
I20260709 13:10:39.992292   392 raft_consensus.cc:606] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Received request to transfer leadership to TS d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:40.036067   609 consensus_queue.cc:1048] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.041980   609 consensus_queue.cc:1048] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.043376   609 raft_consensus.cc:993] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:40.043985   644 raft_consensus.cc:1081] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:40.047703   321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:40.048261   321 raft_consensus.cc:493] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.048623   321 raft_consensus.cc:3060] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.052682   644 raft_consensus.cc:993] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:40.053812   609 consensus_queue.cc:1048] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:40.053244   644 raft_consensus.cc:1081] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:40.060284   321 raft_consensus.cc:515] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.063385   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:40.063555   321 leader_election.cc:290] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.063767   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2129ab623d475783bc6781676dbc7a" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.064065   390 raft_consensus.cc:3060] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.064371   468 raft_consensus.cc:3055] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.064777   468 raft_consensus.cc:740] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:40.063197   644 consensus_queue.cc:1048] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:40.065868   468 consensus_queue.cc:260] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.067016   468 raft_consensus.cc:3060] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.063109   319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:40.073060   319 raft_consensus.cc:493] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.073508   319 raft_consensus.cc:3060] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.074445   468 raft_consensus.cc:2468] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:40.076134 32752 leader_election.cc:304] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:40.077102   626 raft_consensus.cc:2804] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.076493   390 raft_consensus.cc:2468] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:40.077590   626 raft_consensus.cc:697] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:40.078600   626 consensus_queue.cc:237] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.079458   392 raft_consensus.cc:1240] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Rejecting Update request from peer d0e9408b380e414dbe5daf286b6a0a41 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:10:40.081322   319 raft_consensus.cc:515] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.086841   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.087476   468 raft_consensus.cc:3055] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.087567   392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb5083bb35f41eb906f8c0cc6821424" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:40.087842   468 raft_consensus.cc:740] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:40.088831   319 leader_election.cc:290] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.088943   468 consensus_queue.cc:260] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.089989   468 raft_consensus.cc:3060] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.096227 32683 catalog_manager.cc:5697] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.097713   468 raft_consensus.cc:2468] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:40.099009 32752 leader_election.cc:304] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 5f9b867a960045008f271fe1fc3a447d
I20260709 13:10:40.100235   626 raft_consensus.cc:2804] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.100682   626 raft_consensus.cc:697] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:40.100569   640 consensus_queue.cc:1048] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:40.101631   626 consensus_queue.cc:237] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.108335   619 raft_consensus.cc:993] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: : Instructing follower 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:40.109535   640 raft_consensus.cc:1081] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Signalling peer 5592f0c3492e4e11b54a6f21387bef95 to start an election
I20260709 13:10:40.112530   319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:10:40.113144   319 raft_consensus.cc:493] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.113668   319 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.115613 32686 catalog_manager.cc:5697] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.121219   319 raft_consensus.cc:515] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.128036   640 consensus_queue.cc:1048] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.126727   319 leader_election.cc:290] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Requested vote from peers 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.129791   396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:40.131317   396 raft_consensus.cc:3055] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.131657   396 raft_consensus.cc:740] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:40.132479   396 consensus_queue.cc:260] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.129442   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "5592f0c3492e4e11b54a6f21387bef95" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.137351   396 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.143822   396 raft_consensus.cc:2468] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5592f0c3492e4e11b54a6f21387bef95 in term 2.
I20260709 13:10:40.144930 32754 leader_election.cc:304] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:40.145764   626 raft_consensus.cc:2804] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.146291   626 raft_consensus.cc:697] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Leader. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:40.147028   626 consensus_queue.cc:237] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.154531 32683 catalog_manager.cc:5697] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 reported cstate change: term changed from 1 to 2, leader changed from 5f9b867a960045008f271fe1fc3a447d (127.20.66.130) to 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "5592f0c3492e4e11b54a6f21387bef95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.207589   319 raft_consensus.cc:3060] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:40.208081   626 consensus_queue.cc:1048] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.215816   642 consensus_queue.cc:1048] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.216614   629 raft_consensus.cc:993] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.217146   629 raft_consensus.cc:1081] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.218303   529 consensus_queue.cc:1048] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.221760   396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41184
I20260709 13:10:40.222473   396 raft_consensus.cc:493] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.222957   396 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.233398   609 consensus_queue.cc:1048] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:40.235347   396 raft_consensus.cc:515] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.243826   396 leader_election.cc:290] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.244598   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:40.244874   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.245643   321 raft_consensus.cc:3055] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.245680   468 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.246133   321 raft_consensus.cc:740] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:40.247021   321 consensus_queue.cc:260] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.248176   321 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.248989   642 raft_consensus.cc:993] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.249549   642 raft_consensus.cc:1081] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.251730   392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41194
I20260709 13:10:40.252467   392 raft_consensus.cc:493] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.252923   392 raft_consensus.cc:3060] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.253269   468 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.254938   359 leader_election.cc:304] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:40.255867   640 raft_consensus.cc:2804] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.256337   640 raft_consensus.cc:697] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:40.257161   640 consensus_queue.cc:237] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.258771   321 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.263432   392 raft_consensus.cc:515] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.265646   392 leader_election.cc:290] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.266511   321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:40.267174   321 raft_consensus.cc:3060] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.267273   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b913f3515b545c88b5298fb10c97a6c" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.268007   468 raft_consensus.cc:3055] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.268373   468 raft_consensus.cc:740] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:40.269274   468 consensus_queue.cc:260] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.269744 32684 catalog_manager.cc:5697] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.270612   468 raft_consensus.cc:3060] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.274773   321 raft_consensus.cc:2468] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.276067   362 leader_election.cc:304] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:40.276762   640 raft_consensus.cc:2804] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.277252   640 raft_consensus.cc:697] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:40.277675   609 consensus_queue.cc:1048] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.278046   640 consensus_queue.cc:237] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.280510   468 raft_consensus.cc:2468] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.288789 32683 catalog_manager.cc:5697] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.293515   609 consensus_queue.cc:1048] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.304620   638 raft_consensus.cc:993] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.305193   609 raft_consensus.cc:1081] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.306849   396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41194
I20260709 13:10:40.307484   396 raft_consensus.cc:493] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.307801   396 raft_consensus.cc:3060] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.312892   396 raft_consensus.cc:515] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.314505   396 leader_election.cc:290] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.315335   319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:40.315711   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dd0d5bcb96486d96ad9152c34bbcfc" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.316018   319 raft_consensus.cc:3060] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.316231   468 raft_consensus.cc:3055] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.316500   468 raft_consensus.cc:740] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:40.317194   468 consensus_queue.cc:260] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.318112   468 raft_consensus.cc:3060] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.321321   319 raft_consensus.cc:2468] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.322517   362 leader_election.cc:304] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, 5f9b867a960045008f271fe1fc3a447d; no voters: 
I20260709 13:10:40.323230   640 raft_consensus.cc:2804] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.323666   640 raft_consensus.cc:697] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:40.324522   640 consensus_queue.cc:237] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.324637   468 raft_consensus.cc:2468] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.330528   640 consensus_queue.cc:1048] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.331909 32686 catalog_manager.cc:5697] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.337641   676 raft_consensus.cc:993] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: : Instructing follower d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:40.338171   676 raft_consensus.cc:1081] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Signalling peer d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:40.340466   468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:50394
I20260709 13:10:40.341140   468 raft_consensus.cc:493] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.341547   468 raft_consensus.cc:3060] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.343621   640 consensus_queue.cc:1048] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:40.349293   468 raft_consensus.cc:515] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.350637   470 raft_consensus.cc:1240] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Rejecting Update request from peer 5f9b867a960045008f271fe1fc3a447d for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:40.351732   468 leader_election.cc:290] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:40.353582   396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:40.354417   396 raft_consensus.cc:3055] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.354745   396 raft_consensus.cc:740] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:40.354786   640 consensus_queue.cc:1059] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:40.355078   319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48f09b3a8da74c6d914203d040e3b786" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:40.356364   319 raft_consensus.cc:3060] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.356045   396 consensus_queue.cc:260] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.357471   396 raft_consensus.cc:3060] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.363005   319 raft_consensus.cc:2468] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 2.
I20260709 13:10:40.364151   396 raft_consensus.cc:2468] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 2.
I20260709 13:10:40.364293   436 leader_election.cc:304] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:40.365218   609 raft_consensus.cc:2804] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.365733   609 raft_consensus.cc:697] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:40.366613   609 consensus_queue.cc:237] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.373723 32684 catalog_manager.cc:5697] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 1 to 2, leader changed from 5f9b867a960045008f271fe1fc3a447d (127.20.66.130) to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:40.379572   655 consensus_queue.cc:1048] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.391077   529 consensus_queue.cc:1048] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.399451   626 raft_consensus.cc:993] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: : Instructing follower 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.399952   529 raft_consensus.cc:1081] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Signalling peer 5f9b867a960045008f271fe1fc3a447d to start an election
I20260709 13:10:40.401592   396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
 from {username='slave'} at 127.0.0.1:41184
I20260709 13:10:40.402185   396 raft_consensus.cc:493] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.402590   396 raft_consensus.cc:3060] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.408004   396 raft_consensus.cc:515] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.410071   396 leader_election.cc:290] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131:42073)
I20260709 13:10:40.410856   319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:40.411430   468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fb913ccb2043d08d0dcd002bc8b70b" candidate_uuid: "5f9b867a960045008f271fe1fc3a447d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
I20260709 13:10:40.411577   319 raft_consensus.cc:3055] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:40.412093   319 raft_consensus.cc:740] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:40.412117   468 raft_consensus.cc:3060] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.413252   319 consensus_queue.cc:260] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.415179   319 raft_consensus.cc:3060] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.420046   468 raft_consensus.cc:2468] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.421530   359 leader_election.cc:304] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:40.422361   640 raft_consensus.cc:2804] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.422870   640 raft_consensus.cc:697] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Leader. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:40.423758   640 consensus_queue.cc:237] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:40.422849   319 raft_consensus.cc:2468] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b867a960045008f271fe1fc3a447d in term 2.
I20260709 13:10:40.431007 32686 catalog_manager.cc:5697] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d reported cstate change: term changed from 1 to 2, leader changed from 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129) to 5f9b867a960045008f271fe1fc3a447d (127.20.66.130). New cstate: current_term: 2 leader_uuid: "5f9b867a960045008f271fe1fc3a447d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.512789   468 raft_consensus.cc:1275] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.514233   529 consensus_queue.cc:1048] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.522323   396 raft_consensus.cc:3060] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.531167   396 raft_consensus.cc:1275] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.532405   626 consensus_queue.cc:1048] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.579777   396 raft_consensus.cc:1275] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.581187   626 consensus_queue.cc:1048] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.589463   468 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.598206   468 raft_consensus.cc:1275] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.599432   655 consensus_queue.cc:1048] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.621479   468 raft_consensus.cc:1275] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.622747   655 consensus_queue.cc:1048] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.632299   396 raft_consensus.cc:1275] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Refusing update from remote peer 5592f0c3492e4e11b54a6f21387bef95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.634011   655 consensus_queue.cc:1048] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:40.667433   319 raft_consensus.cc:1275] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.668766   619 consensus_queue.cc:1048] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.676293   468 raft_consensus.cc:1275] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.677704   640 consensus_queue.cc:1048] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.730029   319 raft_consensus.cc:1275] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.731748   640 consensus_queue.cc:1048] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.740936   468 raft_consensus.cc:1275] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.742482   619 consensus_queue.cc:1048] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:40.859189   396 raft_consensus.cc:1275] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.860648   609 consensus_queue.cc:1048] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.871392   319 raft_consensus.cc:1275] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.872721   689 consensus_queue.cc:1048] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.893224   468 raft_consensus.cc:1275] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.894582   640 consensus_queue.cc:1048] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.901968   319 raft_consensus.cc:1275] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.903946   675 consensus_queue.cc:1048] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.957296   468 raft_consensus.cc:1275] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.958494   675 consensus_queue.cc:1048] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.965770   319 raft_consensus.cc:1275] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Refusing update from remote peer 5f9b867a960045008f271fe1fc3a447d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.967576   676 consensus_queue.cc:1048] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:42.998481 32686 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48262:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:10:43.037807 32686 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48262:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:10:43.108826 20746 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:10:43.315353   395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
mode: GRACEFUL
new_leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:35184
I20260709 13:10:43.322144   750 tablet.cc:1724] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.342574   741 tablet.cc:1724] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.384101   758 tablet.cc:1724] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.386078   744 tablet.cc:1724] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.387166   749 tablet.cc:1724] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.390419   745 tablet.cc:1724] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.395589   755 tablet.cc:1724] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.407572   757 tablet.cc:1724] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.411043   754 tablet.cc:1724] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.414976   753 tablet.cc:1724] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.405217   395 raft_consensus.cc:606] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Received request to transfer leadership to TS d0e9408b380e414dbe5daf286b6a0a41
W20260709 13:10:43.436827   422 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:48f09b3a8da74c6d914203d040e3b786)
I20260709 13:10:43.455292   744 tablet.cc:1724] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:10:43.468217   422 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:48f09b3a8da74c6d914203d040e3b786)
W20260709 13:10:43.469429   422 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:48f09b3a8da74c6d914203d040e3b786)
I20260709 13:10:43.471199   744 tablet.cc:1724] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.479506   756 tablet.cc:1724] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.493780   757 tablet.cc:1724] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.514281   760 tablet.cc:1724] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:10:43.521166   359 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:42073, user_credentials={real_user=slave}} blocked reactor thread for 91916.6us
I20260709 13:10:43.526750   750 tablet.cc:1724] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.529254   745 tablet.cc:1724] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.531176   758 tablet.cc:1724] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.534099   741 tablet.cc:1724] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.547547   744 tablet.cc:1724] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.490618   749 tablet.cc:1724] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.550987   756 tablet.cc:1724] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.565698   754 tablet.cc:1724] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.587599   760 tablet.cc:1724] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.595336   753 tablet.cc:1724] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:10:43.613507   359 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:42073, user_credentials={real_user=slave}} blocked reactor thread for 58809.4us
I20260709 13:10:43.616106   318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
mode: GRACEFUL
new_leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:39398
I20260709 13:10:43.616824   318 raft_consensus.cc:606] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Received request to transfer leadership to TS d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:43.622642   754 tablet.cc:1724] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.621181 20746 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:10:43.627000 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:43.630652   758 tablet.cc:1724] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:43.637612   395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7"
dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
mode: GRACEFUL
new_leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:35184
I20260709 13:10:43.638444   395 raft_consensus.cc:606] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Received request to transfer leadership to TS d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:43.642591   318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1"
dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
mode: GRACEFUL
new_leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:39398
I20260709 13:10:43.643255   318 raft_consensus.cc:606] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Received request to transfer leadership to TS d0e9408b380e414dbe5daf286b6a0a41
I20260709 13:10:43.647951 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:43.648591 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:43.651149   766 raft_consensus.cc:993] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: : Instructing follower d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:43.651757   766 raft_consensus.cc:1081] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Signalling peer d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:43.653863   471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:50394
I20260709 13:10:43.654546   471 raft_consensus.cc:493] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:43.654927   471 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.657876 32683 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52: soft_deleted_table [id=cfee467534c34a54a685b71ebc6bc0d8] alter complete (version 1)
I20260709 13:10:43.662601   471 raft_consensus.cc:515] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:43.665686   318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:43.666419   318 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.669116   471 leader_election.cc:290] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:43.670262   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da00b8b1f394254bb3e4070096361c7" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:43.670900   390 raft_consensus.cc:3055] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:43.671224   390 raft_consensus.cc:740] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:43.672174   390 consensus_queue.cc:260] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:43.673776   390 raft_consensus.cc:3060] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.676584   318 raft_consensus.cc:2468] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:43.678004   436 leader_election.cc:304] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:43.679068   691 raft_consensus.cc:2804] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:43.679569   691 raft_consensus.cc:697] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:43.680486   390 raft_consensus.cc:2468] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:43.680579   691 consensus_queue.cc:237] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:43.689862 32684 catalog_manager.cc:5697] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 2 to 3, leader changed from 5f9b867a960045008f271fe1fc3a447d (127.20.66.130) to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:43.871809   675 raft_consensus.cc:993] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: : Instructing follower d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:43.872403   766 raft_consensus.cc:1081] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Signalling peer d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:43.873970   471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:50394
I20260709 13:10:43.874517   471 raft_consensus.cc:493] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:43.874903   471 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.880185   471 raft_consensus.cc:515] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:43.882548   471 leader_election.cc:290] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:43.882859   318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:43.883596   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f2face53ea443e6b104f8a4062cdd3f" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:43.883613   318 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.884321   390 raft_consensus.cc:3055] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:43.884613   390 raft_consensus.cc:740] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f9b867a960045008f271fe1fc3a447d, State: Running, Role: LEADER
I20260709 13:10:43.885360   390 consensus_queue.cc:260] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:43.886363   390 raft_consensus.cc:3060] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.890499   318 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:43.891824   436 leader_election.cc:304] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5592f0c3492e4e11b54a6f21387bef95, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:43.892536   691 raft_consensus.cc:2804] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:43.893020   691 raft_consensus.cc:697] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:43.894127   390 raft_consensus.cc:2468] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:43.893909   691 consensus_queue.cc:237] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:43.901218 32684 catalog_manager.cc:5697] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 2 to 3, leader changed from 5f9b867a960045008f271fe1fc3a447d (127.20.66.130) to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.026932   761 raft_consensus.cc:993] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: : Instructing follower d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:44.027498   731 raft_consensus.cc:1081] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Signalling peer d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:44.028934   471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:50388
I20260709 13:10:44.029395   471 raft_consensus.cc:493] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.029655   471 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.035370   471 raft_consensus.cc:515] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:44.037369   471 leader_election.cc:290] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:44.038295   318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:44.038309   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610388a9a21e472f910feae8f7b149b9" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:44.039093   318 raft_consensus.cc:3055] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:44.039243   390 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.039530   318 raft_consensus.cc:740] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:44.040431   318 consensus_queue.cc:260] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:44.041607   318 raft_consensus.cc:3060] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.045593   761 raft_consensus.cc:993] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: : Instructing follower d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:44.046141   731 raft_consensus.cc:1081] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Signalling peer d0e9408b380e414dbe5daf286b6a0a41 to start an election
I20260709 13:10:44.046375   390 raft_consensus.cc:2468] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:44.047586   435 leader_election.cc:304] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:44.047876   471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1"
dest_uuid: "d0e9408b380e414dbe5daf286b6a0a41"
 from {username='slave'} at 127.0.0.1:50388
I20260709 13:10:44.048485   691 raft_consensus.cc:2804] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:44.048501   471 raft_consensus.cc:493] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.049157   691 raft_consensus.cc:697] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:44.049234   471 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.049549   318 raft_consensus.cc:2468] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:44.050176   691 consensus_queue.cc:237] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:44.055428   471 raft_consensus.cc:515] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:44.057309 32684 catalog_manager.cc:5697] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 2 to 3, leader changed from 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129) to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.057732   471 leader_election.cc:290] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Requested vote from peers 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573), 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289)
I20260709 13:10:44.058741   390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f9b867a960045008f271fe1fc3a447d"
I20260709 13:10:44.059295   390 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.062156   318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f28674dd0a884252b8d11396c9d8c3e1" candidate_uuid: "d0e9408b380e414dbe5daf286b6a0a41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5592f0c3492e4e11b54a6f21387bef95"
I20260709 13:10:44.062801   318 raft_consensus.cc:3055] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:44.063123   318 raft_consensus.cc:740] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5592f0c3492e4e11b54a6f21387bef95, State: Running, Role: LEADER
I20260709 13:10:44.064114   318 consensus_queue.cc:260] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:44.065620   318 raft_consensus.cc:3060] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.070103   390 raft_consensus.cc:2468] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:44.071511   435 leader_election.cc:304] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b867a960045008f271fe1fc3a447d, d0e9408b380e414dbe5daf286b6a0a41; no voters: 
I20260709 13:10:44.072286   726 raft_consensus.cc:2804] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:44.074252   318 raft_consensus.cc:2468] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e9408b380e414dbe5daf286b6a0a41 in term 3.
I20260709 13:10:44.075488   726 raft_consensus.cc:697] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Becoming Leader. State: Replica: d0e9408b380e414dbe5daf286b6a0a41, State: Running, Role: LEADER
I20260709 13:10:44.076440   726 consensus_queue.cc:237] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } }
I20260709 13:10:44.085083   319 raft_consensus.cc:1275] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:44.085879 32684 catalog_manager.cc:5697] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 reported cstate change: term changed from 2 to 3, leader changed from 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129) to d0e9408b380e414dbe5daf286b6a0a41 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "d0e9408b380e414dbe5daf286b6a0a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e9408b380e414dbe5daf286b6a0a41" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.086458   726 consensus_queue.cc:1048] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:44.100966   390 raft_consensus.cc:1275] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:44.103477   772 consensus_queue.cc:1048] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:44.391465   319 raft_consensus.cc:1275] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:10:44.392982   772 consensus_queue.cc:1048] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:44.401399   390 raft_consensus.cc:1275] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:10:44.404284   726 consensus_queue.cc:1048] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:44.460454   319 raft_consensus.cc:1275] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:10:44.461623   726 consensus_queue.cc:1048] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:44.471737   390 raft_consensus.cc:1275] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:10:44.473040   772 consensus_queue.cc:1048] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:44.643525   319 raft_consensus.cc:1275] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:44.644666   772 consensus_queue.cc:1048] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592f0c3492e4e11b54a6f21387bef95" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:44.657471   390 raft_consensus.cc:1275] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Refusing update from remote peer d0e9408b380e414dbe5daf286b6a0a41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:44.658910   776 consensus_queue.cc:1048] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b867a960045008f271fe1fc3a447d" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:44.714525 20746 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:10:44.720435 20746 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:10:44.720906 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:44.725875 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:10:44.726285 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:46.358819 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
I20260709 13:10:46.388458 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:46.389422 20746 tablet_replica.cc:333] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.390265 20746 raft_consensus.cc:2243] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.390851 20746 raft_consensus.cc:2272] T 48f09b3a8da74c6d914203d040e3b786 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.393600 20746 tablet_replica.cc:333] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.394320 20746 raft_consensus.cc:2243] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.394824 20746 raft_consensus.cc:2272] T e1fb913ccb2043d08d0dcd002bc8b70b P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.397616 20746 tablet_replica.cc:333] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.398326 20746 raft_consensus.cc:2243] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.399293 20746 raft_consensus.cc:2272] T dfb5083bb35f41eb906f8c0cc6821424 P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.401777 20746 tablet_replica.cc:333] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.402577 20746 raft_consensus.cc:2243] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.403112 20746 raft_consensus.cc:2272] T 5f2face53ea443e6b104f8a4062cdd3f P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.405622 20746 tablet_replica.cc:333] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.406286 20746 raft_consensus.cc:2243] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.406756 20746 raft_consensus.cc:2272] T 610388a9a21e472f910feae8f7b149b9 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.409272 20746 tablet_replica.cc:333] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.409989 20746 raft_consensus.cc:2243] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.410990 20746 raft_consensus.cc:2272] T 6709838ded034f6599c94a98e115fbeb P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.413636 20746 tablet_replica.cc:333] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.414319 20746 raft_consensus.cc:2243] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.414836 20746 raft_consensus.cc:2272] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.417335 20746 tablet_replica.cc:333] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.418010 20746 raft_consensus.cc:2243] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.418493 20746 raft_consensus.cc:2272] T f28674dd0a884252b8d11396c9d8c3e1 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.421083 20746 tablet_replica.cc:333] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.421804 20746 raft_consensus.cc:2243] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.422302 20746 raft_consensus.cc:2272] T 5b913f3515b545c88b5298fb10c97a6c P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.424818 20746 tablet_replica.cc:333] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.425572 20746 raft_consensus.cc:2243] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.426569 20746 raft_consensus.cc:2272] T f514b957834b4abab3de4af238532008 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.429255 20746 tablet_replica.cc:333] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.429848 20746 raft_consensus.cc:2243] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.430356 20746 raft_consensus.cc:2272] T e242b10da2df46778fbbd4910a97808f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.432888 20746 tablet_replica.cc:333] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.433542 20746 raft_consensus.cc:2243] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.434507 20746 raft_consensus.cc:2272] T da123722312340b088054d80b503c16f P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.436933 20746 tablet_replica.cc:333] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.437582 20746 raft_consensus.cc:2243] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.438053 20746 raft_consensus.cc:2272] T 70f28fd9f6fa4c4799b20e357dc87813 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.440464 20746 tablet_replica.cc:333] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.441112 20746 raft_consensus.cc:2243] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.442059 20746 raft_consensus.cc:2272] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.444526 20746 tablet_replica.cc:333] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.445183 20746 raft_consensus.cc:2243] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.445642 20746 raft_consensus.cc:2272] T 9da00b8b1f394254bb3e4070096361c7 P 5592f0c3492e4e11b54a6f21387bef95 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.448124 20746 tablet_replica.cc:333] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.448704 20746 raft_consensus.cc:2243] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.449638 20746 raft_consensus.cc:2272] T fa2129ab623d475783bc6781676dbc7a P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.452704 20746 tablet_replica.cc:333] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.453504 20746 raft_consensus.cc:2243] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.454130 20746 raft_consensus.cc:2272] T 49cf29e468ff44aba1ab2ab8981ad02a P 5592f0c3492e4e11b54a6f21387bef95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.457572 20746 tablet_replica.cc:333] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95: stopping tablet replica
I20260709 13:10:46.458354 20746 raft_consensus.cc:2243] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.459136 20746 raft_consensus.cc:2272] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5592f0c3492e4e11b54a6f21387bef95 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.508630 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
I20260709 13:10:46.547526 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
I20260709 13:10:46.578512 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:46.579233 20746 tablet_replica.cc:333] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.579873 20746 raft_consensus.cc:2243] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.580358 20746 raft_consensus.cc:2272] T 48f09b3a8da74c6d914203d040e3b786 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.582635 20746 tablet_replica.cc:333] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.583362 20746 raft_consensus.cc:2243] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.584110 20746 raft_consensus.cc:2272] T e1fb913ccb2043d08d0dcd002bc8b70b P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.586112 20746 tablet_replica.cc:333] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.586634 20746 raft_consensus.cc:2243] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.587003 20746 raft_consensus.cc:2272] T dfb5083bb35f41eb906f8c0cc6821424 P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.588969 20746 tablet_replica.cc:333] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.589617 20746 raft_consensus.cc:2243] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.589982 20746 raft_consensus.cc:2272] T 5f2face53ea443e6b104f8a4062cdd3f P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.591787 20746 tablet_replica.cc:333] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.592290 20746 raft_consensus.cc:2243] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.592662 20746 raft_consensus.cc:2272] T fa2129ab623d475783bc6781676dbc7a P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.594538 20746 tablet_replica.cc:333] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.595038 20746 raft_consensus.cc:2243] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.595438 20746 raft_consensus.cc:2272] T 610388a9a21e472f910feae8f7b149b9 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.597316 20746 tablet_replica.cc:333] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.597795 20746 raft_consensus.cc:2243] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.598311 20746 raft_consensus.cc:2272] T 6709838ded034f6599c94a98e115fbeb P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.600440 20746 tablet_replica.cc:333] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.601014 20746 raft_consensus.cc:2243] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.601461 20746 raft_consensus.cc:2272] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.603549 20746 tablet_replica.cc:333] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.604074 20746 raft_consensus.cc:2243] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.604480 20746 raft_consensus.cc:2272] T 9da00b8b1f394254bb3e4070096361c7 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.606382 20746 tablet_replica.cc:333] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.606945 20746 raft_consensus.cc:2243] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.607384 20746 raft_consensus.cc:2272] T 49cf29e468ff44aba1ab2ab8981ad02a P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.609637 20746 tablet_replica.cc:333] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.610169 20746 raft_consensus.cc:2243] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.611007 20746 raft_consensus.cc:2272] T 5b913f3515b545c88b5298fb10c97a6c P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.613679 20746 tablet_replica.cc:333] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
W20260709 13:10:46.614512   435 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260709 13:10:46.614723 20746 raft_consensus.cc:2243] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.615906 20746 raft_consensus.cc:2272] T e242b10da2df46778fbbd4910a97808f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:46.617885   435 consensus_peers.cc:597] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Couldn't send request to peer 5f9b867a960045008f271fe1fc3a447d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.618630 20746 tablet_replica.cc:333] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.619238 20746 raft_consensus.cc:2243] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.619690 20746 raft_consensus.cc:2272] T f514b957834b4abab3de4af238532008 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.622027 20746 tablet_replica.cc:333] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.622545 20746 raft_consensus.cc:2243] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.623389 20746 raft_consensus.cc:2272] T 70f28fd9f6fa4c4799b20e357dc87813 P 5f9b867a960045008f271fe1fc3a447d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.625564 20746 tablet_replica.cc:333] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.626116 20746 raft_consensus.cc:2243] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.626528 20746 raft_consensus.cc:2272] T da123722312340b088054d80b503c16f P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.628861 20746 tablet_replica.cc:333] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.629406 20746 raft_consensus.cc:2243] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.630313 20746 raft_consensus.cc:2272] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.632423 20746 tablet_replica.cc:333] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.632915 20746 raft_consensus.cc:2243] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.633715 20746 raft_consensus.cc:2272] T c7dd0d5bcb96486d96ad9152c34bbcfc P 5f9b867a960045008f271fe1fc3a447d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.635810 20746 tablet_replica.cc:333] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d: stopping tablet replica
I20260709 13:10:46.636315 20746 raft_consensus.cc:2243] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.636713 20746 raft_consensus.cc:2272] T f28674dd0a884252b8d11396c9d8c3e1 P 5f9b867a960045008f271fe1fc3a447d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:46.648280   436 consensus_peers.cc:597] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Couldn't send request to peer 5592f0c3492e4e11b54a6f21387bef95. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:40573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:46.662760   435 consensus_peers.cc:597] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Couldn't send request to peer 5f9b867a960045008f271fe1fc3a447d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:46.671350   435 consensus_peers.cc:597] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Couldn't send request to peer 5f9b867a960045008f271fe1fc3a447d. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:41289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.682518 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
W20260709 13:10:46.698184   435 consensus_peers.cc:597] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Couldn't send request to peer 5f9b867a960045008f271fe1fc3a447d. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:41289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.719705 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:10:46.740710   435 consensus_peers.cc:597] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Couldn't send request to peer 5f9b867a960045008f271fe1fc3a447d. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:41289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.752417 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:46.753365 20746 tablet_replica.cc:333] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
W20260709 13:10:46.753947   436 consensus_peers.cc:597] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Couldn't send request to peer 5592f0c3492e4e11b54a6f21387bef95. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:40573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.754129 20746 raft_consensus.cc:2243] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.755079 20746 raft_consensus.cc:2272] T 48f09b3a8da74c6d914203d040e3b786 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.757352 20746 tablet_replica.cc:333] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.757972 20746 raft_consensus.cc:2243] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.758774 20746 raft_consensus.cc:2272] T 5f2face53ea443e6b104f8a4062cdd3f P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.760980 20746 tablet_replica.cc:333] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.761649 20746 raft_consensus.cc:2243] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.762138 20746 raft_consensus.cc:2272] T dfb5083bb35f41eb906f8c0cc6821424 P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.764057 20746 tablet_replica.cc:333] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.764554 20746 raft_consensus.cc:2243] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.764933 20746 raft_consensus.cc:2272] T e1fb913ccb2043d08d0dcd002bc8b70b P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.766901 20746 tablet_replica.cc:333] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.767431 20746 raft_consensus.cc:2243] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.768147 20746 raft_consensus.cc:2272] T 610388a9a21e472f910feae8f7b149b9 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.770347 20746 tablet_replica.cc:333] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.771124 20746 raft_consensus.cc:2243] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.771657 20746 raft_consensus.cc:2272] T 6709838ded034f6599c94a98e115fbeb P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.774768 20746 tablet_replica.cc:333] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.775614 20746 raft_consensus.cc:2243] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.776280 20746 raft_consensus.cc:2272] T da123722312340b088054d80b503c16f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:46.776467   435 consensus_peers.cc:597] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5f9b867a960045008f271fe1fc3a447d (127.20.66.130:41289): Couldn't send request to peer 5f9b867a960045008f271fe1fc3a447d. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:41289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.779047 20746 tablet_replica.cc:333] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.779958 20746 raft_consensus.cc:2243] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:46.780232   436 consensus_peers.cc:597] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 -> Peer 5592f0c3492e4e11b54a6f21387bef95 (127.20.66.129:40573): Couldn't send request to peer 5592f0c3492e4e11b54a6f21387bef95. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:40573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:46.780474 20746 raft_consensus.cc:2272] T 70f28fd9f6fa4c4799b20e357dc87813 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.783020 20746 tablet_replica.cc:333] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.783646 20746 raft_consensus.cc:2243] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.784106 20746 raft_consensus.cc:2272] T 5b913f3515b545c88b5298fb10c97a6c P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.785997 20746 tablet_replica.cc:333] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.786545 20746 raft_consensus.cc:2243] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.786983 20746 raft_consensus.cc:2272] T e242b10da2df46778fbbd4910a97808f P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.788970 20746 tablet_replica.cc:333] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.789525 20746 raft_consensus.cc:2243] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.789919 20746 raft_consensus.cc:2272] T f514b957834b4abab3de4af238532008 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.791747 20746 tablet_replica.cc:333] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.792263 20746 raft_consensus.cc:2243] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.792683 20746 raft_consensus.cc:2272] T fa2129ab623d475783bc6781676dbc7a P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.794557 20746 tablet_replica.cc:333] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.795077 20746 raft_consensus.cc:2243] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.795892 20746 raft_consensus.cc:2272] T 49cf29e468ff44aba1ab2ab8981ad02a P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.797890 20746 tablet_replica.cc:333] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.798439 20746 raft_consensus.cc:2243] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.799269 20746 raft_consensus.cc:2272] T 9da00b8b1f394254bb3e4070096361c7 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.801867 20746 tablet_replica.cc:333] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.802345 20746 raft_consensus.cc:2243] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.802714 20746 raft_consensus.cc:2272] T 4e124b2bdf5a4807a77b0b9a4df0c3a0 P d0e9408b380e414dbe5daf286b6a0a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.804714 20746 tablet_replica.cc:333] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.805225 20746 raft_consensus.cc:2243] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.805605 20746 raft_consensus.cc:2272] T c7dd0d5bcb96486d96ad9152c34bbcfc P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.807379 20746 tablet_replica.cc:333] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.807827 20746 raft_consensus.cc:2243] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:46.808506 20746 raft_consensus.cc:2272] T f28674dd0a884252b8d11396c9d8c3e1 P d0e9408b380e414dbe5daf286b6a0a41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.810403 20746 tablet_replica.cc:333] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41: stopping tablet replica
I20260709 13:10:46.810834 20746 raft_consensus.cc:2243] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.811234 20746 raft_consensus.cc:2272] T 9b8bb5f5852b4bd89fa8abeb60d6ad8e P d0e9408b380e414dbe5daf286b6a0a41 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:47.254346   492 debug-util.cc:398] Leaking SignalData structure 0x7b08001fca80 after lost signal to thread 20749
W20260709 13:10:47.255517   492 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9a60 after lost signal to thread 32721
W20260709 13:10:47.847513 20746 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 492)
W20260709 13:10:48.848203 20746 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 492)
W20260709 13:10:49.848809 20746 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 492)
I20260709 13:10:50.166950 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:10:50.215469 20746 master.cc:562] Master@127.20.66.190:40127 shutting down...
I20260709 13:10:50.248397 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.249203 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.249595 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P 14e6a3ba75c4405e80e3a8840d54dd52: stopping tablet replica
I20260709 13:10:50.273360 20746 master.cc:584] Master@127.20.66.190:40127 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (16566 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:10:50.338279 20746 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.20.66.190:46621
I20260709 13:10:50.339520 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:50.345978   798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.346890   800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.347170   797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.349056 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:50.350240 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:50.350473 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:50.350592 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602650350580 us; error 0 us; skew 500 ppm
I20260709 13:10:50.351181 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:50.354314 20746 webserver.cc:533] Webserver started at http://127.20.66.190:36727/ using document root <none> and password file <none>
I20260709 13:10:50.354856 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:50.355021 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:50.355257 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:50.356699 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/master-0-root/instance:
uuid: "c7834b353523407b8b642f7929985bcb"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.362536 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:10:50.366693   805 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.367718 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:10:50.368149 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/master-0-root
uuid: "c7834b353523407b8b642f7929985bcb"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.368597 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:50.379462 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:50.381388 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:50.433815 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.190:46621
I20260709 13:10:50.433928   856 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.190:46621 every 8 connection(s)
I20260709 13:10:50.438967   857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.453963   857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb: Bootstrap starting.
I20260709 13:10:50.460709   857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.466547   857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb: No bootstrap required, opened a new log
I20260709 13:10:50.469370   857 raft_consensus.cc:359] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7834b353523407b8b642f7929985bcb" member_type: VOTER }
I20260709 13:10:50.469825   857 raft_consensus.cc:385] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.470021   857 raft_consensus.cc:740] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7834b353523407b8b642f7929985bcb, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.470755   857 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7834b353523407b8b642f7929985bcb" member_type: VOTER }
I20260709 13:10:50.471273   857 raft_consensus.cc:399] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:50.471480   857 raft_consensus.cc:493] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:50.471858   857 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.479661   857 raft_consensus.cc:515] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7834b353523407b8b642f7929985bcb" member_type: VOTER }
I20260709 13:10:50.480309   857 leader_election.cc:304] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7834b353523407b8b642f7929985bcb; no voters: 
I20260709 13:10:50.481657   857 leader_election.cc:290] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:50.482111   860 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.483753   860 raft_consensus.cc:697] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 1 LEADER]: Becoming Leader. State: Replica: c7834b353523407b8b642f7929985bcb, State: Running, Role: LEADER
I20260709 13:10:50.484593   860 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7834b353523407b8b642f7929985bcb" member_type: VOTER }
I20260709 13:10:50.485225   857 sys_catalog.cc:565] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:50.490711   862 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7834b353523407b8b642f7929985bcb. Latest consensus state: current_term: 1 leader_uuid: "c7834b353523407b8b642f7929985bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7834b353523407b8b642f7929985bcb" member_type: VOTER } }
I20260709 13:10:50.491847   862 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:50.494506   861 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7834b353523407b8b642f7929985bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7834b353523407b8b642f7929985bcb" member_type: VOTER } }
I20260709 13:10:50.495445   861 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:50.497560   869 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:50.506312   869 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:50.507570 20746 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:50.518038   869 catalog_manager.cc:1383] Generated new cluster ID: b41971e4076e4b75bbb7604bfe65624e
I20260709 13:10:50.518366   869 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:50.541543   869 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:50.543181   869 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:50.563244   869 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb: Generated new TSK 0
I20260709 13:10:50.564016   869 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:50.575223 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:50.582600   878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.584235   879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.588174 20746 server_base.cc:1061] running on GCE node
W20260709 13:10:50.588497   881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.591784 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:50.592108 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:50.592312 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602650592293 us; error 0 us; skew 500 ppm
I20260709 13:10:50.592934 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:50.601861 20746 webserver.cc:533] Webserver started at http://127.20.66.129:38835/ using document root <none> and password file <none>
I20260709 13:10:50.602528 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:50.602761 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:50.603065 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:50.604403 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-0-root/instance:
uuid: "f082b9e30dac407eb4b01fcd8ff50941"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.610683 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:10:50.615063   886 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.616271 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:10:50.616640 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-0-root
uuid: "f082b9e30dac407eb4b01fcd8ff50941"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.616963 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:50.634112 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:50.635519 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:50.637321 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:50.640023 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:50.640275 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.640532 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:50.640726 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.690357 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.129:32961
I20260709 13:10:50.690477   948 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.129:32961 every 8 connection(s)
I20260709 13:10:50.696497 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:50.705961   953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.707655   954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.710386 20746 server_base.cc:1061] running on GCE node
W20260709 13:10:50.710484   956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.712581 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:50.712811 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:50.713083 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602650713059 us; error 0 us; skew 500 ppm
I20260709 13:10:50.713804 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:50.717221 20746 webserver.cc:533] Webserver started at http://127.20.66.130:44897/ using document root <none> and password file <none>
I20260709 13:10:50.717415   949 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46621
I20260709 13:10:50.717957   949 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:50.717985 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:50.718561 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:50.718968 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:50.719379   949 heartbeater.cc:507] Master 127.20.66.190:46621 requested a full tablet report, sending...
I20260709 13:10:50.720883 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-1-root/instance:
uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.722453   822 ts_manager.cc:194] Registered new tserver with Master: f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961)
I20260709 13:10:50.725564   822 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48406
I20260709 13:10:50.728489 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:50.733279   961 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.734220 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:10:50.734614 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-1-root
uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.734943 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:50.750506 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:50.751884 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:50.753722 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:50.756616 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:50.756827 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.757151 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:50.757341 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.830416 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.130:38267
I20260709 13:10:50.830614  1023 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.130:38267 every 8 connection(s)
I20260709 13:10:50.836138 20746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:50.845144  1027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.846807  1028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.849462  1030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.850322 20746 server_base.cc:1061] running on GCE node
I20260709 13:10:50.852026 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:50.852314 20746 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:50.852133  1024 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46621
I20260709 13:10:50.852703 20746 hybrid_clock.cc:648] HybridClock initialized: now 1783602650852679 us; error 0 us; skew 500 ppm
I20260709 13:10:50.853166  1024 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:50.853618 20746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:50.854169  1024 heartbeater.cc:507] Master 127.20.66.190:46621 requested a full tablet report, sending...
I20260709 13:10:50.856786 20746 webserver.cc:533] Webserver started at http://127.20.66.131:33617/ using document root <none> and password file <none>
I20260709 13:10:50.856989   822 ts_manager.cc:194] Registered new tserver with Master: 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:50.857482 20746 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:50.857800 20746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:50.858093 20746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:50.858776   822 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48410
I20260709 13:10:50.859650 20746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-2-root/instance:
uuid: "10abbd8bcc06430894c465db6ba1c1e3"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.866186 20746 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:50.871188  1035 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.872289 20746 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:10:50.872623 20746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-2-root
uuid: "10abbd8bcc06430894c465db6ba1c1e3"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-rgrd"
I20260709 13:10:50.872942 20746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-tasku7t2HP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602258489487-20746-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:50.894023 20746 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:50.895447 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:50.897235 20746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:50.899938 20746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:50.900197 20746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.900489 20746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:50.900689 20746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.950840 20746 rpc_server.cc:307] RPC server started. Bound to: 127.20.66.131:39805
I20260709 13:10:50.950994  1097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.66.131:39805 every 8 connection(s)
I20260709 13:10:50.967834  1098 heartbeater.cc:344] Connected to a master server at 127.20.66.190:46621
I20260709 13:10:50.968401  1098 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:50.969322  1098 heartbeater.cc:507] Master 127.20.66.190:46621 requested a full tablet report, sending...
I20260709 13:10:50.972000   822 ts_manager.cc:194] Registered new tserver with Master: 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:50.972158 20746 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017810325s
I20260709 13:10:50.973762   822 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48414
I20260709 13:10:51.002305   822 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48430:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:51.005368   822 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:51.087635   914 tablet_service.cc:1511] Processing CreateTablet for tablet 360a901b5e1947db83ce7fd9190c7ee6 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:51.089501   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 360a901b5e1947db83ce7fd9190c7ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.091073   913 tablet_service.cc:1511] Processing CreateTablet for tablet 684feeef469040c1bce9199d5510e30d (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:51.092944   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684feeef469040c1bce9199d5510e30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.098491   912 tablet_service.cc:1511] Processing CreateTablet for tablet 304180aa5ffa4d76865a1867048b6ea1 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:51.101817   911 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3fde5a7ea7497f9658945d8d794518 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:51.104681   910 tablet_service.cc:1511] Processing CreateTablet for tablet 810e75e22d7a44158f99e29bf93557b0 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:51.100448   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304180aa5ffa4d76865a1867048b6ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.107416   909 tablet_service.cc:1511] Processing CreateTablet for tablet 7c489bd1e018427485257e79220aef24 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:51.108670   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3fde5a7ea7497f9658945d8d794518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.110397   910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 810e75e22d7a44158f99e29bf93557b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.114694   908 tablet_service.cc:1511] Processing CreateTablet for tablet 67b77525557d47208b1a7ee595db73dd (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:51.116601   909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c489bd1e018427485257e79220aef24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.122123   989 tablet_service.cc:1511] Processing CreateTablet for tablet 360a901b5e1947db83ce7fd9190c7ee6 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:51.123481   908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b77525557d47208b1a7ee595db73dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.123188   986 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3fde5a7ea7497f9658945d8d794518 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:51.126376   987 tablet_service.cc:1511] Processing CreateTablet for tablet 304180aa5ffa4d76865a1867048b6ea1 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:51.128432   981 tablet_service.cc:1511] Processing CreateTablet for tablet 9ffae32838b94fcab46259eec44d7b28 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:51.122408   907 tablet_service.cc:1511] Processing CreateTablet for tablet 885c11b7479d48968b92acefaf22c6f4 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:51.128944   982 tablet_service.cc:1511] Processing CreateTablet for tablet 885c11b7479d48968b92acefaf22c6f4 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:51.134872   907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 885c11b7479d48968b92acefaf22c6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.130167   984 tablet_service.cc:1511] Processing CreateTablet for tablet 7c489bd1e018427485257e79220aef24 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:51.137063   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 360a901b5e1947db83ce7fd9190c7ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.131181   985 tablet_service.cc:1511] Processing CreateTablet for tablet 810e75e22d7a44158f99e29bf93557b0 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:51.138401   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 810e75e22d7a44158f99e29bf93557b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.132265   983 tablet_service.cc:1511] Processing CreateTablet for tablet 67b77525557d47208b1a7ee595db73dd (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:51.141422   981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffae32838b94fcab46259eec44d7b28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.142416   983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b77525557d47208b1a7ee595db73dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.145442   986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3fde5a7ea7497f9658945d8d794518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.133564   988 tablet_service.cc:1511] Processing CreateTablet for tablet 684feeef469040c1bce9199d5510e30d (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:51.146193   987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304180aa5ffa4d76865a1867048b6ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.133574   906 tablet_service.cc:1511] Processing CreateTablet for tablet 9ffae32838b94fcab46259eec44d7b28 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:51.148722   984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c489bd1e018427485257e79220aef24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.150988   982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 885c11b7479d48968b92acefaf22c6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.153250   906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffae32838b94fcab46259eec44d7b28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.157507   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684feeef469040c1bce9199d5510e30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.171561  1063 tablet_service.cc:1511] Processing CreateTablet for tablet 360a901b5e1947db83ce7fd9190c7ee6 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:51.173954  1058 tablet_service.cc:1511] Processing CreateTablet for tablet 7c489bd1e018427485257e79220aef24 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:51.174315  1062 tablet_service.cc:1511] Processing CreateTablet for tablet 684feeef469040c1bce9199d5510e30d (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:51.175827  1062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684feeef469040c1bce9199d5510e30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.176405  1057 tablet_service.cc:1511] Processing CreateTablet for tablet 67b77525557d47208b1a7ee595db73dd (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:51.178021  1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b77525557d47208b1a7ee595db73dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.178223  1060 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3fde5a7ea7497f9658945d8d794518 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:51.179665  1060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3fde5a7ea7497f9658945d8d794518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.182296  1059 tablet_service.cc:1511] Processing CreateTablet for tablet 810e75e22d7a44158f99e29bf93557b0 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:51.185081  1063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 360a901b5e1947db83ce7fd9190c7ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.184530  1061 tablet_service.cc:1511] Processing CreateTablet for tablet 304180aa5ffa4d76865a1867048b6ea1 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:51.189172  1056 tablet_service.cc:1511] Processing CreateTablet for tablet 885c11b7479d48968b92acefaf22c6f4 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:51.190768  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 885c11b7479d48968b92acefaf22c6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.194702  1055 tablet_service.cc:1511] Processing CreateTablet for tablet 9ffae32838b94fcab46259eec44d7b28 (DEFAULT_TABLE table=test-workload [id=9b86e3375c474856a1dcb90155279040]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:51.196969  1061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304180aa5ffa4d76865a1867048b6ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.199075  1059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 810e75e22d7a44158f99e29bf93557b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.204258  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffae32838b94fcab46259eec44d7b28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.209635  1058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c489bd1e018427485257e79220aef24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.233740  1114 tablet_bootstrap.cc:492] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.247788  1114 tablet_bootstrap.cc:654] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.248767  1115 tablet_bootstrap.cc:492] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.259919  1115 tablet_bootstrap.cc:654] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.298951  1114 tablet_bootstrap.cc:492] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.299597  1114 ts_tablet_manager.cc:1403] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.003s
I20260709 13:10:51.321173  1113 tablet_bootstrap.cc:492] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.345793  1114 raft_consensus.cc:359] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.346695  1114 raft_consensus.cc:385] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.347028  1114 raft_consensus.cc:740] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.348155  1114 consensus_queue.cc:260] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.406903  1024 heartbeater.cc:499] Master 127.20.66.190:46621 was elected leader, sending a full tablet report...
I20260709 13:10:51.407244  1115 tablet_bootstrap.cc:492] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.407934  1115 ts_tablet_manager.cc:1403] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.160s	user 0.037s	sys 0.050s
I20260709 13:10:51.408963  1114 ts_tablet_manager.cc:1434] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.109s	user 0.019s	sys 0.011s
I20260709 13:10:51.410230  1114 tablet_bootstrap.cc:492] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.411423  1115 raft_consensus.cc:359] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.412318  1115 raft_consensus.cc:385] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.412643  1115 raft_consensus.cc:740] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.413669  1115 consensus_queue.cc:260] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.419132  1114 tablet_bootstrap.cc:654] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.420207  1113 tablet_bootstrap.cc:654] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.420640  1098 heartbeater.cc:499] Master 127.20.66.190:46621 was elected leader, sending a full tablet report...
I20260709 13:10:51.427982  1115 ts_tablet_manager.cc:1434] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.005s
I20260709 13:10:51.429718  1115 tablet_bootstrap.cc:492] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.437855  1113 tablet_bootstrap.cc:492] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.438372  1114 tablet_bootstrap.cc:492] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.438647  1113 ts_tablet_manager.cc:1403] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.118s	user 0.015s	sys 0.004s
I20260709 13:10:51.439005  1114 ts_tablet_manager.cc:1403] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.029s	user 0.026s	sys 0.000s
I20260709 13:10:51.441495  1115 tablet_bootstrap.cc:654] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.442454  1113 raft_consensus.cc:359] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.442628  1114 raft_consensus.cc:359] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.443398  1113 raft_consensus.cc:385] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.443511  1114 raft_consensus.cc:385] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.443799  1113 raft_consensus.cc:740] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.443970  1114 raft_consensus.cc:740] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.444826  1113 consensus_queue.cc:260] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.444964  1114 consensus_queue.cc:260] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.447827  1114 ts_tablet_manager.cc:1434] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:51.449344  1114 tablet_bootstrap.cc:492] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.451676  1113 ts_tablet_manager.cc:1434] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.003s
I20260709 13:10:51.452623   949 heartbeater.cc:499] Master 127.20.66.190:46621 was elected leader, sending a full tablet report...
W20260709 13:10:51.455730  1099 tablet.cc:2406] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:51.461973  1114 tablet_bootstrap.cc:654] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.464550  1113 tablet_bootstrap.cc:492] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.480340  1115 tablet_bootstrap.cc:492] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.481220  1115 ts_tablet_manager.cc:1403] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.023s
I20260709 13:10:51.481431  1113 tablet_bootstrap.cc:654] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.485219  1115 raft_consensus.cc:359] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.486276  1115 raft_consensus.cc:385] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.486866  1115 raft_consensus.cc:740] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.487685  1114 tablet_bootstrap.cc:492] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.488344  1114 ts_tablet_manager.cc:1403] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.015s
I20260709 13:10:51.488481  1115 consensus_queue.cc:260] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.491714  1114 raft_consensus.cc:359] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.494302  1114 raft_consensus.cc:385] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.494648  1114 raft_consensus.cc:740] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.495113  1115 ts_tablet_manager.cc:1434] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 13:10:51.495597  1114 consensus_queue.cc:260] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.498302  1114 ts_tablet_manager.cc:1434] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
W20260709 13:10:51.498349   870 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:51.499677  1114 tablet_bootstrap.cc:492] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.503422  1113 tablet_bootstrap.cc:492] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.504151  1113 ts_tablet_manager.cc:1403] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.009s
I20260709 13:10:51.507758  1114 tablet_bootstrap.cc:654] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.507718  1113 raft_consensus.cc:359] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.508865  1113 raft_consensus.cc:385] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.499679  1115 tablet_bootstrap.cc:492] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.509275  1113 raft_consensus.cc:740] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.510221  1113 consensus_queue.cc:260] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.512434  1118 raft_consensus.cc:493] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.512694  1113 ts_tablet_manager.cc:1434] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:51.513100  1118 raft_consensus.cc:515] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.513880  1113 tablet_bootstrap.cc:492] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.519438  1115 tablet_bootstrap.cc:654] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.522244  1113 tablet_bootstrap.cc:654] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.527118  1118 leader_election.cc:290] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.535396  1115 tablet_bootstrap.cc:492] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.536568  1115 ts_tablet_manager.cc:1403] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.014s
I20260709 13:10:51.541482  1115 raft_consensus.cc:359] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.542735  1115 raft_consensus.cc:385] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.543222  1115 raft_consensus.cc:740] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.544684  1115 consensus_queue.cc:260] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.548218  1115 ts_tablet_manager.cc:1434] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:10:51.550443  1115 tablet_bootstrap.cc:492] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.562618  1115 tablet_bootstrap.cc:654] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.569108  1114 tablet_bootstrap.cc:492] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.569975  1114 ts_tablet_manager.cc:1403] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.026s
I20260709 13:10:51.574371  1114 raft_consensus.cc:359] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.575516  1114 raft_consensus.cc:385] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.575847  1118 raft_consensus.cc:493] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.575960  1114 raft_consensus.cc:740] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.576871  1118 raft_consensus.cc:515] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.577623  1114 consensus_queue.cc:260] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.580171  1118 leader_election.cc:290] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.599825   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.600188   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.600765   924 raft_consensus.cc:2468] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
I20260709 13:10:51.601733  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:51.602633  1073 raft_consensus.cc:2468] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
W20260709 13:10:51.602821   962 leader_election.cc:343] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.602941  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:51.603749  1113 tablet_bootstrap.cc:492] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.603767   962 leader_election.cc:304] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:51.604564  1113 ts_tablet_manager.cc:1403] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.091s	user 0.019s	sys 0.052s
W20260709 13:10:51.605382   965 leader_election.cc:343] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.605573  1132 raft_consensus.cc:2804] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.606065  1132 raft_consensus.cc:493] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.605988   965 leader_election.cc:304] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:51.606263  1115 tablet_bootstrap.cc:492] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.606596  1132 raft_consensus.cc:3060] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.607226  1115 ts_tablet_manager.cc:1403] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.057s	user 0.025s	sys 0.015s
I20260709 13:10:51.607478  1118 raft_consensus.cc:2749] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.608253  1113 raft_consensus.cc:359] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.609140  1113 raft_consensus.cc:385] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.609457  1113 raft_consensus.cc:740] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.610291  1113 consensus_queue.cc:260] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.610893  1115 raft_consensus.cc:359] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.611733  1115 raft_consensus.cc:385] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.612064  1115 raft_consensus.cc:740] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.612964  1115 consensus_queue.cc:260] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.613509  1113 ts_tablet_manager.cc:1434] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:10:51.614779  1113 tablet_bootstrap.cc:492] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.615955  1132 raft_consensus.cc:515] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.617364  1114 ts_tablet_manager.cc:1434] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.047s	user 0.014s	sys 0.016s
I20260709 13:10:51.617825  1115 ts_tablet_manager.cc:1434] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:10:51.619185  1115 tablet_bootstrap.cc:492] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.621254  1114 tablet_bootstrap.cc:492] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.622174  1113 tablet_bootstrap.cc:654] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.629436  1115 tablet_bootstrap.cc:654] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.630040  1114 tablet_bootstrap.cc:654] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.635378  1113 tablet_bootstrap.cc:492] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.635752   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:51.636174  1113 ts_tablet_manager.cc:1403] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:10:51.636598   924 raft_consensus.cc:3060] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.640672  1113 raft_consensus.cc:359] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.641611  1113 raft_consensus.cc:385] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.641932  1113 raft_consensus.cc:740] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.642867  1115 tablet_bootstrap.cc:492] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.643054  1113 consensus_queue.cc:260] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.643594  1115 ts_tablet_manager.cc:1403] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 13:10:51.645066  1119 raft_consensus.cc:493] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.645956  1113 ts_tablet_manager.cc:1434] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 13:10:51.645640  1119 raft_consensus.cc:515] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.647740   924 raft_consensus.cc:2468] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:51.648836  1113 tablet_bootstrap.cc:492] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.649335  1132 leader_election.cc:290] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.647737  1115 raft_consensus.cc:359] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.650311   962 leader_election.cc:304] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:51.650740  1115 raft_consensus.cc:385] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.651141  1115 raft_consensus.cc:740] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.636744  1118 raft_consensus.cc:493] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.652014  1115 consensus_queue.cc:260] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.652781  1123 raft_consensus.cc:493] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.653472  1119 leader_election.cc:290] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:51.653349  1123 raft_consensus.cc:515] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.654192  1119 raft_consensus.cc:493] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.654697  1119 raft_consensus.cc:515] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.657552  1113 tablet_bootstrap.cc:654] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.659030  1123 leader_election.cc:290] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.656914  1119 leader_election.cc:290] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:51.676455  1132 raft_consensus.cc:2804] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.651916  1118 raft_consensus.cc:515] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.688167  1132 raft_consensus.cc:697] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:51.689699  1132 consensus_queue.cc:237] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.693819  1118 leader_election.cc:290] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.722474   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.723737   924 raft_consensus.cc:2468] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
I20260709 13:10:51.723790  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:51.724819  1072 raft_consensus.cc:3060] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.725754   962 leader_election.cc:304] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:51.734413  1143 raft_consensus.cc:2804] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.735280  1143 raft_consensus.cc:493] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.735723  1143 raft_consensus.cc:3060] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.736342  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:51.737517  1073 raft_consensus.cc:2468] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
I20260709 13:10:51.743831  1113 tablet_bootstrap.cc:492] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.744572  1113 ts_tablet_manager.cc:1403] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.096s	user 0.036s	sys 0.055s
I20260709 13:10:51.748116  1113 raft_consensus.cc:359] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.748907  1113 raft_consensus.cc:385] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.749239  1113 raft_consensus.cc:740] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.750345  1113 consensus_queue.cc:260] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.752686   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.752856  1113 ts_tablet_manager.cc:1434] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:51.753600   924 raft_consensus.cc:2468] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:51.755779  1036 leader_election.cc:304] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:51.756939  1113 tablet_bootstrap.cc:492] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.757153   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.758282   923 raft_consensus.cc:2468] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:51.759699  1036 leader_election.cc:304] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:51.760751  1119 raft_consensus.cc:2804] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.761190  1119 raft_consensus.cc:493] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.761564  1119 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.775498  1138 raft_consensus.cc:2804] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.776129  1138 raft_consensus.cc:493] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.776229  1114 tablet_bootstrap.cc:492] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.776862  1138 raft_consensus.cc:3060] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.777165  1114 ts_tablet_manager.cc:1403] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.156s	user 0.025s	sys 0.050s
I20260709 13:10:51.764904  1143 raft_consensus.cc:515] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.787853  1143 leader_election.cc:290] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.788383   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:51.779901   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:51.780498  1115 ts_tablet_manager.cc:1434] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.136s	user 0.023s	sys 0.046s
I20260709 13:10:51.789316   923 raft_consensus.cc:3060] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.789814  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
W20260709 13:10:51.790526  1036 leader_election.cc:343] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.790705  1073 raft_consensus.cc:3060] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.792644  1115 tablet_bootstrap.cc:492] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.794080   822 catalog_manager.cc:5697] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.781255   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
W20260709 13:10:51.798607  1036 leader_election.cc:343] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.783160  1114 raft_consensus.cc:359] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.784034  1119 raft_consensus.cc:515] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.802138  1119 leader_election.cc:290] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:51.802331  1072 raft_consensus.cc:2468] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:51.803606   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:51.804629  1114 raft_consensus.cc:385] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.804986  1114 raft_consensus.cc:740] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
W20260709 13:10:51.805667   887 leader_election.cc:343] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.805960  1114 consensus_queue.cc:260] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.806967   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:51.808354   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:51.808758  1114 ts_tablet_manager.cc:1434] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:10:51.809090   924 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.810047  1114 tablet_bootstrap.cc:492] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.810698  1115 tablet_bootstrap.cc:654] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.811405  1073 raft_consensus.cc:2468] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
W20260709 13:10:51.813758  1036 leader_election.cc:343] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.813822   965 leader_election.cc:304] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:51.815140  1132 raft_consensus.cc:2804] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.815774  1132 raft_consensus.cc:697] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:51.815374  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:51.817495  1072 raft_consensus.cc:2393] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f082b9e30dac407eb4b01fcd8ff50941 in current term 1: Already voted for candidate 10abbd8bcc06430894c465db6ba1c1e3 in this term.
I20260709 13:10:51.816861  1132 consensus_queue.cc:237] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.819042   890 leader_election.cc:304] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f082b9e30dac407eb4b01fcd8ff50941; no voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:51.820247  1114 tablet_bootstrap.cc:654] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.827975   924 raft_consensus.cc:2468] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:51.830574  1123 raft_consensus.cc:2749] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.832204  1113 tablet_bootstrap.cc:654] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.832304  1036 leader_election.cc:304] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:51.836437  1119 raft_consensus.cc:2804] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.837380  1119 raft_consensus.cc:697] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:51.828244  1138 raft_consensus.cc:515] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.844132  1138 leader_election.cc:290] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:51.845327   923 raft_consensus.cc:2468] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:51.838714  1119 consensus_queue.cc:237] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.846884   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:51.846972   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:51.848069   924 raft_consensus.cc:3060] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.849810  1114 tablet_bootstrap.cc:492] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.850531  1114 ts_tablet_manager.cc:1403] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.011s
W20260709 13:10:51.852814  1036 leader_election.cc:343] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.854131  1114 raft_consensus.cc:359] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.857452  1114 raft_consensus.cc:385] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.860051  1115 tablet_bootstrap.cc:492] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.860844  1115 ts_tablet_manager.cc:1403] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.007s
I20260709 13:10:51.861035  1113 tablet_bootstrap.cc:492] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.861706  1113 ts_tablet_manager.cc:1403] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.105s	user 0.010s	sys 0.012s
I20260709 13:10:51.857903  1114 raft_consensus.cc:740] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.862179   924 raft_consensus.cc:2468] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:51.867345  1036 leader_election.cc:304] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:51.865314  1115 raft_consensus.cc:359] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.868373  1119 raft_consensus.cc:2804] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.868522  1115 raft_consensus.cc:385] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.868950  1115 raft_consensus.cc:740] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.867520  1114 consensus_queue.cc:260] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.869887  1115 consensus_queue.cc:260] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.871966  1114 ts_tablet_manager.cc:1434] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.003s
I20260709 13:10:51.872481  1115 ts_tablet_manager.cc:1434] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:51.873191  1114 tablet_bootstrap.cc:492] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.870173   821 catalog_manager.cc:5697] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.873744  1115 tablet_bootstrap.cc:492] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.868963  1119 raft_consensus.cc:697] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:51.877805  1119 consensus_queue.cc:237] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.881243  1114 tablet_bootstrap.cc:654] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.883277  1115 tablet_bootstrap.cc:654] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.880836   822 catalog_manager.cc:5697] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.884343  1132 raft_consensus.cc:493] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.885032  1132 raft_consensus.cc:515] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.888044  1132 leader_election.cc:290] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:51.890443  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:51.890443   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.893378   924 raft_consensus.cc:2393] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35bb9a7c4fd74b6d80bfc578303bc288 in current term 1: Already voted for candidate 10abbd8bcc06430894c465db6ba1c1e3 in this term.
I20260709 13:10:51.894836   962 leader_election.cc:304] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:51.865513  1113 raft_consensus.cc:359] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.896404  1132 raft_consensus.cc:3060] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.896649  1113 raft_consensus.cc:385] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.897058  1113 raft_consensus.cc:740] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.897971  1113 consensus_queue.cc:260] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.906108  1113 ts_tablet_manager.cc:1434] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:10:51.907389  1113 tablet_bootstrap.cc:492] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:51.909621  1132 raft_consensus.cc:2749] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.913556  1114 tablet_bootstrap.cc:492] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:51.914408  1114 ts_tablet_manager.cc:1403] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.009s
I20260709 13:10:51.915522  1113 tablet_bootstrap.cc:654] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.928772  1114 raft_consensus.cc:359] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.930203  1114 raft_consensus.cc:385] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.930605  1114 raft_consensus.cc:740] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.931708  1114 consensus_queue.cc:260] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.936594   821 catalog_manager.cc:5697] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.943372  1114 ts_tablet_manager.cc:1434] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.004s
I20260709 13:10:51.952467  1114 tablet_bootstrap.cc:492] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:51.954099  1132 raft_consensus.cc:493] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.954767  1132 raft_consensus.cc:515] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.979350  1115 tablet_bootstrap.cc:492] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:51.979902  1114 tablet_bootstrap.cc:654] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.980156  1115 ts_tablet_manager.cc:1403] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.107s	user 0.041s	sys 0.040s
I20260709 13:10:51.982976  1119 raft_consensus.cc:493] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.983549  1119 raft_consensus.cc:515] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.984254  1115 raft_consensus.cc:359] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.986310  1115 raft_consensus.cc:385] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.986076   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "885c11b7479d48968b92acefaf22c6f4" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:51.986805  1115 raft_consensus.cc:740] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.987969  1115 consensus_queue.cc:260] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
W20260709 13:10:51.989163   962 leader_election.cc:343] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.990753  1115 ts_tablet_manager.cc:1434] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:10:51.992094  1115 tablet_bootstrap.cc:492] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:51.992452  1113 tablet_bootstrap.cc:492] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:51.993273  1113 ts_tablet_manager.cc:1403] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.030s
I20260709 13:10:51.996250  1123 raft_consensus.cc:493] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.996997  1123 raft_consensus.cc:515] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:51.998503  1119 leader_election.cc:290] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.000692  1115 tablet_bootstrap.cc:654] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.001842   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.000720   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.004276   998 raft_consensus.cc:2468] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:52.006233  1036 leader_election.cc:304] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
W20260709 13:10:52.008369   887 leader_election.cc:343] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.997254   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:52.009511  1119 raft_consensus.cc:2804] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.010239  1119 raft_consensus.cc:493] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.010823  1119 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:52.011029  1036 leader_election.cc:343] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.014218  1123 leader_election.cc:290] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.016386  1138 raft_consensus.cc:493] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.017408  1138 raft_consensus.cc:515] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.020045  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "885c11b7479d48968b92acefaf22c6f4" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:52.016247  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:52.021152  1073 raft_consensus.cc:2468] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
I20260709 13:10:52.021667  1072 raft_consensus.cc:2468] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:52.024125   965 leader_election.cc:304] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.020804  1138 leader_election.cc:290] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.026188  1143 raft_consensus.cc:2804] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.026652  1143 raft_consensus.cc:493] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.027040  1143 raft_consensus.cc:3060] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.029546   890 leader_election.cc:304] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:52.031966   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "885c11b7479d48968b92acefaf22c6f4" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.037189   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "885c11b7479d48968b92acefaf22c6f4" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
W20260709 13:10:52.039532  1036 leader_election.cc:343] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.041147  1123 raft_consensus.cc:2804] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.040968  1119 raft_consensus.cc:515] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.041850  1123 raft_consensus.cc:493] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.042390  1123 raft_consensus.cc:3060] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.047042  1119 leader_election.cc:290] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.048269   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:52.049511   999 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.053789   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
W20260709 13:10:52.056476  1036 leader_election.cc:343] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.064412   999 raft_consensus.cc:2468] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:52.065163  1132 leader_election.cc:290] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.066875  1036 leader_election.cc:304] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.070161  1119 raft_consensus.cc:2804] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.072697  1119 raft_consensus.cc:697] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:52.074244  1119 consensus_queue.cc:237] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.076727  1143 raft_consensus.cc:515] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.083961   998 raft_consensus.cc:2393] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10abbd8bcc06430894c465db6ba1c1e3 in current term 1: Already voted for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in this term.
I20260709 13:10:52.081735  1113 raft_consensus.cc:359] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.089805  1036 leader_election.cc:304] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3; no voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.091645  1123 raft_consensus.cc:515] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.091979  1138 raft_consensus.cc:3060] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.092343  1132 raft_consensus.cc:493] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.094801  1113 raft_consensus.cc:385] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.096288  1113 raft_consensus.cc:740] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.096042  1132 raft_consensus.cc:515] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.098845  1123 leader_election.cc:290] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.101076  1132 leader_election.cc:290] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.101229  1143 leader_election.cc:290] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.103988  1114 tablet_bootstrap.cc:492] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:52.105347   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "885c11b7479d48968b92acefaf22c6f4" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:52.106509  1114 ts_tablet_manager.cc:1403] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.154s	user 0.056s	sys 0.063s
I20260709 13:10:52.106827  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "885c11b7479d48968b92acefaf22c6f4" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:52.107847  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:52.109388   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
W20260709 13:10:52.109681   965 leader_election.cc:343] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.110863   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:52.110873  1114 raft_consensus.cc:359] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.111831  1115 tablet_bootstrap.cc:492] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
W20260709 13:10:52.110808   962 leader_election.cc:343] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.112166  1114 raft_consensus.cc:385] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.112567  1114 raft_consensus.cc:740] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.112489  1115 ts_tablet_manager.cc:1403] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.121s	user 0.038s	sys 0.061s
I20260709 13:10:52.112424   962 leader_election.cc:304] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.111940  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:52.113624  1114 consensus_queue.cc:260] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.114567  1072 raft_consensus.cc:3060] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.114879  1132 raft_consensus.cc:2749] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:52.115139   887 leader_election.cc:343] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.117161  1114 ts_tablet_manager.cc:1434] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:10:52.117250  1115 raft_consensus.cc:359] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.118052  1115 raft_consensus.cc:385] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.118266  1115 raft_consensus.cc:740] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.118548  1114 tablet_bootstrap.cc:492] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:52.119032  1115 consensus_queue.cc:260] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.121202  1115 ts_tablet_manager.cc:1434] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:52.120829   820 catalog_manager.cc:5697] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.122275  1115 tablet_bootstrap.cc:492] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:52.125902  1114 tablet_bootstrap.cc:654] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.130790  1115 tablet_bootstrap.cc:654] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.150972  1072 raft_consensus.cc:2468] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:52.156404  1138 raft_consensus.cc:2749] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.172168   890 leader_election.cc:304] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:52.173849  1123 raft_consensus.cc:2804] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.174342  1113 consensus_queue.cc:260] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.175992   923 raft_consensus.cc:3060] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.177362  1115 tablet_bootstrap.cc:492] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:52.178030  1073 raft_consensus.cc:2468] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:52.178198  1132 consensus_queue.cc:1048] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.178046  1115 ts_tablet_manager.cc:1403] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.031s
I20260709 13:10:52.180658   965 leader_election.cc:304] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:52.181635  1132 raft_consensus.cc:2804] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.182962  1132 raft_consensus.cc:697] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:52.183566  1115 raft_consensus.cc:359] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.183880  1132 consensus_queue.cc:237] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.184443  1115 raft_consensus.cc:385] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.184836  1115 raft_consensus.cc:740] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.185671  1115 consensus_queue.cc:260] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.186225  1114 tablet_bootstrap.cc:492] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:52.186918  1114 ts_tablet_manager.cc:1403] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.069s	user 0.027s	sys 0.027s
I20260709 13:10:52.187717   923 raft_consensus.cc:2468] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:52.191203  1138 raft_consensus.cc:493] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.191850  1138 raft_consensus.cc:515] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.194031  1167 raft_consensus.cc:493] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.194545  1138 leader_election.cc:290] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.194658  1167 raft_consensus.cc:515] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.196918  1115 ts_tablet_manager.cc:1434] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:10:52.197118  1113 ts_tablet_manager.cc:1434] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.203s	user 0.014s	sys 0.004s
I20260709 13:10:52.196771   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.198055  1114 raft_consensus.cc:359] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.198503  1143 consensus_queue.cc:1048] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.198985  1114 raft_consensus.cc:385] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.199296  1114 raft_consensus.cc:740] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.198444  1113 tablet_bootstrap.cc:492] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:52.200109  1114 consensus_queue.cc:260] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.202728  1114 ts_tablet_manager.cc:1434] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:10:52.202658   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.204118  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:52.202023  1167 leader_election.cc:290] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.198021   998 raft_consensus.cc:2468] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:52.205121  1072 raft_consensus.cc:2468] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:52.207195  1036 leader_election.cc:304] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:52.208663  1138 raft_consensus.cc:2804] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.209240  1138 raft_consensus.cc:493] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.209708  1138 raft_consensus.cc:3060] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.213527   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:52.213879  1113 tablet_bootstrap.cc:654] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.214617  1123 raft_consensus.cc:697] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:52.214668   924 raft_consensus.cc:2468] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:52.215111   890 leader_election.cc:304] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:52.217612  1164 raft_consensus.cc:2804] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.218124  1164 raft_consensus.cc:493] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.218549  1164 raft_consensus.cc:3060] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.217338  1123 consensus_queue.cc:237] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.219841   820 catalog_manager.cc:5697] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.224247  1143 raft_consensus.cc:493] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.225479  1143 raft_consensus.cc:515] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.229810   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:52.231184  1143 leader_election.cc:290] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.231383  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:52.232568  1072 raft_consensus.cc:2393] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35bb9a7c4fd74b6d80bfc578303bc288 in current term 1: Already voted for candidate f082b9e30dac407eb4b01fcd8ff50941 in this term.
I20260709 13:10:52.234615   962 leader_election.cc:304] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.235553  1143 raft_consensus.cc:3060] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.242208  1164 raft_consensus.cc:515] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.247612  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:52.247519   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:52.248476   998 raft_consensus.cc:3060] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.270795  1138 raft_consensus.cc:515] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.289665  1072 raft_consensus.cc:2393] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f082b9e30dac407eb4b01fcd8ff50941 in current term 1: Already voted for candidate 10abbd8bcc06430894c465db6ba1c1e3 in this term.
I20260709 13:10:52.261669   821 catalog_manager.cc:5697] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.290719  1132 consensus_queue.cc:1048] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.290781  1138 leader_election.cc:290] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.293898  1113 tablet_bootstrap.cc:492] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:52.294608  1113 ts_tablet_manager.cc:1403] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.004s
I20260709 13:10:52.259080  1136 consensus_queue.cc:1048] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:10:52.258601  1119 raft_consensus.cc:493] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.248463  1164 leader_election.cc:290] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.299717  1113 raft_consensus.cc:359] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.305768  1113 raft_consensus.cc:385] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.306326  1113 raft_consensus.cc:740] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.307485  1113 consensus_queue.cc:260] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.309118  1119 raft_consensus.cc:515] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.310914   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:52.311935   922 raft_consensus.cc:2393] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 10abbd8bcc06430894c465db6ba1c1e3 in current term 1: Already voted for candidate f082b9e30dac407eb4b01fcd8ff50941 in this term.
I20260709 13:10:52.311859  1119 leader_election.cc:290] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.311626  1143 raft_consensus.cc:2749] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.315784   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:52.316208  1113 ts_tablet_manager.cc:1434] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:10:52.318267  1113 tablet_bootstrap.cc:492] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:52.328608   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:52.343932   921 raft_consensus.cc:2468] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:52.328780   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.345598   999 raft_consensus.cc:2468] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:52.329859  1143 consensus_queue.cc:1048] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.331988  1036 leader_election.cc:304] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3; no voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.350343  1113 tablet_bootstrap.cc:654] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.351824  1036 leader_election.cc:304] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:52.355206  1119 raft_consensus.cc:2749] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.356139  1119 raft_consensus.cc:2804] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.356571  1119 raft_consensus.cc:493] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.356947  1119 raft_consensus.cc:3060] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.366083   999 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.335085   998 raft_consensus.cc:2468] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:52.373785   887 leader_election.cc:304] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 10abbd8bcc06430894c465db6ba1c1e3
I20260709 13:10:52.375433  1123 raft_consensus.cc:2804] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.376356  1123 raft_consensus.cc:697] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:52.377681  1123 consensus_queue.cc:237] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.411908  1172 consensus_queue.cc:1048] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.427600  1119 raft_consensus.cc:515] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.438625   820 catalog_manager.cc:5697] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:52.443522   965 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:39805, user_credentials={real_user=slave}} blocked reactor thread for 89822.4us
I20260709 13:10:52.464740  1172 consensus_queue.cc:1048] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.491263   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:52.492342   924 raft_consensus.cc:3060] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:52.498449  1036 consensus_peers.cc:597] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:52.503978   870 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:52.510695   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:52.511582   998 raft_consensus.cc:3060] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.511819   924 raft_consensus.cc:2468] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:52.514005  1036 leader_election.cc:304] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:52.514832  1119 leader_election.cc:290] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:52.515944  1172 raft_consensus.cc:2804] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.518059  1172 raft_consensus.cc:697] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:52.518654  1113 tablet_bootstrap.cc:492] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:52.519315  1113 ts_tablet_manager.cc:1403] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.202s	user 0.079s	sys 0.082s
I20260709 13:10:52.518996  1172 consensus_queue.cc:237] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.522714  1113 raft_consensus.cc:359] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.523558  1113 raft_consensus.cc:385] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.523886  1113 raft_consensus.cc:740] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.524731  1113 consensus_queue.cc:260] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.525696   998 raft_consensus.cc:2468] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:52.525528  1136 consensus_queue.cc:1048] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.527299  1113 ts_tablet_manager.cc:1434] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:52.540490   819 catalog_manager.cc:5697] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.608834  1172 consensus_queue.cc:1048] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.720968  1123 consensus_queue.cc:1048] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.726660  1123 raft_consensus.cc:493] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.727690  1123 raft_consensus.cc:515] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.732722  1123 leader_election.cc:290] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.733878   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:52.735523  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:52.739488   890 leader_election.cc:304] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f082b9e30dac407eb4b01fcd8ff50941; no voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:52.744887  1123 raft_consensus.cc:2749] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.750703  1175 consensus_queue.cc:1048] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:52.752530  1192 consensus_queue.cc:1048] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.785890  1178 consensus_queue.cc:1048] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.842320  1192 consensus_queue.cc:1048] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.847931  1164 consensus_queue.cc:1048] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.899051  1073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:52.899935  1073 raft_consensus.cc:606] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.904932  1172 consensus_queue.cc:1048] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.909286  1073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684feeef469040c1bce9199d5510e30d"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:52.909937  1073 raft_consensus.cc:606] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.917266  1073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:52.918046  1073 raft_consensus.cc:606] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.920436  1136 raft_consensus.cc:993] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:52.921078  1136 raft_consensus.cc:1081] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:52.923558   924 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:49588
I20260709 13:10:52.924257   924 raft_consensus.cc:493] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:52.924754   924 raft_consensus.cc:3060] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:52.929082  1138 consensus_queue.cc:1048] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.941391   924 raft_consensus.cc:515] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.942747   922 raft_consensus.cc:1240] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Rejecting Update request from peer 10abbd8bcc06430894c465db6ba1c1e3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:52.945499   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:52.946346   998 raft_consensus.cc:3060] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:52.952525  1172 consensus_queue.cc:1059] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.954011  1172 raft_consensus.cc:3055] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:52.954396  1172 raft_consensus.cc:740] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:52.955579  1172 consensus_queue.cc:260] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.957118   924 leader_election.cc:290] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:52.957831  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304180aa5ffa4d76865a1867048b6ea1" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:52.957708  1172 raft_consensus.cc:3060] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:52.961730   999 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
mode: GRACEFUL
new_leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:48872
I20260709 13:10:52.962663   999 raft_consensus.cc:606] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Received request to transfer leadership to TS f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:52.965931   998 raft_consensus.cc:2468] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:52.971763   887 leader_election.cc:304] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:52.973140  1123 raft_consensus.cc:2804] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:52.973853  1123 raft_consensus.cc:697] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:52.975536  1123 consensus_queue.cc:237] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:52.982107  1073 raft_consensus.cc:2468] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:52.989956   820 catalog_manager.cc:5697] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.994624   924 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:49604
I20260709 13:10:52.995684   924 raft_consensus.cc:606] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:53.004467   924 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c489bd1e018427485257e79220aef24"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:49604
I20260709 13:10:53.005198   924 raft_consensus.cc:606] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:53.011780  1073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:53.012485  1073 raft_consensus.cc:606] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:53.024842   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
mode: GRACEFUL
new_leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:48872
I20260709 13:10:53.026074   998 raft_consensus.cc:606] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Received request to transfer leadership to TS 10abbd8bcc06430894c465db6ba1c1e3
I20260709 13:10:53.041273  1138 raft_consensus.cc:993] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:53.041893  1172 raft_consensus.cc:1081] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:53.043738   924 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:49588
I20260709 13:10:53.044304   924 raft_consensus.cc:493] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.044633   924 raft_consensus.cc:3060] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.051831   924 raft_consensus.cc:515] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.054046   924 leader_election.cc:290] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:53.055130   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:53.055282  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "360a901b5e1947db83ce7fd9190c7ee6" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:53.055866   998 raft_consensus.cc:3060] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.056063  1073 raft_consensus.cc:3055] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.056463  1073 raft_consensus.cc:740] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:53.057426  1073 consensus_queue.cc:260] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.058633  1073 raft_consensus.cc:3060] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.062708   998 raft_consensus.cc:2468] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:53.064325   887 leader_election.cc:304] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:53.065331  1073 raft_consensus.cc:2468] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:53.065395  1123 raft_consensus.cc:2804] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.066080  1123 raft_consensus.cc:697] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:53.067245  1123 consensus_queue.cc:237] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.075438   821 catalog_manager.cc:5697] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.098644   924 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.112736  1138 raft_consensus.cc:993] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:53.113329  1138 raft_consensus.cc:1081] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:53.116564   921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "684feeef469040c1bce9199d5510e30d"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:49588
I20260709 13:10:53.117261   921 raft_consensus.cc:493] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.117635   921 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.127497   921 raft_consensus.cc:515] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.130154   921 leader_election.cc:290] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:53.131085   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:53.131842   998 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.132390  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:53.133188  1073 raft_consensus.cc:3055] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.133560  1073 raft_consensus.cc:740] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:53.134845  1073 consensus_queue.cc:260] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.136571  1073 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.142117   998 raft_consensus.cc:2468] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:53.144260   887 leader_election.cc:304] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:53.145351  1123 raft_consensus.cc:2804] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.145834  1123 raft_consensus.cc:697] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:53.147001  1123 consensus_queue.cc:237] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.147167  1073 raft_consensus.cc:2468] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:53.155486   821 catalog_manager.cc:5697] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.290059  1164 raft_consensus.cc:993] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:53.290628  1123 raft_consensus.cc:1081] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:53.292397   998 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48854
I20260709 13:10:53.292939   998 raft_consensus.cc:493] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.293298   998 raft_consensus.cc:3060] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.300203   998 raft_consensus.cc:515] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.302511   998 leader_election.cc:290] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:53.304241   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:53.304643  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810e75e22d7a44158f99e29bf93557b0" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:53.305143   921 raft_consensus.cc:3055] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.305480  1073 raft_consensus.cc:3060] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.305594   921 raft_consensus.cc:740] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:53.307039   921 consensus_queue.cc:260] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.308357   921 raft_consensus.cc:3060] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.313828  1073 raft_consensus.cc:2468] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:53.315443   965 leader_election.cc:304] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:53.316493  1175 raft_consensus.cc:2804] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.317493   921 raft_consensus.cc:2468] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:53.318039  1175 raft_consensus.cc:697] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:53.319691  1205 raft_consensus.cc:993] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: : Instructing follower 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:10:53.319469  1175 consensus_queue.cc:237] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.320252  1205 raft_consensus.cc:1081] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Signalling peer 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:10:53.322539  1072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:40316
I20260709 13:10:53.323259  1072 raft_consensus.cc:493] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.323658  1072 raft_consensus.cc:3060] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.329196   821 catalog_manager.cc:5697] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.333273  1072 raft_consensus.cc:515] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.336135  1072 leader_election.cc:290] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:53.337406   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:53.337759   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffae32838b94fcab46259eec44d7b28" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:53.338220   921 raft_consensus.cc:3060] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.338727   998 raft_consensus.cc:3055] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.339123   998 raft_consensus.cc:740] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:53.340241   998 consensus_queue.cc:260] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.341831   998 raft_consensus.cc:3060] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.345770   921 raft_consensus.cc:2468] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 2.
I20260709 13:10:53.347570  1036 leader_election.cc:304] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:53.348601  1172 raft_consensus.cc:2804] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.348615   998 raft_consensus.cc:2468] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 2.
I20260709 13:10:53.349192  1172 raft_consensus.cc:697] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:53.350494  1172 consensus_queue.cc:237] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.358757   821 catalog_manager.cc:5697] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 1 to 2, leader changed from 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130) to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.465197  1164 raft_consensus.cc:993] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:53.465731  1123 raft_consensus.cc:1081] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:53.467468   998 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c489bd1e018427485257e79220aef24"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48854
I20260709 13:10:53.468065   998 raft_consensus.cc:493] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.468439   998 raft_consensus.cc:3060] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.475265   998 raft_consensus.cc:515] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.477321   998 leader_election.cc:290] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:53.478502  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:53.478309   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c489bd1e018427485257e79220aef24" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:53.479229  1072 raft_consensus.cc:3060] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.479380   921 raft_consensus.cc:3055] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.479832   921 raft_consensus.cc:740] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:53.480755   921 consensus_queue.cc:260] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.481997   921 raft_consensus.cc:3060] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.486251  1072 raft_consensus.cc:2468] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:53.488158   998 raft_consensus.cc:1275] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.488243   965 leader_election.cc:304] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:53.490145  1175 raft_consensus.cc:2804] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.490695  1175 raft_consensus.cc:697] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:53.491652  1123 consensus_queue.cc:1048] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:53.491959  1175 consensus_queue.cc:237] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.496546   921 raft_consensus.cc:2468] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:53.499310  1072 raft_consensus.cc:1275] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.501929  1192 consensus_queue.cc:1048] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:53.506652  1073 raft_consensus.cc:1275] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.506256   821 catalog_manager.cc:5697] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.510076  1175 raft_consensus.cc:993] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: : Instructing follower f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:53.510725  1175 raft_consensus.cc:1081] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Signalling peer f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:53.510898  1164 consensus_queue.cc:1048] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:53.513264   921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:10:53.513936   921 raft_consensus.cc:493] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.514384   921 raft_consensus.cc:3060] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.529078   921 raft_consensus.cc:515] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.532786   921 leader_election.cc:290] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:53.535205   998 raft_consensus.cc:1275] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.535995   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:53.537101   999 raft_consensus.cc:3055] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.537544  1192 consensus_queue.cc:1048] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:53.537675   999 raft_consensus.cc:740] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:53.541472   999 consensus_queue.cc:260] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.544085   999 raft_consensus.cc:3060] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.547868  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3fde5a7ea7497f9658945d8d794518" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:53.548813  1071 raft_consensus.cc:3060] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.575102  1071 raft_consensus.cc:2468] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:53.577277   890 leader_election.cc:304] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:53.578269  1192 raft_consensus.cc:2804] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.575101   999 raft_consensus.cc:2468] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:53.578871  1192 raft_consensus.cc:697] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:53.580078  1192 consensus_queue.cc:237] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.589612   819 catalog_manager.cc:5697] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 1 to 2, leader changed from 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130) to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.615440  1119 raft_consensus.cc:993] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:53.616035  1172 raft_consensus.cc:1081] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:53.617893   999 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:10:53.618508   999 raft_consensus.cc:493] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:53.618902   999 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.626256  1071 raft_consensus.cc:1275] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.627640   999 raft_consensus.cc:515] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.627912  1192 consensus_queue.cc:1048] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:53.630741   999 leader_election.cc:290] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:53.631392   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:53.632050  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:53.632247   921 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.632797  1073 raft_consensus.cc:3055] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:53.633216  1073 raft_consensus.cc:740] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:53.634400  1073 consensus_queue.cc:260] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.636231  1073 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:53.646385  1073 raft_consensus.cc:2468] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:53.648989   999 raft_consensus.cc:1275] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.649400   965 leader_election.cc:304] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:53.650713  1206 raft_consensus.cc:2804] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:53.651213  1206 raft_consensus.cc:697] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:53.651645   921 raft_consensus.cc:2468] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:53.651343  1192 consensus_queue.cc:1048] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:53.652458  1206 consensus_queue.cc:237] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:53.665241   821 catalog_manager.cc:5697] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.861124   921 raft_consensus.cc:1275] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.862777  1206 consensus_queue.cc:1048] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:53.873741  1071 raft_consensus.cc:1275] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.876286  1175 consensus_queue.cc:1048] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:53.891476   921 raft_consensus.cc:1275] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.892923  1172 consensus_queue.cc:1048] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:53.908154   999 raft_consensus.cc:1275] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:53.909826  1172 consensus_queue.cc:1048] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:54.007802   921 raft_consensus.cc:1275] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.009675  1175 consensus_queue.cc:1048] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:54.020735  1071 raft_consensus.cc:1275] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.023049  1221 consensus_queue.cc:1048] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:54.064361   999 raft_consensus.cc:1275] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.066243  1164 consensus_queue.cc:1048] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:54.077992  1071 raft_consensus.cc:1275] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.080302  1123 consensus_queue.cc:1048] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:54.117619   921 raft_consensus.cc:1275] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.119215  1221 consensus_queue.cc:1048] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:54.130759  1073 raft_consensus.cc:1275] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.132804  1205 consensus_queue.cc:1048] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:56.067289   821 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42302:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:56.070111   821 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:56.125247   913 tablet_service.cc:1511] Processing CreateTablet for tablet f95e186d7cc54f54b94d75c75caf2559 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:56.126567   985 tablet_service.cc:1511] Processing CreateTablet for tablet f95e186d7cc54f54b94d75c75caf2559 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:56.127240   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95e186d7cc54f54b94d75c75caf2559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.128384   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95e186d7cc54f54b94d75c75caf2559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.128261  1061 tablet_service.cc:1511] Processing CreateTablet for tablet f95e186d7cc54f54b94d75c75caf2559 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:56.130079  1061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95e186d7cc54f54b94d75c75caf2559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.141552   907 tablet_service.cc:1511] Processing CreateTablet for tablet c8b2063c07ab4aa78745f92e65497503 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:56.143488   907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b2063c07ab4aa78745f92e65497503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.145885   983 tablet_service.cc:1511] Processing CreateTablet for tablet c8b2063c07ab4aa78745f92e65497503 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:56.147800   983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b2063c07ab4aa78745f92e65497503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.155531  1058 tablet_service.cc:1511] Processing CreateTablet for tablet c8b2063c07ab4aa78745f92e65497503 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:56.156980   912 tablet_service.cc:1511] Processing CreateTablet for tablet 90a69418a2e24142b8f493f95b791fa6 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:56.157570  1058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b2063c07ab4aa78745f92e65497503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.158679   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a69418a2e24142b8f493f95b791fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.165024   982 tablet_service.cc:1511] Processing CreateTablet for tablet 90a69418a2e24142b8f493f95b791fa6 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:56.167483   982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a69418a2e24142b8f493f95b791fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.202560  1242 tablet_bootstrap.cc:492] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.202560  1241 tablet_bootstrap.cc:492] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.210006  1059 tablet_service.cc:1511] Processing CreateTablet for tablet 90a69418a2e24142b8f493f95b791fa6 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:56.211925  1059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a69418a2e24142b8f493f95b791fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.218479  1062 tablet_service.cc:1511] Processing CreateTablet for tablet 0069a174783946138754fb3f83a2cff0 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:56.220427  1062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0069a174783946138754fb3f83a2cff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.221704  1241 tablet_bootstrap.cc:654] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.223348  1242 tablet_bootstrap.cc:654] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.227232   912 tablet_service.cc:1511] Processing CreateTablet for tablet 0069a174783946138754fb3f83a2cff0 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:56.230551   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0069a174783946138754fb3f83a2cff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.233115  1055 tablet_service.cc:1511] Processing CreateTablet for tablet e0262128210e42f3a6e000dbe2558957 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:56.233876  1243 tablet_bootstrap.cc:492] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.234822  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0262128210e42f3a6e000dbe2558957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.235620   913 tablet_service.cc:1511] Processing CreateTablet for tablet e0262128210e42f3a6e000dbe2558957 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:56.237110   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0262128210e42f3a6e000dbe2558957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.238212   982 tablet_service.cc:1511] Processing CreateTablet for tablet 0069a174783946138754fb3f83a2cff0 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:56.239759   982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0069a174783946138754fb3f83a2cff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.247193   985 tablet_service.cc:1511] Processing CreateTablet for tablet e0262128210e42f3a6e000dbe2558957 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:56.247517   908 tablet_service.cc:1511] Processing CreateTablet for tablet b381fc146f684b719f6f6fac201a0f33 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:56.248765   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0262128210e42f3a6e000dbe2558957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.249073   908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b381fc146f684b719f6f6fac201a0f33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.245523  1058 tablet_service.cc:1511] Processing CreateTablet for tablet b381fc146f684b719f6f6fac201a0f33 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:56.252105   983 tablet_service.cc:1511] Processing CreateTablet for tablet b381fc146f684b719f6f6fac201a0f33 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:56.254045   983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b381fc146f684b719f6f6fac201a0f33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.267822   909 tablet_service.cc:1511] Processing CreateTablet for tablet a7b9397a82af41768cf4d00b94c76319 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:56.270918   909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b9397a82af41768cf4d00b94c76319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.271601  1058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b381fc146f684b719f6f6fac201a0f33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.272154  1243 tablet_bootstrap.cc:654] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.277684  1242 tablet_bootstrap.cc:492] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.278728   988 tablet_service.cc:1511] Processing CreateTablet for tablet a7b9397a82af41768cf4d00b94c76319 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:56.285681  1242 ts_tablet_manager.cc:1403] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.083s	user 0.007s	sys 0.013s
I20260709 13:10:56.286165   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b9397a82af41768cf4d00b94c76319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.291079  1242 raft_consensus.cc:359] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.299253  1241 tablet_bootstrap.cc:492] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.305815  1241 ts_tablet_manager.cc:1403] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.009s
I20260709 13:10:56.307597  1061 tablet_service.cc:1511] Processing CreateTablet for tablet a7b9397a82af41768cf4d00b94c76319 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:56.309595  1061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b9397a82af41768cf4d00b94c76319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.318826   907 tablet_service.cc:1511] Processing CreateTablet for tablet 979a0bee5c50496996be259fd01a6f23 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:56.320542   907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979a0bee5c50496996be259fd01a6f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.321475  1242 raft_consensus.cc:385] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.334103  1242 raft_consensus.cc:740] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.331979  1059 tablet_service.cc:1511] Processing CreateTablet for tablet 979a0bee5c50496996be259fd01a6f23 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:56.338021  1059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979a0bee5c50496996be259fd01a6f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.339537  1243 tablet_bootstrap.cc:492] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.343353  1243 ts_tablet_manager.cc:1403] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.010s
I20260709 13:10:56.342259   914 tablet_service.cc:1511] Processing CreateTablet for tablet 4e5ec99df3194749ab8172588bc38bfb (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:56.336643  1242 consensus_queue.cc:260] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.345321   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e5ec99df3194749ab8172588bc38bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.343684  1241 raft_consensus.cc:359] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.350391   985 tablet_service.cc:1511] Processing CreateTablet for tablet 4e5ec99df3194749ab8172588bc38bfb (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:56.346024   982 tablet_service.cc:1511] Processing CreateTablet for tablet 979a0bee5c50496996be259fd01a6f23 (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:56.363471   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e5ec99df3194749ab8172588bc38bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.355711  1241 raft_consensus.cc:385] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.364606  1241 raft_consensus.cc:740] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.365677  1241 consensus_queue.cc:260] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.355219  1242 ts_tablet_manager.cc:1434] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:10:56.368381  1242 tablet_bootstrap.cc:492] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.347076  1243 raft_consensus.cc:359] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.372123  1243 raft_consensus.cc:385] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.372536  1243 raft_consensus.cc:740] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.370131   982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979a0bee5c50496996be259fd01a6f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.378068  1241 ts_tablet_manager.cc:1434] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.005s
I20260709 13:10:56.377662  1058 tablet_service.cc:1511] Processing CreateTablet for tablet 4e5ec99df3194749ab8172588bc38bfb (DEFAULT_TABLE table=soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:56.381839  1058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e5ec99df3194749ab8172588bc38bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.373606  1243 consensus_queue.cc:260] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.399282  1241 tablet_bootstrap.cc:492] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.401763  1243 ts_tablet_manager.cc:1434] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.058s	user 0.000s	sys 0.009s
I20260709 13:10:56.408815  1242 tablet_bootstrap.cc:654] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.433964  1243 tablet_bootstrap.cc:492] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.442637  1243 tablet_bootstrap.cc:654] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.447101  1241 tablet_bootstrap.cc:654] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.462203  1243 tablet_bootstrap.cc:492] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.462802  1243 ts_tablet_manager.cc:1403] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.008s
I20260709 13:10:56.465776  1243 raft_consensus.cc:359] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.466642  1243 raft_consensus.cc:385] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.466996  1243 raft_consensus.cc:740] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.467871  1243 consensus_queue.cc:260] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.470048  1242 tablet_bootstrap.cc:492] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.470801  1242 ts_tablet_manager.cc:1403] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.013s
I20260709 13:10:56.471328  1243 ts_tablet_manager.cc:1434] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:10:56.472697  1243 tablet_bootstrap.cc:492] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.474712  1242 raft_consensus.cc:359] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.475845  1242 raft_consensus.cc:385] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.476209  1242 raft_consensus.cc:740] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.477648  1242 consensus_queue.cc:260] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.481123  1241 tablet_bootstrap.cc:492] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.481779  1241 ts_tablet_manager.cc:1403] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.012s
I20260709 13:10:56.481849  1243 tablet_bootstrap.cc:654] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.484179  1242 ts_tablet_manager.cc:1434] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:10:56.485522  1242 tablet_bootstrap.cc:492] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.494494  1243 tablet_bootstrap.cc:492] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.495275  1243 ts_tablet_manager.cc:1403] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.016s
I20260709 13:10:56.495740  1242 tablet_bootstrap.cc:654] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.497500  1241 raft_consensus.cc:359] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.498513  1241 raft_consensus.cc:385] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.498843  1241 raft_consensus.cc:740] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.498826  1243 raft_consensus.cc:359] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.499744  1243 raft_consensus.cc:385] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.500052  1243 raft_consensus.cc:740] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.500023  1241 consensus_queue.cc:260] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.501130  1243 consensus_queue.cc:260] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.503799  1241 ts_tablet_manager.cc:1434] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:10:56.507608  1243 ts_tablet_manager.cc:1434] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:10:56.509065  1243 tablet_bootstrap.cc:492] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.509773  1241 tablet_bootstrap.cc:492] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.510921  1242 tablet_bootstrap.cc:492] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.511659  1242 ts_tablet_manager.cc:1403] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.010s
I20260709 13:10:56.516064  1242 raft_consensus.cc:359] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.517463  1243 tablet_bootstrap.cc:654] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.518258  1228 raft_consensus.cc:493] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.517520  1242 raft_consensus.cc:385] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.518961  1228 raft_consensus.cc:515] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.519610  1242 raft_consensus.cc:740] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.520776  1242 consensus_queue.cc:260] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.523900  1242 ts_tablet_manager.cc:1434] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:10:56.525684  1242 tablet_bootstrap.cc:492] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.527119   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:56.532573  1243 tablet_bootstrap.cc:492] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.533461  1243 ts_tablet_manager.cc:1403] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.013s
I20260709 13:10:56.534439  1242 tablet_bootstrap.cc:654] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:56.536312  1036 leader_election.cc:343] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.537581  1228 leader_election.cc:290] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.538301  1241 tablet_bootstrap.cc:654] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.537923  1243 raft_consensus.cc:359] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.539464   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.541715   999 raft_consensus.cc:2468] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:56.541999  1243 raft_consensus.cc:385] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.542389  1243 raft_consensus.cc:740] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.543349  1243 consensus_queue.cc:260] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.543478  1036 leader_election.cc:304] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:56.544819  1228 raft_consensus.cc:2804] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.545279  1228 raft_consensus.cc:493] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.550294  1243 ts_tablet_manager.cc:1434] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.005s
I20260709 13:10:56.550948  1228 raft_consensus.cc:3060] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.551201  1242 tablet_bootstrap.cc:492] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.551905  1242 ts_tablet_manager.cc:1403] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.004s
I20260709 13:10:56.552142  1243 tablet_bootstrap.cc:492] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.555509  1242 raft_consensus.cc:359] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.560424  1242 raft_consensus.cc:385] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.560923  1242 raft_consensus.cc:740] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.562088  1242 consensus_queue.cc:260] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.575338  1243 tablet_bootstrap.cc:654] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.583225  1241 tablet_bootstrap.cc:492] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.583948  1241 ts_tablet_manager.cc:1403] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.012s
I20260709 13:10:56.585991  1228 raft_consensus.cc:515] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.587589  1241 raft_consensus.cc:359] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.588549  1228 leader_election.cc:290] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.588907  1241 raft_consensus.cc:385] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.589380  1241 raft_consensus.cc:740] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.590523  1242 ts_tablet_manager.cc:1434] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.038s	user 0.014s	sys 0.012s
I20260709 13:10:56.590332  1241 consensus_queue.cc:260] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.591976   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:56.592697   921 raft_consensus.cc:3060] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.592945  1241 ts_tablet_manager.cc:1434] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:10:56.594306  1241 tablet_bootstrap.cc:492] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.595996  1242 tablet_bootstrap.cc:492] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.602267  1241 tablet_bootstrap.cc:654] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.602607   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:56.603395   999 raft_consensus.cc:3060] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.605289  1242 tablet_bootstrap.cc:654] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:56.606793   870 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:56.614797  1243 tablet_bootstrap.cc:492] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.615618  1243 ts_tablet_manager.cc:1403] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.003s
I20260709 13:10:56.617877   921 raft_consensus.cc:2468] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:56.619283  1242 tablet_bootstrap.cc:492] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.620285  1243 raft_consensus.cc:359] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.621143  1243 raft_consensus.cc:385] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.621430  1243 raft_consensus.cc:740] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.622576  1242 ts_tablet_manager.cc:1403] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.007s
I20260709 13:10:56.622282  1243 consensus_queue.cc:260] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.627171   999 raft_consensus.cc:2468] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:56.629498  1243 ts_tablet_manager.cc:1434] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:10:56.631727  1243 tablet_bootstrap.cc:492] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.631880  1036 leader_election.cc:304] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:56.634152  1228 raft_consensus.cc:2804] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.634873  1228 raft_consensus.cc:697] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:56.638110  1228 consensus_queue.cc:237] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.629779  1242 raft_consensus.cc:359] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.644002  1242 raft_consensus.cc:385] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.644416  1242 raft_consensus.cc:740] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.645257  1242 consensus_queue.cc:260] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.647897  1242 ts_tablet_manager.cc:1434] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.006s
I20260709 13:10:56.648314  1243 tablet_bootstrap.cc:654] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.652549  1241 tablet_bootstrap.cc:492] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.652884  1242 tablet_bootstrap.cc:492] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.653388  1241 ts_tablet_manager.cc:1403] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.059s	user 0.023s	sys 0.025s
I20260709 13:10:56.657442  1241 raft_consensus.cc:359] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.661814  1241 raft_consensus.cc:385] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.662060  1242 tablet_bootstrap.cc:654] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.663544  1241 raft_consensus.cc:740] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.664588  1241 consensus_queue.cc:260] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.666343   821 catalog_manager.cc:5697] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:56.667308  1243 tablet_bootstrap.cc:492] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.668025  1243 ts_tablet_manager.cc:1403] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.000s
I20260709 13:10:56.669548  1241 ts_tablet_manager.cc:1434] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.007s
I20260709 13:10:56.671130  1241 tablet_bootstrap.cc:492] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.674542  1243 raft_consensus.cc:359] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.675549  1243 raft_consensus.cc:385] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.675910  1243 raft_consensus.cc:740] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.676852  1243 consensus_queue.cc:260] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.680481  1243 ts_tablet_manager.cc:1434] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 13:10:56.680698  1241 tablet_bootstrap.cc:654] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.681820  1243 tablet_bootstrap.cc:492] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.685819  1229 raft_consensus.cc:493] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.686671  1229 raft_consensus.cc:515] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.691082   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a69418a2e24142b8f493f95b791fa6" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.691905  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a69418a2e24142b8f493f95b791fa6" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:56.693356  1071 raft_consensus.cc:2468] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:56.692159   999 raft_consensus.cc:2468] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:56.695863   890 leader_election.cc:304] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:56.697870  1229 leader_election.cc:290] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:56.698922  1229 raft_consensus.cc:493] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.700989  1228 raft_consensus.cc:493] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.701994  1228 raft_consensus.cc:515] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.705775  1228 leader_election.cc:290] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.699689  1229 raft_consensus.cc:515] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.711014  1229 leader_election.cc:290] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:56.711582   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.713035   999 raft_consensus.cc:2468] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:56.715654  1036 leader_election.cc:304] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:56.716972  1228 raft_consensus.cc:2804] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.717599  1228 raft_consensus.cc:493] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.718458  1228 raft_consensus.cc:3060] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.717468   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:56.719519   921 raft_consensus.cc:2468] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:56.736717   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95e186d7cc54f54b94d75c75caf2559" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.737879   999 raft_consensus.cc:2468] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:56.740563   887 leader_election.cc:304] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:56.711918  1229 raft_consensus.cc:2804] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.742303  1229 raft_consensus.cc:493] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.744618  1265 raft_consensus.cc:2804] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.745203  1265 raft_consensus.cc:493] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.745537  1265 raft_consensus.cc:3060] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.742866  1229 raft_consensus.cc:3060] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:56.757877   890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.66.131:39805, user_credentials={real_user=slave}} blocked reactor thread for 62187.5us
I20260709 13:10:56.760712  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95e186d7cc54f54b94d75c75caf2559" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:56.761755  1071 raft_consensus.cc:2468] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:56.767416  1228 raft_consensus.cc:515] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.768743  1229 raft_consensus.cc:515] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.771112  1241 tablet_bootstrap.cc:492] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.771329  1229 leader_election.cc:290] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:56.771983  1241 ts_tablet_manager.cc:1403] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.101s	user 0.028s	sys 0.065s
I20260709 13:10:56.771916   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:56.774816  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a69418a2e24142b8f493f95b791fa6" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:56.775372   921 raft_consensus.cc:3060] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.775699  1071 raft_consensus.cc:3060] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.776211  1241 raft_consensus.cc:359] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.776597  1265 raft_consensus.cc:515] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.777148  1241 raft_consensus.cc:385] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.777606  1241 raft_consensus.cc:740] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.778623  1241 consensus_queue.cc:260] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.779246  1265 leader_election.cc:290] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:56.774737   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a69418a2e24142b8f493f95b791fa6" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:56.780427   999 raft_consensus.cc:3060] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.780325   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95e186d7cc54f54b94d75c75caf2559" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:56.781735   997 raft_consensus.cc:3060] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.782320  1241 ts_tablet_manager.cc:1434] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:56.783754  1241 tablet_bootstrap.cc:492] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.784855  1242 tablet_bootstrap.cc:492] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.785521  1242 ts_tablet_manager.cc:1403] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.133s	user 0.016s	sys 0.009s
I20260709 13:10:56.789319  1242 raft_consensus.cc:359] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.790184  1242 raft_consensus.cc:385] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.790513  1242 raft_consensus.cc:740] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.791361  1242 consensus_queue.cc:260] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.791965  1241 tablet_bootstrap.cc:654] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.794356  1242 ts_tablet_manager.cc:1434] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:56.794802  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95e186d7cc54f54b94d75c75caf2559" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:56.795570  1242 tablet_bootstrap.cc:492] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.795605  1073 raft_consensus.cc:3060] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.806769  1242 tablet_bootstrap.cc:654] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.808461   999 raft_consensus.cc:2468] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:56.810158   887 leader_election.cc:304] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:56.811403  1229 raft_consensus.cc:2804] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.811936  1229 raft_consensus.cc:697] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:56.812825  1229 consensus_queue.cc:237] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.814164   921 raft_consensus.cc:2468] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:56.815435  1071 raft_consensus.cc:2468] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:56.817523  1036 leader_election.cc:304] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:56.818888  1254 raft_consensus.cc:2804] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.838822  1254 raft_consensus.cc:697] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:56.829195  1228 leader_election.cc:290] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.822721   997 raft_consensus.cc:2468] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:56.845445  1254 consensus_queue.cc:237] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.820811  1073 raft_consensus.cc:2468] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:56.846720  1241 tablet_bootstrap.cc:492] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.841437   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:56.847600   887 leader_election.cc:304] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:56.847851   999 raft_consensus.cc:3060] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.830781  1243 tablet_bootstrap.cc:654] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.848829  1265 raft_consensus.cc:2804] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.849821  1265 raft_consensus.cc:697] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:56.850478  1242 tablet_bootstrap.cc:492] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:56.851120  1242 ts_tablet_manager.cc:1403] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.017s
I20260709 13:10:56.850809  1265 consensus_queue.cc:237] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.847453  1241 ts_tablet_manager.cc:1403] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.017s
I20260709 13:10:56.855618  1242 raft_consensus.cc:359] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.856439  1242 raft_consensus.cc:385] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.856818  1242 raft_consensus.cc:740] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.858973  1242 consensus_queue.cc:260] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.860039  1241 raft_consensus.cc:359] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.861431  1241 raft_consensus.cc:385] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.861896  1241 raft_consensus.cc:740] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.863302  1241 consensus_queue.cc:260] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.864761  1242 ts_tablet_manager.cc:1434] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:10:56.868067  1243 tablet_bootstrap.cc:492] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.868408  1242 tablet_bootstrap.cc:492] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:56.869099  1243 ts_tablet_manager.cc:1403] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.188s	user 0.019s	sys 0.012s
I20260709 13:10:56.870139  1241 ts_tablet_manager.cc:1434] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 13:10:56.873368  1243 raft_consensus.cc:359] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.874675  1243 raft_consensus.cc:385] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.875249  1243 raft_consensus.cc:740] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.876257  1243 consensus_queue.cc:260] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.874842  1241 tablet_bootstrap.cc:492] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.878872  1242 tablet_bootstrap.cc:654] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.880163  1243 ts_tablet_manager.cc:1434] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:10:56.881557  1243 tablet_bootstrap.cc:492] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.887249  1241 tablet_bootstrap.cc:654] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.889667  1243 tablet_bootstrap.cc:654] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.889828  1228 raft_consensus.cc:493] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.893427  1228 raft_consensus.cc:515] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.900456  1241 tablet_bootstrap.cc:492] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:56.901234  1241 ts_tablet_manager.cc:1403] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.005s
I20260709 13:10:56.903164   819 catalog_manager.cc:5697] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:56.904690  1241 raft_consensus.cc:359] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.905618  1241 raft_consensus.cc:385] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.906013  1241 raft_consensus.cc:740] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.907029   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.907385  1241 consensus_queue.cc:260] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.908154   997 raft_consensus.cc:2468] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:56.909119  1228 leader_election.cc:290] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.911212  1241 ts_tablet_manager.cc:1434] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:10:56.911057  1036 leader_election.cc:304] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:56.912768  1241 tablet_bootstrap.cc:492] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:56.912878  1272 raft_consensus.cc:493] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.914422  1243 tablet_bootstrap.cc:492] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:56.913772  1272 raft_consensus.cc:515] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.916602  1272 leader_election.cc:290] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.917253  1272 raft_consensus.cc:2804] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.917630  1272 raft_consensus.cc:493] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.918036  1272 raft_consensus.cc:3060] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.921589  1241 tablet_bootstrap.cc:654] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.922220  1243 ts_tablet_manager.cc:1403] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.011s
I20260709 13:10:56.925207   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.923182   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:56.927081   924 raft_consensus.cc:2468] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:56.926231   997 raft_consensus.cc:2468] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
I20260709 13:10:56.930066  1036 leader_election.cc:304] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:56.937179  1228 raft_consensus.cc:493] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.936745  1243 raft_consensus.cc:359] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.938212  1243 raft_consensus.cc:385] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.937914  1228 raft_consensus.cc:515] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.938680  1243 raft_consensus.cc:740] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.939991  1243 consensus_queue.cc:260] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.941612   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:56.945940   819 catalog_manager.cc:5697] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:56.944011  1272 raft_consensus.cc:515] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.952741  1228 leader_election.cc:290] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:56.953693  1228 raft_consensus.cc:2804] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.954187  1228 raft_consensus.cc:493] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.954650  1228 raft_consensus.cc:3060] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.959785  1243 ts_tablet_manager.cc:1434] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.034s	user 0.011s	sys 0.002s
I20260709 13:10:56.961676  1243 tablet_bootstrap.cc:492] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: Bootstrap starting.
I20260709 13:10:56.965692   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:56.966764   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:56.970420   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:56.971369   997 raft_consensus.cc:3060] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.974177  1243 tablet_bootstrap.cc:654] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.975556   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:56.977643   921 raft_consensus.cc:2468] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 0.
W20260709 13:10:56.974078  1036 leader_election.cc:343] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:56.980808  1036 leader_election.cc:343] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:56.982376  1036 leader_election.cc:343] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.983330  1036 leader_election.cc:304] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:56.985222  1254 raft_consensus.cc:2804] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.985884  1254 raft_consensus.cc:493] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.988495  1254 raft_consensus.cc:3060] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.992034   997 raft_consensus.cc:2468] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:56.993489  1036 leader_election.cc:304] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:57.005884  1280 raft_consensus.cc:2804] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.007400  1280 raft_consensus.cc:697] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.020536  1243 tablet_bootstrap.cc:492] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: No bootstrap required, opened a new log
I20260709 13:10:57.021314  1243 ts_tablet_manager.cc:1403] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.007s
I20260709 13:10:57.024987  1243 raft_consensus.cc:359] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.025810  1243 raft_consensus.cc:385] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.026140  1243 raft_consensus.cc:740] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.027421  1243 consensus_queue.cc:260] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:56.952804  1272 leader_election.cc:290] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:57.008948  1280 consensus_queue.cc:237] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.028386   999 raft_consensus.cc:2468] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:57.038985  1243 ts_tablet_manager.cc:1434] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:10:57.042240  1241 tablet_bootstrap.cc:492] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:57.043048  1241 ts_tablet_manager.cc:1403] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.131s	user 0.069s	sys 0.042s
I20260709 13:10:57.042577  1228 raft_consensus.cc:515] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.046520  1242 tablet_bootstrap.cc:492] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:57.047147  1242 ts_tablet_manager.cc:1403] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.179s	user 0.040s	sys 0.088s
I20260709 13:10:57.049131   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.049948   921 raft_consensus.cc:3060] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.049607  1241 raft_consensus.cc:359] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.050135  1228 leader_election.cc:290] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:57.050431  1241 raft_consensus.cc:385] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.050908  1241 raft_consensus.cc:740] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.050743   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:57.051898   999 raft_consensus.cc:3060] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.051862  1241 consensus_queue.cc:260] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.052877  1242 raft_consensus.cc:359] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.053629  1242 raft_consensus.cc:385] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.053903  1242 raft_consensus.cc:740] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.054279  1241 ts_tablet_manager.cc:1434] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:10:57.054647  1242 consensus_queue.cc:260] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.055652  1241 tablet_bootstrap.cc:492] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: Bootstrap starting.
I20260709 13:10:57.057363  1242 ts_tablet_manager.cc:1434] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:57.058434  1242 tablet_bootstrap.cc:492] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: Bootstrap starting.
I20260709 13:10:57.063637  1241 tablet_bootstrap.cc:654] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.065246   921 raft_consensus.cc:2468] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:57.066583  1242 tablet_bootstrap.cc:654] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.067695  1036 leader_election.cc:304] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.068823  1228 raft_consensus.cc:2804] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.069461  1228 raft_consensus.cc:697] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.070765  1228 consensus_queue.cc:237] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.083664  1265 raft_consensus.cc:493] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.084493  1265 raft_consensus.cc:515] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.067744  1254 raft_consensus.cc:515] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.088640   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.089334   921 raft_consensus.cc:3060] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.103247   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
W20260709 13:10:57.107650  1036 leader_election.cc:343] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.109705  1265 leader_election.cc:290] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.110810  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:57.111972  1073 raft_consensus.cc:2468] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:57.112524   921 raft_consensus.cc:2468] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:57.113849  1254 leader_election.cc:290] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:57.114748  1254 consensus_queue.cc:1048] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.114533  1036 leader_election.cc:304] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:57.116598   890 leader_election.cc:304] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.095306   821 catalog_manager.cc:5697] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.118875  1265 raft_consensus.cc:2804] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.119704  1265 raft_consensus.cc:493] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.120460  1265 raft_consensus.cc:3060] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.109174   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:57.124930   997 raft_consensus.cc:2468] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 0.
I20260709 13:10:57.128609  1254 raft_consensus.cc:2804] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.132445  1254 raft_consensus.cc:697] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.136341  1242 tablet_bootstrap.cc:492] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: No bootstrap required, opened a new log
I20260709 13:10:57.135859  1265 raft_consensus.cc:515] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.140986  1242 ts_tablet_manager.cc:1403] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent bootstrapping tablet: real 0.083s	user 0.023s	sys 0.013s
I20260709 13:10:57.143373  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:57.144254  1073 raft_consensus.cc:3060] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.145553   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:57.146348   997 raft_consensus.cc:3060] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.146704  1242 raft_consensus.cc:359] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.147850  1242 raft_consensus.cc:385] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.148233  1242 raft_consensus.cc:740] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.149544  1242 consensus_queue.cc:260] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.153623  1242 ts_tablet_manager.cc:1434] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.003s
I20260709 13:10:57.162732  1073 raft_consensus.cc:2468] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:57.163570  1265 leader_election.cc:290] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.165519   890 leader_election.cc:304] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.167306  1265 raft_consensus.cc:2804] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.168118  1265 raft_consensus.cc:697] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:57.169113  1265 consensus_queue.cc:237] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.171581   997 raft_consensus.cc:2468] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 1.
I20260709 13:10:57.137504  1254 consensus_queue.cc:237] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.176191  1241 tablet_bootstrap.cc:492] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: No bootstrap required, opened a new log
I20260709 13:10:57.135155   999 raft_consensus.cc:2468] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 1.
I20260709 13:10:57.176877  1241 ts_tablet_manager.cc:1403] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: Time spent bootstrapping tablet: real 0.121s	user 0.038s	sys 0.075s
I20260709 13:10:57.180493  1241 raft_consensus.cc:359] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.181572  1241 raft_consensus.cc:385] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.181948  1241 raft_consensus.cc:740] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.183003  1241 consensus_queue.cc:260] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.186784  1241 ts_tablet_manager.cc:1434] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:10:57.197180  1229 consensus_queue.cc:1048] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.201961  1256 raft_consensus.cc:493] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.203001  1256 raft_consensus.cc:515] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.206941  1256 leader_election.cc:290] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.207564  1073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:57.207857   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:57.208487  1073 raft_consensus.cc:2468] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
I20260709 13:10:57.208793   924 raft_consensus.cc:2468] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 0.
I20260709 13:10:57.210487   962 leader_election.cc:304] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.211495  1256 raft_consensus.cc:2804] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.211882  1256 raft_consensus.cc:493] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.212219  1256 raft_consensus.cc:3060] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.223796  1254 consensus_queue.cc:1048] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.223843  1256 raft_consensus.cc:515] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.224920  1229 consensus_queue.cc:1048] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.215848   821 catalog_manager.cc:5697] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.226908  1256 leader_election.cc:290] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.226934   821 catalog_manager.cc:5697] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.228533  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:57.229308  1071 raft_consensus.cc:3060] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.229365   821 catalog_manager.cc:5697] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 1 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.238322   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.239194   924 raft_consensus.cc:3060] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.252525   924 raft_consensus.cc:2468] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:57.253876   962 leader_election.cc:304] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.253449  1071 raft_consensus.cc:2468] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 1.
I20260709 13:10:57.255110  1256 raft_consensus.cc:2804] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.255769  1256 raft_consensus.cc:697] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:57.256844  1256 consensus_queue.cc:237] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.269264   822 catalog_manager.cc:5697] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 1 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.282110   819 catalog_manager.cc:5697] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 1 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.295408  1254 consensus_queue.cc:1048] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.308761  1228 consensus_queue.cc:1048] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.341951  1229 raft_consensus.cc:493] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.342517  1229 raft_consensus.cc:515] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.344453  1229 leader_election.cc:290] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.345619  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:57.345425   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" is_pre_election: true
I20260709 13:10:57.346675   997 raft_consensus.cc:2393] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f082b9e30dac407eb4b01fcd8ff50941 in current term 1: Already voted for candidate 10abbd8bcc06430894c465db6ba1c1e3 in this term.
I20260709 13:10:57.347872   887 leader_election.cc:304] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f082b9e30dac407eb4b01fcd8ff50941; no voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:57.348742  1229 raft_consensus.cc:3060] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.355635  1229 raft_consensus.cc:2749] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.409440  1229 consensus_queue.cc:1048] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.415154  1229 consensus_queue.cc:1048] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.453732  1256 raft_consensus.cc:493] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.454366  1256 raft_consensus.cc:515] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.456410  1256 leader_election.cc:290] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.457437   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941" is_pre_election: true
I20260709 13:10:57.457724  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3" is_pre_election: true
I20260709 13:10:57.458436   924 raft_consensus.cc:2393] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35bb9a7c4fd74b6d80bfc578303bc288 in current term 1: Already voted for candidate 10abbd8bcc06430894c465db6ba1c1e3 in this term.
I20260709 13:10:57.459921   962 leader_election.cc:304] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:57.460669  1256 raft_consensus.cc:3060] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.467375  1256 raft_consensus.cc:2749] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.526423  1228 consensus_queue.cc:1048] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.539743  1228 consensus_queue.cc:1048] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.566376  1071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:57.566989  1071 raft_consensus.cc:606] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:57.576373  1071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0069a174783946138754fb3f83a2cff0"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:57.577081  1071 raft_consensus.cc:606] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS f082b9e30dac407eb4b01fcd8ff50941
I20260709 13:10:57.583546  1071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:57.584226  1071 raft_consensus.cc:606] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:57.590301  1071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:57.590950  1071 raft_consensus.cc:606] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:57.596894   924 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:49604
I20260709 13:10:57.597612   924 raft_consensus.cc:606] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:57.603694  1071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
mode: GRACEFUL
new_leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:40338
I20260709 13:10:57.604403  1071 raft_consensus.cc:606] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Received request to transfer leadership to TS 35bb9a7c4fd74b6d80bfc578303bc288
I20260709 13:10:57.610834   997 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
mode: GRACEFUL
new_leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:48872
I20260709 13:10:57.611546   997 raft_consensus.cc:606] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Received request to transfer leadership to TS 10abbd8bcc06430894c465db6ba1c1e3
I20260709 13:10:57.675640  1228 consensus_queue.cc:1048] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.684514  1280 raft_consensus.cc:993] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.685087  1280 raft_consensus.cc:1081] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.684867  1272 consensus_queue.cc:1048] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.687822   997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:10:57.689229   997 raft_consensus.cc:493] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.689764   997 raft_consensus.cc:3060] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.705575   997 raft_consensus.cc:515] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.706763   999 raft_consensus.cc:1240] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Rejecting Update request from peer 10abbd8bcc06430894c465db6ba1c1e3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.708005   997 leader_election.cc:290] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.709486  1272 consensus_queue.cc:1059] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.710126  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:57.710477  1272 raft_consensus.cc:3055] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.710199   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0262128210e42f3a6e000dbe2558957" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.711555   924 raft_consensus.cc:3060] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.710927  1272 raft_consensus.cc:740] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.712967  1272 consensus_queue.cc:260] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.713189  1229 consensus_queue.cc:1048] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.715165  1272 raft_consensus.cc:3060] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.723229   924 raft_consensus.cc:2468] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.729614  1256 consensus_queue.cc:1048] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.734261  1272 consensus_queue.cc:1048] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.738729  1071 raft_consensus.cc:2468] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.741897  1265 consensus_queue.cc:1048] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.747903   965 leader_election.cc:304] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:10:57.749948  1206 raft_consensus.cc:2804] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.750658  1206 raft_consensus.cc:697] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:57.751703  1206 consensus_queue.cc:237] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.760829  1228 raft_consensus.cc:993] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.761955  1280 raft_consensus.cc:1081] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.764680   999 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:10:57.765416   999 raft_consensus.cc:493] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.765880   999 raft_consensus.cc:3060] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.769284  1266 raft_consensus.cc:993] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.769903  1266 raft_consensus.cc:1081] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.771878   997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48854
I20260709 13:10:57.772503   997 raft_consensus.cc:493] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.772907   997 raft_consensus.cc:3060] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.777595  1256 raft_consensus.cc:993] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: : Instructing follower 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:10:57.781912  1206 raft_consensus.cc:1081] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Signalling peer 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:10:57.783847  1073 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:40316
I20260709 13:10:57.784560  1073 raft_consensus.cc:493] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.784981  1073 raft_consensus.cc:3060] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.784065   821 catalog_manager.cc:5697] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.787398  1228 consensus_queue.cc:1048] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.790640   999 raft_consensus.cc:515] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.793972  1073 raft_consensus.cc:515] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.796196  1073 leader_election.cc:290] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:10:57.798118  1070 raft_consensus.cc:1240] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Rejecting Update request from peer 35bb9a7c4fd74b6d80bfc578303bc288 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.802354   999 leader_election.cc:290] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.803292   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.803995   921 raft_consensus.cc:3060] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.812227   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.813028   922 raft_consensus.cc:3060] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.811968  1256 consensus_queue.cc:1059] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.816505  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b381fc146f684b719f6f6fac201a0f33" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:57.819804   921 raft_consensus.cc:2468] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.820538  1070 raft_consensus.cc:3055] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.821069  1070 raft_consensus.cc:740] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.821256   962 leader_election.cc:304] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.822114  1070 consensus_queue.cc:260] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.823563  1070 raft_consensus.cc:3060] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.827090   922 raft_consensus.cc:2468] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 2.
I20260709 13:10:57.828469   997 raft_consensus.cc:515] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.831964   998 raft_consensus.cc:1240] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Rejecting Update request from peer f082b9e30dac407eb4b01fcd8ff50941 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.833366  1310 raft_consensus.cc:2804] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.834033  1310 raft_consensus.cc:697] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:57.835074  1310 consensus_queue.cc:237] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.835659   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.836583   924 raft_consensus.cc:3055] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.837016   924 raft_consensus.cc:740] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:57.837297  1070 raft_consensus.cc:2468] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.837707   921 raft_consensus.cc:1240] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Rejecting Update request from peer 35bb9a7c4fd74b6d80bfc578303bc288 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.838142   924 consensus_queue.cc:260] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.841547   924 raft_consensus.cc:3060] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.850078  1036 leader_election.cc:304] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.851239  1228 raft_consensus.cc:2804] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.853382  1228 raft_consensus.cc:697] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.856266  1228 consensus_queue.cc:237] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.859642   997 leader_election.cc:290] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.860343  1310 consensus_queue.cc:1059] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:57.882778   965 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:39805, user_credentials={real_user=slave}} blocked reactor thread for 63767.7us
I20260709 13:10:57.888353  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b9397a82af41768cf4d00b94c76319" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:57.889191  1070 raft_consensus.cc:3060] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.894577   924 raft_consensus.cc:2468] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.898999   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5ec99df3194749ab8172588bc38bfb" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:57.902308   998 raft_consensus.cc:3055] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.902827   998 raft_consensus.cc:740] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:57.905320  1070 raft_consensus.cc:2468] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.908525   998 consensus_queue.cc:260] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.910107   998 raft_consensus.cc:3060] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.913502   921 raft_consensus.cc:1240] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Rejecting Update request from peer 35bb9a7c4fd74b6d80bfc578303bc288 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:10:57.919752   819 catalog_manager.cc:5697] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 1 to 2, leader changed from 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130) to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 2 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.923925   962 leader_election.cc:304] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.925949  1315 raft_consensus.cc:2804] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.928249   998 raft_consensus.cc:2468] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 2.
I20260709 13:10:57.929864  1272 raft_consensus.cc:993] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
I20260709 13:10:57.930531  1315 raft_consensus.cc:697] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:57.930644  1272 raft_consensus.cc:1081] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer 35bb9a7c4fd74b6d80bfc578303bc288 to start an election
W20260709 13:10:57.932042  1312 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:10:57.931682  1315 consensus_queue.cc:237] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.932771   997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:10:57.933772   997 raft_consensus.cc:493] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.934248   997 raft_consensus.cc:3060] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.942634   997 raft_consensus.cc:515] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.942755   821 catalog_manager.cc:5697] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.944465   997 leader_election.cc:290] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:57.945578   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:10:57.946339   924 raft_consensus.cc:3060] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.946389  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979a0bee5c50496996be259fd01a6f23" candidate_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:57.947180  1070 raft_consensus.cc:3055] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.947568  1070 raft_consensus.cc:740] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:57.948529  1070 consensus_queue.cc:260] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.950309  1070 raft_consensus.cc:3060] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.957434   924 raft_consensus.cc:2468] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.959162   962 leader_election.cc:304] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:57.960278  1315 raft_consensus.cc:2804] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.960811  1315 raft_consensus.cc:697] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Leader. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:10:57.961771  1315 consensus_queue.cc:237] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:57.963791  1070 raft_consensus.cc:2468] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35bb9a7c4fd74b6d80bfc578303bc288 in term 2.
I20260709 13:10:57.966813   821 catalog_manager.cc:5697] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.983114   821 catalog_manager.cc:5697] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130). New cstate: current_term: 2 leader_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.035144  1272 raft_consensus.cc:993] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:58.035683  1228 raft_consensus.cc:1081] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:58.037407   924 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0069a174783946138754fb3f83a2cff0"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:49588
I20260709 13:10:58.037914   924 raft_consensus.cc:493] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.038163   924 raft_consensus.cc:3060] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.043632   924 raft_consensus.cc:515] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:58.045377   924 leader_election.cc:290] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:58.046484  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:58.046501   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0069a174783946138754fb3f83a2cff0" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:58.047204  1070 raft_consensus.cc:3055] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.047338   997 raft_consensus.cc:3060] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.047544  1070 raft_consensus.cc:740] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:58.048414  1070 consensus_queue.cc:260] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:58.049557  1070 raft_consensus.cc:3060] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.055358   997 raft_consensus.cc:2468] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:58.055387  1070 raft_consensus.cc:2468] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:58.056983   890 leader_election.cc:304] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:58.057930  1229 raft_consensus.cc:2804] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.058495  1229 raft_consensus.cc:697] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:58.059422  1229 consensus_queue.cc:237] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:58.068382   821 catalog_manager.cc:5697] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.097519  1272 raft_consensus.cc:993] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: : Instructing follower f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:58.098112  1228 raft_consensus.cc:1081] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Signalling peer f082b9e30dac407eb4b01fcd8ff50941 to start an election
I20260709 13:10:58.099901   924 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
 from {username='slave'} at 127.0.0.1:49588
I20260709 13:10:58.100435   924 raft_consensus.cc:493] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.100697   924 raft_consensus.cc:3060] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.106523   924 raft_consensus.cc:515] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:58.108196   924 leader_election.cc:290] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Requested vote from peers 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267), 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131:39805)
I20260709 13:10:58.109611   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:10:58.109674  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b2063c07ab4aa78745f92e65497503" candidate_uuid: "f082b9e30dac407eb4b01fcd8ff50941" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
I20260709 13:10:58.110595  1070 raft_consensus.cc:3055] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.110675   997 raft_consensus.cc:3060] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.111028  1070 raft_consensus.cc:740] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:10:58.112020  1070 consensus_queue.cc:260] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:58.113682  1070 raft_consensus.cc:3060] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.120910   997 raft_consensus.cc:2468] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:58.122571   887 leader_election.cc:304] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35bb9a7c4fd74b6d80bfc578303bc288, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:10:58.122882  1070 raft_consensus.cc:2468] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f082b9e30dac407eb4b01fcd8ff50941 in term 2.
I20260709 13:10:58.123437  1229 raft_consensus.cc:2804] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.123984  1229 raft_consensus.cc:697] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Leader. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:10:58.125222  1229 consensus_queue.cc:237] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:10:58.133656   821 catalog_manager.cc:5697] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 reported cstate change: term changed from 1 to 2, leader changed from 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131) to f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129). New cstate: current_term: 2 leader_uuid: "f082b9e30dac407eb4b01fcd8ff50941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.183410  1070 raft_consensus.cc:1275] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.185123  1315 consensus_queue.cc:1048] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.195214   924 raft_consensus.cc:1275] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.196763  1315 consensus_queue.cc:1048] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:58.254249   924 raft_consensus.cc:1275] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.255870  1314 consensus_queue.cc:1048] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:58.267506  1073 raft_consensus.cc:1275] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.269376  1315 consensus_queue.cc:1048] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:58.284292   924 raft_consensus.cc:1275] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.286669  1228 consensus_queue.cc:1048] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:58.298668   997 raft_consensus.cc:1275] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.300379  1280 consensus_queue.cc:1048] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:58.413414   924 raft_consensus.cc:1275] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.414971  1315 consensus_queue.cc:1048] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.424057  1073 raft_consensus.cc:1275] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.427008  1315 consensus_queue.cc:1048] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.446318   924 raft_consensus.cc:1275] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.447754  1315 consensus_queue.cc:1048] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.457676  1073 raft_consensus.cc:1275] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer 35bb9a7c4fd74b6d80bfc578303bc288: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.460111  1315 consensus_queue.cc:1048] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.556763  1070 raft_consensus.cc:1275] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.558517  1229 consensus_queue.cc:1048] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.579912   997 raft_consensus.cc:1275] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.581651  1320 consensus_queue.cc:1048] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.611477   997 raft_consensus.cc:1275] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.612988  1321 consensus_queue.cc:1048] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.624466  1070 raft_consensus.cc:1275] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Refusing update from remote peer f082b9e30dac407eb4b01fcd8ff50941: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.626137  1321 consensus_queue.cc:1048] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.615463   822 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:11:00.639935   822 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:11:00.768886 20746 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:00.854288  1357 log.cc:927] Time spent T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:11:00.892313  1355 log.cc:927] Time spent T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:11:00.955729  1375 tablet.cc:1724] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:00.965219   887 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:38267, user_credentials={real_user=slave}} blocked reactor thread for 87335.4us
W20260709 13:11:00.979918   890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:39805, user_credentials={real_user=slave}} blocked reactor thread for 57409.8us
I20260709 13:11:00.990263  1380 tablet.cc:1724] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:00.991079  1369 log.cc:927] Time spent T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:11:00.959537  1371 tablet.cc:1724] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:01.001592  1347 log.cc:927] Time spent T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:11:01.071641  1386 tablet.cc:1724] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.010461  1373 tablet.cc:1724] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:01.089812   890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.131:39805, user_credentials={real_user=slave}} blocked reactor thread for 91288.1us
W20260709 13:11:01.091257   887 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.66.130:38267, user_credentials={real_user=slave}} blocked reactor thread for 88341.4us
I20260709 13:11:01.105875  1393 tablet.cc:1724] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.140636  1380 tablet.cc:1724] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.143775  1400 tablet.cc:1724] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.071525  1382 tablet.cc:1724] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.111493  1394 tablet.cc:1724] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:01.189370  1388 log.cc:927] Time spent T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:11:01.191604  1402 tablet.cc:1724] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:01.212458  1401 log.cc:927] Time spent T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:11:01.196780  1371 tablet.cc:1724] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.215324   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd"
dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
mode: GRACEFUL
new_leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:48882
I20260709 13:11:01.132766  1399 tablet.cc:1724] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.216766   998 raft_consensus.cc:606] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Received request to transfer leadership to TS 10abbd8bcc06430894c465db6ba1c1e3
I20260709 13:11:01.216976  1373 tablet.cc:1724] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.234547  1375 tablet.cc:1724] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.236645  1386 tablet.cc:1724] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.246984  1394 tablet.cc:1724] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.238492  1393 tablet.cc:1724] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.268868  1380 tablet.cc:1724] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.289197  1406 tablet.cc:1724] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.269686  1400 tablet.cc:1724] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.302716  1402 tablet.cc:1724] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.315776  1373 tablet.cc:1724] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.337109  1399 tablet.cc:1724] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.318006  1406 tablet.cc:1724] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.346498  1393 tablet.cc:1724] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.268983  1382 tablet.cc:1724] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:01.355350   923 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684feeef469040c1bce9199d5510e30d"
dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
mode: GRACEFUL
new_leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:49618
I20260709 13:11:01.356104   923 raft_consensus.cc:606] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Received request to transfer leadership to TS 10abbd8bcc06430894c465db6ba1c1e3
I20260709 13:11:01.358141 20746 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:01.358801 20746 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:01.382742   821 catalog_manager.cc:6242] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb: soft_deleted_table [id=8150463e5b6a4754bd46bf3c73792c81] alter complete (version 1)
I20260709 13:11:01.498322  1405 raft_consensus.cc:993] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: : Instructing follower 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:11:01.498827  1405 raft_consensus.cc:1081] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Signalling peer 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:11:01.500507  1070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:40316
I20260709 13:11:01.501116  1070 raft_consensus.cc:493] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:01.501492  1070 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.506973  1070 raft_consensus.cc:515] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:11:01.508843  1070 leader_election.cc:290] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 3 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:11:01.509744   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:11:01.509958   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b77525557d47208b1a7ee595db73dd" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:11:01.510356   923 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.510578   998 raft_consensus.cc:3055] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:01.510943   998 raft_consensus.cc:740] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 35bb9a7c4fd74b6d80bfc578303bc288, State: Running, Role: LEADER
I20260709 13:11:01.511762   998 consensus_queue.cc:260] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:11:01.513046   998 raft_consensus.cc:3060] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.516618   923 raft_consensus.cc:2468] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 3.
I20260709 13:11:01.517599   998 raft_consensus.cc:2468] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 3.
I20260709 13:11:01.518031  1036 leader_election.cc:304] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, f082b9e30dac407eb4b01fcd8ff50941; no voters: 
I20260709 13:11:01.519069  1374 raft_consensus.cc:2804] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.519646  1374 raft_consensus.cc:697] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:11:01.520454  1374 consensus_queue.cc:237] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:11:01.532958   821 catalog_manager.cc:5697] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 2 to 3, leader changed from 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130) to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:01.629454  1392 raft_consensus.cc:993] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: : Instructing follower 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:11:01.630406  1370 raft_consensus.cc:1081] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Signalling peer 10abbd8bcc06430894c465db6ba1c1e3 to start an election
I20260709 13:11:01.632179  1070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "684feeef469040c1bce9199d5510e30d"
dest_uuid: "10abbd8bcc06430894c465db6ba1c1e3"
 from {username='slave'} at 127.0.0.1:40322
I20260709 13:11:01.632736  1070 raft_consensus.cc:493] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:01.633085  1070 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.638113  1070 raft_consensus.cc:515] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:11:01.639459  1070 leader_election.cc:290] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 3 election: Requested vote from peers f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961), 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267)
I20260709 13:11:01.640508   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f082b9e30dac407eb4b01fcd8ff50941"
I20260709 13:11:01.640623   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684feeef469040c1bce9199d5510e30d" candidate_uuid: "10abbd8bcc06430894c465db6ba1c1e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35bb9a7c4fd74b6d80bfc578303bc288"
I20260709 13:11:01.641382   923 raft_consensus.cc:3055] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:01.641407   998 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.641932   923 raft_consensus.cc:740] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f082b9e30dac407eb4b01fcd8ff50941, State: Running, Role: LEADER
I20260709 13:11:01.642783   923 consensus_queue.cc:260] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:11:01.644066   923 raft_consensus.cc:3060] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.648618   998 raft_consensus.cc:2468] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 3.
I20260709 13:11:01.649979  1036 leader_election.cc:304] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10abbd8bcc06430894c465db6ba1c1e3, 35bb9a7c4fd74b6d80bfc578303bc288; no voters: 
I20260709 13:11:01.651047  1325 raft_consensus.cc:2804] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.651176   923 raft_consensus.cc:2468] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10abbd8bcc06430894c465db6ba1c1e3 in term 3.
I20260709 13:11:01.651612  1325 raft_consensus.cc:697] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 LEADER]: Becoming Leader. State: Replica: 10abbd8bcc06430894c465db6ba1c1e3, State: Running, Role: LEADER
I20260709 13:11:01.652456  1325 consensus_queue.cc:237] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } }
I20260709 13:11:01.663280   821 catalog_manager.cc:5697] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 reported cstate change: term changed from 2 to 3, leader changed from f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129) to 10abbd8bcc06430894c465db6ba1c1e3 (127.20.66.131). New cstate: current_term: 3 leader_uuid: "10abbd8bcc06430894c465db6ba1c1e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10abbd8bcc06430894c465db6ba1c1e3" member_type: VOTER last_known_addr { host: "127.20.66.131" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:01.945417   998 raft_consensus.cc:1275] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:01.948241  1374 consensus_queue.cc:1048] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:01.960929   923 raft_consensus.cc:1275] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:01.965651  1416 consensus_queue.cc:1048] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.102196   923 raft_consensus.cc:1275] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:02.104089  1416 consensus_queue.cc:1048] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f082b9e30dac407eb4b01fcd8ff50941" member_type: VOTER last_known_addr { host: "127.20.66.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.115578   996 raft_consensus.cc:1275] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Refusing update from remote peer 10abbd8bcc06430894c465db6ba1c1e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:02.117769  1416 consensus_queue.cc:1048] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35bb9a7c4fd74b6d80bfc578303bc288" member_type: VOTER last_known_addr { host: "127.20.66.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.424742 20746 tablet_server.cc:179] TabletServer@127.20.66.129:0 shutting down...
W20260709 13:11:02.428937   962 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 13:11:02.440047   962 consensus_peers.cc:597] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.472105 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:02.473171 20746 tablet_replica.cc:333] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.473985 20746 raft_consensus.cc:2243] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.474571 20746 raft_consensus.cc:2272] T 4e5ec99df3194749ab8172588bc38bfb P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.477149 20746 tablet_replica.cc:333] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.477831 20746 raft_consensus.cc:2243] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.478770 20746 raft_consensus.cc:2272] T 0069a174783946138754fb3f83a2cff0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.481217 20746 tablet_replica.cc:333] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.481870 20746 raft_consensus.cc:2243] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.482381 20746 raft_consensus.cc:2272] T 979a0bee5c50496996be259fd01a6f23 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.484591 20746 tablet_replica.cc:333] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.485340 20746 raft_consensus.cc:2243] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.485831 20746 raft_consensus.cc:2272] T b381fc146f684b719f6f6fac201a0f33 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.488194 20746 tablet_replica.cc:333] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.488876 20746 raft_consensus.cc:2243] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.489842 20746 raft_consensus.cc:2272] T c8b2063c07ab4aa78745f92e65497503 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.492141 20746 tablet_replica.cc:333] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.493206 20746 raft_consensus.cc:2243] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.494199 20746 raft_consensus.cc:2272] T 360a901b5e1947db83ce7fd9190c7ee6 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.496870 20746 tablet_replica.cc:333] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.497732 20746 raft_consensus.cc:2243] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.498769 20746 raft_consensus.cc:2272] T 304180aa5ffa4d76865a1867048b6ea1 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.501521 20746 tablet_replica.cc:333] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.502234 20746 raft_consensus.cc:2243] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.502882 20746 raft_consensus.cc:2272] T e0262128210e42f3a6e000dbe2558957 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.505366 20746 tablet_replica.cc:333] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.506053 20746 raft_consensus.cc:2243] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.506613 20746 raft_consensus.cc:2272] T a7b9397a82af41768cf4d00b94c76319 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.509058 20746 tablet_replica.cc:333] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.509801 20746 raft_consensus.cc:2243] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.510304 20746 raft_consensus.cc:2272] T 885c11b7479d48968b92acefaf22c6f4 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.512706 20746 tablet_replica.cc:333] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.513427 20746 raft_consensus.cc:2243] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.514062 20746 raft_consensus.cc:2272] T 9ffae32838b94fcab46259eec44d7b28 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.517294 20746 tablet_replica.cc:333] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.518185 20746 raft_consensus.cc:2243] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.519351 20746 raft_consensus.cc:2272] T 90a69418a2e24142b8f493f95b791fa6 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.521858 20746 tablet_replica.cc:333] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.522734 20746 raft_consensus.cc:2243] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.523268 20746 raft_consensus.cc:2272] T 684feeef469040c1bce9199d5510e30d P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.527303 20746 tablet_replica.cc:333] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.528064 20746 raft_consensus.cc:2243] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.528986 20746 raft_consensus.cc:2272] T 4f3fde5a7ea7497f9658945d8d794518 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.532371 20746 tablet_replica.cc:333] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.533195 20746 raft_consensus.cc:2243] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.534338 20746 raft_consensus.cc:2272] T f95e186d7cc54f54b94d75c75caf2559 P f082b9e30dac407eb4b01fcd8ff50941 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.537050 20746 tablet_replica.cc:333] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.537608 20746 raft_consensus.cc:2243] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.538017 20746 raft_consensus.cc:2272] T 7c489bd1e018427485257e79220aef24 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.540234 20746 tablet_replica.cc:333] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.540755 20746 raft_consensus.cc:2243] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.541218 20746 raft_consensus.cc:2272] T 810e75e22d7a44158f99e29bf93557b0 P f082b9e30dac407eb4b01fcd8ff50941 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.543614 20746 tablet_replica.cc:333] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941: stopping tablet replica
I20260709 13:11:02.544225 20746 raft_consensus.cc:2243] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.544761 20746 raft_consensus.cc:2272] T 67b77525557d47208b1a7ee595db73dd P f082b9e30dac407eb4b01fcd8ff50941 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:02.548911   962 consensus_peers.cc:597] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:02.563948   962 consensus_peers.cc:597] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.594302 20746 tablet_server.cc:196] TabletServer@127.20.66.129:0 shutdown complete.
I20260709 13:11:02.638013 20746 tablet_server.cc:179] TabletServer@127.20.66.130:0 shutting down...
W20260709 13:11:02.668823  1036 consensus_peers.cc:597] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:02.677776  1036 consensus_peers.cc:597] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.678536 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:02.679431 20746 tablet_replica.cc:333] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.680176 20746 raft_consensus.cc:2243] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.680747 20746 raft_consensus.cc:2272] T 4e5ec99df3194749ab8172588bc38bfb P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.683139 20746 tablet_replica.cc:333] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.683880 20746 raft_consensus.cc:2243] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.684813 20746 raft_consensus.cc:2272] T 979a0bee5c50496996be259fd01a6f23 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.687191 20746 tablet_replica.cc:333] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.687832 20746 raft_consensus.cc:2243] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.688751 20746 raft_consensus.cc:2272] T b381fc146f684b719f6f6fac201a0f33 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.690994 20746 tablet_replica.cc:333] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.691641 20746 raft_consensus.cc:2243] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.692137 20746 raft_consensus.cc:2272] T 0069a174783946138754fb3f83a2cff0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.694355 20746 tablet_replica.cc:333] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.694984 20746 raft_consensus.cc:2243] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.695561 20746 raft_consensus.cc:2272] T 90a69418a2e24142b8f493f95b791fa6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.697841 20746 tablet_replica.cc:333] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.698444 20746 raft_consensus.cc:2243] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.698939 20746 raft_consensus.cc:2272] T c8b2063c07ab4aa78745f92e65497503 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.701184 20746 tablet_replica.cc:333] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.701802 20746 raft_consensus.cc:2243] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.703126 20746 raft_consensus.cc:2272] T 810e75e22d7a44158f99e29bf93557b0 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.706296 20746 tablet_replica.cc:333] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.707026 20746 raft_consensus.cc:2243] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.707576 20746 raft_consensus.cc:2272] T 67b77525557d47208b1a7ee595db73dd P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.711104 20746 tablet_replica.cc:333] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.711865 20746 raft_consensus.cc:2243] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.712483 20746 raft_consensus.cc:2272] T 304180aa5ffa4d76865a1867048b6ea1 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.715036 20746 tablet_replica.cc:333] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.715876 20746 raft_consensus.cc:2243] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.716420 20746 raft_consensus.cc:2272] T 360a901b5e1947db83ce7fd9190c7ee6 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.719414 20746 tablet_replica.cc:333] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.720311 20746 raft_consensus.cc:2243] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.720927 20746 raft_consensus.cc:2272] T 4f3fde5a7ea7497f9658945d8d794518 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:02.723526  1036 consensus_peers.cc:597] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.724339 20746 tablet_replica.cc:333] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.725159 20746 raft_consensus.cc:2243] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.725734 20746 raft_consensus.cc:2272] T 684feeef469040c1bce9199d5510e30d P 35bb9a7c4fd74b6d80bfc578303bc288 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.729411 20746 tablet_replica.cc:333] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.730095 20746 raft_consensus.cc:2243] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:02.730435   962 consensus_peers.cc:597] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.730630 20746 raft_consensus.cc:2272] T f95e186d7cc54f54b94d75c75caf2559 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.773305 20746 tablet_replica.cc:333] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.776865 20746 raft_consensus.cc:2243] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:11:02.777573   962 consensus_peers.cc:597] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.779706 20746 raft_consensus.cc:2272] T 7c489bd1e018427485257e79220aef24 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.783985 20746 tablet_replica.cc:333] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.784873 20746 raft_consensus.cc:2243] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.785684 20746 raft_consensus.cc:2272] T 9ffae32838b94fcab46259eec44d7b28 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.789601 20746 tablet_replica.cc:333] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.790479 20746 raft_consensus.cc:2243] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.792614 20746 raft_consensus.cc:2272] T a7b9397a82af41768cf4d00b94c76319 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.797220 20746 tablet_replica.cc:333] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.798111 20746 raft_consensus.cc:2243] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.800169 20746 raft_consensus.cc:2272] T e0262128210e42f3a6e000dbe2558957 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.805167 20746 tablet_replica.cc:333] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288: stopping tablet replica
I20260709 13:11:02.806324 20746 raft_consensus.cc:2243] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.809268 20746 raft_consensus.cc:2272] T 885c11b7479d48968b92acefaf22c6f4 P 35bb9a7c4fd74b6d80bfc578303bc288 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:02.811519  1036 consensus_peers.cc:597] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Couldn't send request to peer 35bb9a7c4fd74b6d80bfc578303bc288. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:02.863484  1036 consensus_peers.cc:597] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer f082b9e30dac407eb4b01fcd8ff50941 (127.20.66.129:32961): Couldn't send request to peer f082b9e30dac407eb4b01fcd8ff50941. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.129:32961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.881398 20746 tablet_server.cc:196] TabletServer@127.20.66.130:0 shutdown complete.
I20260709 13:11:02.952014 20746 tablet_server.cc:179] TabletServer@127.20.66.131:0 shutting down...
W20260709 13:11:02.967679  1036 consensus_peers.cc:597] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Couldn't send request to peer 35bb9a7c4fd74b6d80bfc578303bc288. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:38267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.994807 20746 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:02.995755 20746 tablet_replica.cc:333] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:02.996702 20746 raft_consensus.cc:2243] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:02.998076 20746 raft_consensus.cc:2272] T 4e5ec99df3194749ab8172588bc38bfb P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.000748 20746 tablet_replica.cc:333] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.001519 20746 raft_consensus.cc:2243] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.002194 20746 raft_consensus.cc:2272] T 979a0bee5c50496996be259fd01a6f23 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.004786 20746 tablet_replica.cc:333] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.005473 20746 raft_consensus.cc:2243] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.006125 20746 raft_consensus.cc:2272] T b381fc146f684b719f6f6fac201a0f33 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.008544 20746 tablet_replica.cc:333] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.009183 20746 raft_consensus.cc:2243] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.009826 20746 raft_consensus.cc:2272] T 0069a174783946138754fb3f83a2cff0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.012209 20746 tablet_replica.cc:333] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.013072 20746 raft_consensus.cc:2243] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.013772 20746 raft_consensus.cc:2272] T 90a69418a2e24142b8f493f95b791fa6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.016517 20746 tablet_replica.cc:333] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.017316 20746 raft_consensus.cc:2243] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.019712 20746 raft_consensus.cc:2272] T c8b2063c07ab4aa78745f92e65497503 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.022593 20746 tablet_replica.cc:333] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.023247 20746 raft_consensus.cc:2243] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.023934 20746 raft_consensus.cc:2272] T f95e186d7cc54f54b94d75c75caf2559 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.026473 20746 tablet_replica.cc:333] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.027123 20746 raft_consensus.cc:2243] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.027722 20746 raft_consensus.cc:2272] T 7c489bd1e018427485257e79220aef24 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.030170 20746 tablet_replica.cc:333] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.030763 20746 raft_consensus.cc:2243] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.032162 20746 raft_consensus.cc:2272] T 67b77525557d47208b1a7ee595db73dd P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.035337 20746 tablet_replica.cc:333] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.036095 20746 raft_consensus.cc:2243] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.036904 20746 raft_consensus.cc:2272] T 810e75e22d7a44158f99e29bf93557b0 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.039319  1036 consensus_peers.cc:597] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 -> Peer 35bb9a7c4fd74b6d80bfc578303bc288 (127.20.66.130:38267): Couldn't send request to peer 35bb9a7c4fd74b6d80bfc578303bc288. Status: Network error: Client connection negotiation failed: client connection to 127.20.66.130:38267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.040148 20746 tablet_replica.cc:333] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.040736 20746 raft_consensus.cc:2243] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.041383 20746 raft_consensus.cc:2272] T 4f3fde5a7ea7497f9658945d8d794518 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.043725 20746 tablet_replica.cc:333] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.044251 20746 raft_consensus.cc:2243] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.045396 20746 raft_consensus.cc:2272] T 684feeef469040c1bce9199d5510e30d P 10abbd8bcc06430894c465db6ba1c1e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.048321 20746 tablet_replica.cc:333] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.048847 20746 raft_consensus.cc:2243] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.049461 20746 raft_consensus.cc:2272] T 360a901b5e1947db83ce7fd9190c7ee6 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.051681 20746 tablet_replica.cc:333] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.052153 20746 raft_consensus.cc:2243] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.052667 20746 raft_consensus.cc:2272] T 304180aa5ffa4d76865a1867048b6ea1 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.054936 20746 tablet_replica.cc:333] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.055408 20746 raft_consensus.cc:2243] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.055948 20746 raft_consensus.cc:2272] T a7b9397a82af41768cf4d00b94c76319 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.058231 20746 tablet_replica.cc:333] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.058687 20746 raft_consensus.cc:2243] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.059219 20746 raft_consensus.cc:2272] T e0262128210e42f3a6e000dbe2558957 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.061509 20746 tablet_replica.cc:333] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.061964 20746 raft_consensus.cc:2243] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.062451 20746 raft_consensus.cc:2272] T 885c11b7479d48968b92acefaf22c6f4 P 10abbd8bcc06430894c465db6ba1c1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.064685 20746 tablet_replica.cc:333] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3: stopping tablet replica
I20260709 13:11:03.065193 20746 raft_consensus.cc:2243] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.066375 20746 raft_consensus.cc:2272] T 9ffae32838b94fcab46259eec44d7b28 P 10abbd8bcc06430894c465db6ba1c1e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.111982 20746 tablet_server.cc:196] TabletServer@127.20.66.131:0 shutdown complete.
I20260709 13:11:03.157666 20746 master.cc:562] Master@127.20.66.190:46621 shutting down...
I20260709 13:11:03.184985 20746 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.185750 20746 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.186105 20746 tablet_replica.cc:333] T 00000000000000000000000000000000 P c7834b353523407b8b642f7929985bcb: stopping tablet replica
I20260709 13:11:03.207870 20746 master.cc:584] Master@127.20.66.190:46621 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12933 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (29499 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (404611 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:11:03.275426 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 426 messages since previous log ~19 seconds ago
I20260709 13:11:03.275703 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 426 messages since previous log ~19 seconds ago
I20260709 13:11:03.275880 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 424 messages since previous log ~19 seconds ago
I20260709 13:11:03.276139 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:11:03.276340 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:11:03.276615 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 3 messages since previous log ~164 seconds ago
I20260709 13:11:03.276805 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 2 messages since previous log ~292 seconds ago
I20260709 13:11:03.277100 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 13:11:03.277423 20746 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~11 seconds ago
ThreadSanitizer: reported 1 warnings
